Meta and BMW collaborated on a mixed reality research where the car’s sensors were linked to Quest Pro headsets thereby enabling virtual objects to follow the vehicle’s movement along.

Meta and BMW announced that they made some progress on the use of augmented reality and virtual reality in cars. This enables the tracking in the Quest Pro headset to work in a moving vehicle. According to the two companies, this could be used in various areas such as in the alleviation of problems arising from the interaction with virtual objects while on the move in the real world at high speed.

Meta and BMW Research
Meta and BMW Research

At the moment, when you use inside-out tracking in a moving car, you will see the virtual worlds or virtual objects drift through the cabin when the car turns or accelerates resulting in unstable content.

This happens because the VR hardware systems use inertial measurement units (IMU) and cameras in determining the position of the user. The cameras in the VR device will ‘see’ the interior of the vehicle as stationary. However, the gyroscope and accelerometer in the IMU will perceive the acceleration and steering in the vehicle. This creates a conflict that results in the drifting and shifting.

The partnership aimed at resolving this problem was first announced in 2021. Its goal was stated as the exploration of how augmented reality glasses could be integrated into vehicles in the future for passenger entertainment and for use as a driver’s aid.

Meta shared the progress the two have made in the research so far. This involved connecting the IMU of the vehicle to the Project Aria research glasses thereby feeding through the acceleration and steering of the vehicle to the glasses in real time. This allows the tracking of the device to filter out the data.

The Aria glasses don’t feature any displays so the team transferred the approach to a Quest Pro headset. The Quest Pro headset has a color-mixed reality functionality. This setup was subsequently used to test the actual anchoring of virtual objects to the car’s interior.

According to Meta, this setup enables the virtual content to remain stable even as the car accelerates, makes quick turns, or goes over road bumps.

According to the researchers, the next phase of the research will involve accessing the car’s precise 6DoF positioning system that will enable users to see world-locked virtual objects outside the windows like the markers on the landmarks along with other points of interest.

This isn’t the first experimentation with in-car headset usage. Audi’s holoride technology is an aftermarket solution for in-car virtual reality entertainment that combines virtual reality experiences with real-time car motion data to create unique, dynamic, and immersive experiences for passengers.

Rather than tackle the drifting problem, Holoride completely avoids it by disabling the positional tracking of the headset and leveraging custom content that makes use of the vehicle’s real acceleration, steering, and braking as virtual movement. As a result, the Holoride implementation will not work for the cabin-locked mixed reality experiences and the user cannot lean.

Currently, the partnership between Meta and BMW is exclusively focused on research and not on product development. There was no information on whether BMW’s current or future cars would support Quest headsets.

The head of BMW Group Technology Office USA Claus Dorrer stated that it was still too early to determine how and when this technology would be implemented for customers. He added that the two companies’ research partnership would enable them to determine what future immersive and in-vehicle mixed reality experiences would look like and help it steer the seamless integration of this system into cars.

Over the long haul, Meta regards this kind of research as having the potential to revolutionize travel, not only in cars but also in trains and planes by unlocking new forms of utility, hands-free communication, and entertainment thereby offering users more value than the instruments and screens that we commonly see in today’s vehicles.

https://virtualrealitytimes.com/wp-content/uploads/2023/05/Meta-and-BMW-Research-600x337.pnghttps://virtualrealitytimes.com/wp-content/uploads/2023/05/Meta-and-BMW-Research-150x90.pngRob GrantTechnologyMeta and BMW collaborated on a mixed reality research where the car’s sensors were linked to Quest Pro headsets thereby enabling virtual objects to follow the vehicle’s movement along. Meta and BMW announced that they made some progress on the use of augmented reality and virtual reality in cars. This...VR, Oculus Rift, and Metaverse News - Cryptocurrency, Adult, Sex, Porn, XXX