Facebook has this week given an insight into the development of its Oculus Insight tracking system for Oculus Rift S and Quest headsets. The tracking technology was developed for the standalone Quest headset and has been a key milestone for Facebook. According to Facebook lead developer Joel Hesch, the Insight technology will also be key to Facebook’s upcoming augmented reality products.

The self-sufficient Quest is Facebook’s finest piece of immersive hardware so far and also an indicator tos the path that Facebook will be taking as it moves next to AR hardware. Facebook has stated that the Oculus Insight tracking technology will also drive its future headsets as it transitions from virtual reality into augmented reality.

Oculus Quest part of the new lineup for Oculus for Business
Oculus Quest’s built-in tracking system could find its way into Augmented Reality

The Oculus Insight is a computer vision and AI-powered tracking technology used in both the Quest and Rift S headsets and it is based on in-headset cameras, motion sensors as well as infrared LEDs positioned in the controllers that do the mapping of real spaces into virtual reality and also makes the virtual reality experience fluid . While Facebook is not the only major hardware manufacturer to leverage this technology, it is now apparent that Facebook regards this technology as a major stepping-stone to its augmented reality future. Hesch has revealed that the Insight technology will form part of the tech work of Facebook’s promised augmented reality glasses.

Blending Virtual Reality With the Real World

With their passthrough and Guardian functionality, Oculus Quest headsets sometimes feel like a mix of virtual reality and actual reality. This is most apparent when the player is setting up their virtual playspace boundary and then immersing themselves into the virtual play-world. The Oculus Insight camera-based tracking is so impressive that the player will feel like they are drawing boundaries in the real word. The tracking could also be leveraged for much larger-scale experiences in future devices.

In a conversation with CNET, Hesch gave the example of the Arena Scale Tracking demo which was showcased last year during the Oculus Connect 5 conference in which players could run around a huge arena with obstacles while playing a Laser Tag-like version of the virtual reality shooter game Dead and Buried.

The demo included real world objects behind which the players could hide and these were aligned with virtual world objects, thereby seamlessly blending the real and the virtual. Hesch says this kind of experience is a harbinger of the kinds of ideas that Facebook is currently exploring where it will be possible to mix real world with virtual world content while also being able to create multi-user experiences where there will be more than a single person in the space.

Being a standalone headset, Oculus Quest would be a great headset choice for companies that create location-based experiences or VR theme park attractions such as The Void. However, these companies like to create tailor-made experiences rather than simply going with the strict title selections on SteamVR or Oculus Store. Given that these location-based VR centers are currently some of the primary drivers of virtual reality uptake experiences, they can drive hardware sales and take virtual reality to the masses. Facebook would benefit massively from such uptake but it is not clear whether it would allow its devices to be customizable for such uses. According to Hesch, that option is “not off the table” but Facebook is currently keeping its options open and casting the net wide.

There are, however, some companies such as MANUVR that have succeeded in blending the Oculus Quest platform into bigger arenas by making use of Microsoft’s Azure spatial anchors. The tracking capabilities for both Quest and Rift S have already been upgraded via a system update.

Oculus Insight Tracking System as a Gateway to an AR Headset

According to a Facebook blog post, the future of the Insight technology is in an “all-day wearable AR glasses” that will be spatially aware. This will need the running of simultaneous location and mapping (SLAM) under very difficult constraints including the reduction of latencies as well as the cutting down of power consumption to a low of 2% of what’s required for SLAM on a head-mounted display (HMD).

The SLAM Technology

The simultaneous localization and mapping or SLAM is the technology on which Oculus Insight’s inside-out tracking is underpinned. The technology utilizes computer vision CV algorithms to fuse the incoming data from multiple sensors so as to fix the position of an object within a constantly updated digital map.

The SLAM technology has also been applied in robotics, in the augmented reality camera effects on smartphones and it was also demoed in Oculus’ Santa Cruz virtual reality headset prototype in 2016. However, Oculus Insight still needed a very high level of precision as well as efficiency which required adapting the latest research on tracking as well as computer vision.

Facebook’s planned augmented reality glasses could also entail having to alter the inputs from the Quest physical controllers into some other format.

In the CNET conversation, Hesch didn’t reveal whether future Facebook augmented reality controllers could have controllers like those of Rift S and Quest. However, he did state that there was a possibility of future Oculus VR or AR recognizing the hand gestures.

With augmented reality, you could end up with inputs that are more hand-based such as those in the Magic Leap and Microsoft HoloLens platforms. Hesch says that Facebook is fully committed to exploring the full spectrum of options for input tracking. The company will be centering its efforts around bringing to fruition the input options that will make the most sense for users for the specific use-cases that they wish to tackle with the devices. The company will also be looking at input options that can create the best user experience for Oculus products.

Uses Beyond Headset

The Facebook’s lead developer Joel Hesch also stated that the spatial tracking enabled by Oculus Insight will go beyond the VR headset-oriented use-cases. According to Hesch, some of the elements of the computer vision tech are already available in Facebook’s phone-based AR. Much of the common building blocks such as Insight system are cross-platform and this is a real opportunity that Oculus can leverage its investments on as it pursues new form factors and the new products that it will introduce into the market. For Hesch, a technology such as spatial AI is one that will be fundamentally important in the future for a multiplicity of platforms.

Facebook’s videoconferencing tablet Facebook Portal is already part of the company’s AR/VR division but it is not yet using the Oculus Insight technology. Hesch is seeing a potential for overlap or collaboration when it comes to the cross-platform intersection between the wearable devices and a base station device.

We are likely going to hear more announcements on Oculus future trajectory during the Oculus Connect 6 to be held in late September. Possibly, a lot of Facebook’s bold augmented reality plans could be unveiled during the event.

https://virtualrealitytimes.com/wp-content/uploads/2019/01/Oculus-Quest-600x400.jpghttps://virtualrealitytimes.com/wp-content/uploads/2019/01/Oculus-Quest-150x90.jpgSam OchanjiAugmented RealityTechnologyTechnology DiscussionFacebook has this week given an insight into the development of its Oculus Insight tracking system for Oculus Rift S and Quest headsets. The tracking technology was developed for the standalone Quest headset and has been a key milestone for Facebook. According to Facebook lead developer Joel Hesch, the...VR, Oculus Rift, and Metaverse News - Cryptocurrency, Adult, Sex, Porn, XXX