How Microsoft Creates VR/AR Video Holograms
When Microsoft gave a live demonstration of the much-anticipated Augmented Reality (AR) headset called the HoloLens last month at E3, scarce details about it has been revealed ever since. And to break the silence, Microsoft has recently revealed a video demonstration showing how holographic 3D video content is made for the HoloLens. The demo video was showing the traditional Maori war dance, along with some other acts such as martial arts, breakdancing, and weightlifting, as well as a kid destroying a high stack of cups.
To capture dynamic actions in such great detail, Microsoft created a huge TV studio in Redmond, WA. This studio has a specially built green screen room with 106 synchronised RGB and infrared cameras located around a central point, where the holographic video capture process starts by creating a silhouette of what is being recorded so that all the background can be removed before processing the various depth masks and textures needed. The cameras turn the action into a 3D point cloud, then algorithms further refine the model into tens of thousands of points, which is ultimately reduced to thousands of triangles per frame. Greater detail is put in places such as hands and faces before texture is finally added. The most highlighted advantage of this capture method is that unlike the traditional process of capturing holographic 3D video, there is no need for any special green screen suit to be worn by the performer.
The HoloLens AR goggles seamlessly mixes physical reality with impressive and lifelike holograms that users can freely interact with. The HoloLens consists of a see-through lens with built-in cameras that allows users to see and interact with holograms. While Microsoft’s live demonstrations have consisted of using entertainment software like Minecraft, the company has previously stated that HoloLens won’t be aimed at videogames first, but for the enterprise sector as a futuristic business solution.
With Microsoft pursuing developers and enterprise scenarios, being able to produce holographic content for the HoloLens with this method is futuristic and necessary for it to be ahead of the AR game.
For more information on the HoloLens, please visit the following websites:
http://www.virtualrealitytimes.com/2015/07/19/first-version-of-hololens-wont-be-game-centric-at-launch-microsoft/
http://www.virtualrealitytimes.com/2015/02/09/in-depth-is-microsoft-placing-its-bet-on-the-hololens/
http://www.virtualrealitytimes.com/2015/02/04/microsoft-hololens-everything-you-need-to-know/
