Google’s Light Field VR Videos Now Available in a Practical Player
In late June, Google researchers showcased a new recording method for transforming traditional video and photography into a more immersive form of media: 3D videos that allow viewers to change their perspective and look around objects within a frame. The technique involved syncing together 46 cameras to create these “light field videos” as shown below: –
The technique uses a camera array which is compact for light field conditions and is capable of recording surroundings in a field of view (FOV) of over 220 degrees and which has a range of motion of 80 centimeters. Within this space, the viewer is able to move freely with the Virtual Reality headsets and view the scene from all the angles.
The individual videos from the 46 cameras in the setup are combined to generate the overall picture through the use Deepview AI algorithm. This algorithm was trained using volumetric videos from a smaller light field camera.
A peculiar feature of this algorithm is that it can automatically reduce the complexity of a 3D scene to enable the videos to be streamed. A gigabit per second should be sufficient. The file size has also been reduced without significantly compromising on the image quality. So far, the main obstacles to the advances in the light field videos have been the massive amounts of data and the complex processing required.
A Video Player for Google’s Light Field Videos
In case you are curious about Google’s light field videos and you have a fast PC along with a virtual reality headset, you can now watch them easily for free.
So I finally got around to finish up a PC build (@oculus / @SteamVR) for the #lightfield video stuff I was working on. The download is available here: https://t.co/upVfI63wSg #madewithunity @unity3d pic.twitter.com/24QqyQcSmG
— Julien Kipp (@julien_kaye) July 17, 2020
A software developer, Julien Kipp, has developed a light field video player and made it freely available on Github. You can download Google’s light field videos directly in the player for free. The developer recommends starting with the Unity video player. According to Kipp, the minimum requirement for playing these high-resolution videos is Nvidia graphics card GTX 1070 or higher.
The Light Field Videos Technique
Google’s new light field videos technique is due to be presented at the SIGGRAPH. It combines footage from the multiple cameras shooting the image simultaneously to form some kind of one big “compound eye”. The multiple viewpoints generated by the cameras are combined to form a single one where the viewer is able to move the viewpoint and the scene changes correspondingly in real time.
It generates footage that have a very high resolution and free viewpoint interactions in that the user has the freedom of movement in interacting with the scene. These “light field videos” offer users a very high degree of immersion/reality.
The current VR-enhanced videos that we are more familiar with use very ordinary stereoscopic 3D that are not free-viewpoint; that is, they don’t allow you to have a change in the viewpoint.
Facebook has a clever way of perceiving depth and adding perspective to photos but its technique is still limited and only results in a slight shift in the perspective.
The Google light field videos, on the other hand, allow users to move their heads a foot to the sides and see around the corner or to the sides of a particular object. It delivers images that are photorealistic and full motion but which have been rendered in 3D. The footage generated accurately reflects even the slightest of changes.
Due to the fact that it uses a wider rig, the parts of the scene that may be hidden from a particular perspective will be visible in other perspectives. When a viewer swings from the far-right side to the far left and the zooms in, they will see entirely new features, an experience not too dissimilar from the “enhance” scene seen in the “Blade Runner”.
You will perceive this much better in virtual reality. However, the static version of the light field videos can still be tested on the project’s website. You can also view the multiple demo light field videos if you have Chrome with the experimental web platform features enabled. Check out the site for instructions.