Sony is Working on VR Face Tracking According to New Patent Filing
The US patent office has granted Sony a new patent on VR face tracking. This will be a crucial technology for the future of social VR and multiplayer experiences.
The new patent is titled Integration of tracked facial features for VR users in virtual reality environments. The patent describes using a single or multiple cameras on the underside or the nosegap in the virtual reality headset in order to capture the wearer’s mouth and facial expressions in real time.
The tracked data is to be subsequently represented on the virtual avatar that is shown to the other users in the online multiplayer games. For example, during the playing sessions, users will be able to see the smiles, gasps or laughs of their friends or playmates in real-time inside the shared virtual world. Should it be realized, this could be the interactive future that we are staring at. Like in Sony’s previous patents, the headset will also have eye tracking to represent the user’s gaze.
The patent has also described techniques that could be used to handle occlusion issues- where the camera is not able to see the entire face. It proposes a flex sensor that will press against the user’s face and a means of sensing the exact position of the user’s nose flap to make sure that this is not detected/perceived as constituting a part of the user’s actual face.
Facebook has also showcased a similar technology in the past. It has released videos in the past that bring into perspective the level of detail that can be captured simply by using cameras on a virtual reality headset.
At the moment, it isn’t clear whether Sony’s face tracking technology will ship in its upcoming PlayStation VR 2 that is expected to launch about the same time as its next-generation console PlayStation 5.
The tech giant may opt to research it further. Patent filings aren’t a guarantee that the product will see the light of the day. More often than not, the patented products never come to fruition at all. However, should Sony’s next VR headset incorporate these features, it will certainly take the multiplayer virtual reality experience to a new level of immersion.