The Quest Pro headset has an Eye Tracked Foveated Rendering feature. What are the performance improvements from this?

Eye Tracked Foveated Rendering (ETFR) refers to the rendering technique where only the section that your eyes are looking at will be rendered in full resolution with the remaining parts remaining blurred. This optimizes the performance of the display and frees up extra computing power that can be used to achieve a higher base resolution and improve the graphical fidelity of the apps.

In foveated rendering, the user does not notice the lower peripheral resolution as the human eye can only capture the higher resolution that is at the center- the fovea. Foveated rendering is the natural mechanism of the eye. For instance, when reading a page of text, we have to move our gazes as the fovea focuses on new areas. This foveal area that delivers maximum resolution is just about 3 degrees wide.

Foveated Rendering
Foveated Rendering

Foveated rendering is considered critical for the future of virtual reality. A GPU that can render just 3 degrees of your field of view in full resolution would produce a 20-fold performance boost. This would make it possible to have ultra-high-resolution displays for highly detailed graphics in your VR headset. To achieve this in VR hardware, there will be a need for perfect eye tracking with zero latency issues. This would require a very high display refresh rate along with an incredibly high-quality reconstruction algorithm that ensures you don’t notice the flickering and shimmering.

The Quest Pro headset is the first shipped by the company with an eye-tracking functionality. Quest Pro’s first-generation eye tracking tech has an end-to-end latency of about 50 milliseconds while the display has a maximum refresh rate of 90Hz. It, therefore, does not deliver the theoretical 20-fold savings in performance that would be expected in a perfect foveated rendering.

Meta’s headsets have had Fixed Foveated Rendering (FFR) to some degree where they render the edges of the lens with lower resolution. Meta issued a talk this week detailing the performance benefits that Eye Tracked Foveated Rendering (ETFR) would give its hardware and contrasted this with the FFR that is already in its previous generation of headsets.

Both of these foveated rendering capabilities are activated by developers on a per-app basis. Of course, they can’t be used simultaneously.

Developers have three options for peripheral resolution reduction: Level 1, Level 2, and Level 3. In the ETRF Level 1, the periphery of the lens is rendered with four times less pixels. In Level 3, it is rendered with 16X less pixels.

The actual performance improvements that foveated rendering provide is also based on the base resolution of the app. If that resolution is higher, there will be more performance savings.

Foveation
Foveation

Meta’s performance test app gave performance savings of between 26% and 36% at default resolution FFR based on the foveation level. The test found that Meta’s new ETFR provides performance savings of between 33% and 45%.

The performance savings were higher at 1.5X default resolution whereby FFR gave between 34% and 43% savings while ETFR gave savings of between 36% and 52%. This was 2X improvement over no foveation at all but it is still only a minuscule boost over the FFR used in Meta’s previous generations of headsets.

What is still unclear is just how noticeable these performance improvements are with the upgraded foveated rendering. This will become more apparent to users as the headset is test-driven with various apps.

It should be noted that the FFR Level 1 is currently not noticeable at all in the Quest 2 headset although the Level 3 is noticeable. The Quest Pro headset has lenses with improved sharpness at the center and edges and will thus exhibit a more noticeable FFR and the RTFR will, therefore, be more beneficial.

The PlayStation VR 2 headset has reported a much higher performance gain from foveated rendering. According to Sony, it has achieved savings of about 60% from FFR while the ETFR has delivered a performance savings of 72% in its PSVR2 headset. The PSVR2 uses a very different GPU architecture (for the consoles and PC GPUs) compared to the mobile GPUs used in the Quest Pro and other Meta headsets. The PSVR2 also has a higher resolution which should also factor into the wide variations in performance savings for the two headset brands. The differences in performance savings may also be due to differences in the headset’s eye-tracking tech. Meta uses an internal eye-tracking mechanism while Sony uses Tobii’s eye-tracking for its PSVR2 headset.

Due to privacy considerations, the Eye Tracking feature is optional in both Meta Quest Pro and PlayStation VR 2. However, when it is disabled, the ETFR is also disabled which means the apps will revert to Fixed Foveated Rendering, with its limitations.

Rob GrantTechnologyThe Quest Pro headset has an Eye Tracked Foveated Rendering feature. What are the performance improvements from this? Eye Tracked Foveated Rendering (ETFR) refers to the rendering technique where only the section that your eyes are looking at will be rendered in full resolution with the remaining parts remaining blurred....VR, Oculus Rift, and Metaverse News - Cryptocurrency, Adult, Sex, Porn, XXX