Quest Pro supports eye-tracking foveal rendering, but exactly how much does it improve performance?
If you are unfamiliar with the term, Eye Tracking Foveal Rendering (ETFR) is a technique where only the region of the screen you are currently looking at is rendered at full resolution, freeing up performance since the rest is at lower resolution. This extra performance can be used to improve graphics fidelity in applications or for higher base resolution.
You don’t notice the lower resolution at the periphery because the human eye itself can only see high resolution at the very center – the fovea. This is why you cannot read a page of text without moving your gaze. Believe it or not, this foveal area is only about 3 degrees wide.
ETFR has long been considered a “holy grail” for virtual reality, because if your GPU really only had to render 3 degrees of your field of view at full resolution, the performance gain could be on the order of 20x. This would allow for ultra-high resolution displays or incredibly detailed graphics. But in reality, achieving this would require perfect eye tracking with zero latency, an absurdly high display refresh rate and a high quality reconstruction algorithm so you don’t notice the flickering and flickering.
Quest Pro is Meta’s first expedition helmet with eye tracking. The end-to-end latency of this first-generation eye-tracking technology is on the order of 50 milliseconds, and the screen refresh rate maxes out at 90Hz. foveal are far from 20x.
Meta headsets supported Fixed Render Fove (FFR) – rendering the edges of the lens in lower resolution – since Oculus Go six years ago. In a conversation this week given to developers, Meta detailed the exact performance benefits of eye tracking Render Fove (ETFR) and compared it to the FFR.
Both types of foveal rendering are enabled by developers on a per-app basis (although obviously both cannot be used at the same time). Developers have three choices for edge resolution reduction: Level 1, Level 2, and Level 3. With ETFR Level 1, the edge is rendered with 4 times fewer pixels, while at Level 3 most of the time with 16 times fewer pixels.
The exact performance benefit of foveal rendering also depends on the base resolution of the application. The higher the resolution, the greater the savings.
In Meta’s benchmark application, they found that at the default resolution, the FFR saves between 26% and 36% performance depending on the level of foveation, while the new ETFR saves between 33% and 45%.
But at 1.5 times the default resolution, the savings were greater, with an FFR of 34% to 43% and an ETFR of 36% to 52%. This is a boost of up to 2x compared to no foveation at all – but only a small advantage over FFR.
Of course, what really matters is what we don’t yet know: how perceptible are each of these ETFR levels? And how does that compare to how noticeable the FFR is? This is what needs to be compared – not a given level of FFR to the same level of ETFR. It’s something we’ll be testing in detail for our Quest Pro review.
On quest 2, level 1 of FFR is not noticeable at all, but level 3 definitely is. And from Quest Pro has sharper lenses both in the center and at the edges, the FFR can be more visible than ever, making the ETFR even more advantageous.
On PlayStation VR2, the claimed performance benefit of foveal rendering is greater. Sony claims its FFR saves about 60%, while its ETFR saves about 72%. This is likely due to the very different GPU architectures of console and PC GPUs compared to mobile GPUs, as well as the higher resolution. It could also be due to differences in eye tracking technology – Meta is in-house while Sony use Tobii.
Eye Tracking on Quest Pro and PlayStation VR2 is optional for privacy reasons. But disabling it will also disable ETFR, so applications will have to fall back on FFR.