Improving VR with NVIDIA’s Foveated Rendering

>And that power could be invested into more convincing graphics and lower latency. Fortunately, that problem is pretty close to being solved on a consumer level. A foveated rendering addon module for the Vive is expected to be shipping soon in China, by Q3 in North America [0]. For anyone unfamiliar with the term, foveated rendering means only rendering high-resolution images in your field of view. Basically, you don't see full detail across your entire field of view - only at the point where your eyes are focused. Everything outside that point can be rendered in lower resolution and you'd never notice the difference. It requires high-speed eye tracking to pull off, but it can greatly reduce the required horsepower to render a convincing VR environment [1]. [0] - [link] [1] - [this video]

Then you render with multiple cameras rotated around their centre, using the same techniques as cubemap rendering. With high field of view, rendering a single spherical view with homogeneous resolution would be inefficient for VR, not only due to the lack of hardware support (without noticeable artefacts or some subdivision scheme) and the created distortion, but also due to the foveated nature of human vision. You'd want a small view frustum rendered at a much higher pixel per angle blended with a lower resolution rendering of the entire field of view (optimally with a medium resolution view frustum surrounding the smaller one for a more progressive blending). See: [this video] With current GPUs and game engines, you need multiple cameras to achieve such effect, but it has the advantage of drastically lowering the computing demands for high-resolution VR graphics (compare 8k per eye, vs just two blended 1080p views). You could also ray trace a single view of non-homogeneous resolution that directly takes into account the distortion characteristics of the HMD lenses, but it would likely a lot less efficient than traditional rasterization with multiple blended cameras.