The demand for high-frame-rate rendering keeps increasing in modern displays. Existing frame generation and super-resolution techniques accelerate rendering by reducing rendering samples across space or time. However, they rely on a uniform sampling reduction strategy, which undersamples areas with complex details or dynamic shading. To address this, we propose to sparsely shade critical areas while reusing generated pixels in low-variation areas for neural extrapolation. Specifically, we introduce the Predictive Error-Flow-eXtrapolation Network (EFXNet)-an architecture that predicts extrapolation errors, estimates flows, and extrapolates frames at once. Firstly, EFXNet leverages temporal coherence to predict extrapolation error and guide the sparse shading of dynamic areas. In addition, EFXNet employs a target-grid correlation module to estimate robust optical flows from pixel correlations rather than pixel values. Finally, EFXNet uses dedicated motion representations for the historical geometric and lighting components, respectively, to extrapolate temporally stable frames. Extensive experimental results show that, compared with state-of-the-art methods, our frame extrapolation method exhibits superior visual quality and temporal stability under a low rendering budget.

Rendering realistic images in real-time on high-frame-rate display devices poses considerable challenges, even with advanced graphics cards. This stimulates a demand for frame prediction technologies to boost frame rates. The key to these algorithms is to exploit spatiotemporal coherence by warping rendered pixels with motion representations. However, existing motion estimation methods can suffer from low precision, high overhead, and incomplete support for visual effects. In this article, we present a rendered frame prediction framework with a novel motion representation, dubbed motion-guided flow (MoFlow) , aiming at overcoming the intrinsic limitations of optical flow and motion vectors and precisely capture the dynamics of intricate geometries, lighting, and translucent objects. Notably, we construct MoFlows using a recurrent feature streaming network, which specializes in learning latent motion features from multiple frames. The results of extensive experiments demonstrate that, compared to state-of-the-art methods, our method achieves superior visual quality and temporal stability with lower latency. The recurrent mechanism allows our method to predict single or multiple consecutive frames, increasing the frame rate by over 2×. The proposed approach represents a flexible pipeline to meet the demands of various graphics applications, devices, and scenarios.