We present a general, fast, and practical solution for interpolating novel views of diverse real-world scenes given a sparse set of nearby views. Existing generic novel view synthesis methods rely on time-consuming scene geometry pre-computation or redundant sampling of the entire space for neural volumetric rendering, limiting the overall efficiency. Instead, we incorporate learned MVS priors into the neural volume rendering pipeline while improving the rendering efficiency by reducing sampling points under the guidance of depth probability distributions. Specifically, fewer but important points are sampled under the guidance of depth probability distributions extracted from the learned MVS architecture. Based on the learned probability-guided sampling, we develop a sophisticated neural volume rendering module that effectively integrates source view information with the learned scene structures. We further propose confidence-aware refinement to improve the rendering results in uncertain, occluded, and unreferenced regions. Moreover, we build a four-view camera system for holographic display and provide a real-time version of our framework for free-viewpoint experience, where novel view images of a spatial resolution of 512×512 can be rendered at around 20 fps on a single GTX 3090 GPU. Experiments show that our method achieves 15 to 40 times faster rendering compared to state-of-the-art baselines, with strong generalization capacity and comparable high-quality novel view synthesis performance.

Today, machine vision experiences large latency due to big data processing, which is a barrier to time-critical applications. To address this issue, in-sensor computing was presented in the past. Here, we present a scheme of computing in a magnetic tunneling junction (MTJ) sensor array for proof-of-principle. Using the MTJ sensor array, the functions of artificial neural network (ANN) classifiers and autoencoders were verified. The time for correct classification of one picture was less than$9~\mu \text{s}$. The power consumed in the sensor array can be decreased according to the square law without affecting the results. Our work shows universal circuits and algorithms to compute in resistance-style ANN image sensors with promising energy efficiency.