This paper proposes a practical two-stage method to enhance the depth channel of low-quality RGB-D videos. At the first stage of the proposed method, we select several key frames from an input RGB-D video and recover them with a new MRF-regularized low-rank matrix completion method. At the second stage, we introduce a novel Laplacian smoothing algorithm to smoothly expand the recovered key depth frames to other in-between frames. With associated weights extracted from color images and a depth confidence constraint, we are able to recover each in-between frame faithfully and guarantee long-range temporal consistency. Experiments show that our method can generate high-quality and spatiotemporally coherent RGB-D videos for a wide range of scene configurations and achieve the state-of-the-art performance. Several applications further validate the effectiveness of the proposed method.

In this paper, we propose a depth restoration method using a novel superpixel technique. Guided by a normal map reconstructed from the raw depth data, this technique over-segments RGB-D images into many small regions where their depth is assumed to be smooth. As the raw depth data is incomplete, we further introduce a depth confidence map to identify the regions which are more reliable. With the produced superpixels, we can restore the incomplete depth map using a per-superpixel linear regression. A multiresolution superpixel strategy is employed when some superpixels do not contain enough valid data. Experiments show that the proposed depth restoration method can effectively fill the wide gaps along depth discontinuities without blurring the object boundaries and the depth discontinuities.

Realistic Rendering of thin transparent layers bounded by rough surfaces involves substantial expense of computation time to account for multiple internal reflections. Resorting to Monte Carlo rendering for such material is usually impractical since recursive importance sampling is inevitable. To reduce the burden of sampling for simulating subsurface scattering and hence improve rendering performance, we adapt the microfacet model to the material with a single thin layer by introducing the extended normal distribution function (ENDF), a new representation of this model, to express visually perceived roughness due to multiple bounces of reflections and refractions. With such a representation, both surface reflection and subsurface scattering can be treated in the same microfacet framework, and the sampling process can be reduced to only once for each bounce of scattering. We derive analytical expressions of the ENDF for several cases using joint spherical warping. We also show how to choose proper shadowing-masking and Fresnel terms to make the proposed bidirectional scattering distribution function (BSDF) model energy-conserving. Experiments demonstrate that our model can be easily incorporated into a Monte Carlo path tracer with little extra computational and storage overhead, enabling some real-time applications.