In this work, we introduce a novel variational model for image restoration. In particular, we study the suitability of exploiting self-similarity of natural images in the fidelity term. Traditionally, this cue has been used for the regularization term, promoting to align similarities in the degraded image with similarities in the restored one. In contrast, our proposed non-local data-fidelity term penalizes deviations of patches after having suffered from the degradation process if they are similar in the degraded image. Experiments on super-resolution, denoising and depth filtering show the competitiveness of this new formulation with respect to traditional nonlocal regularization terms and recent learning-based methods.

Multiplane Images (MPIs) have shown to be excellent scene representations to synthesize new scene views. Indeed, MPIs are able to model challenging occlusions and reflections, and allow to render novel images in real time and with angular consistency. However, their memory footprint constitutes their major limitation. In this work, we propose a learning-based method that computes compact and adaptive MPIs. Our network promotes sparsity in the MPIs to only keep the necessary scene information. Besides, we adapt the depth sampling to the given scene to optimize the available memory and increase the synthesis quality with a restricted number of planes. Moreover, in contrast to recent work, our approach does not need individual training per scene and is able to generalize well to unseen scenarios. An extensive evaluation shows the superiority of our approach with respect to the state of the art on diverse view synthesis datasets.

Recently, learning methods have been designed to create Multiplane Images (MPIs) for view synthesis. While MPIs are extremely powerful and facilitate high quality renderings, a great amount of memory is required, making them impractical for many applications. In this paper, we propose a learning method that optimizes the available memory to render compact and adaptive MPIs. Our MPIs avoid redundant information and take into account the scene geometry to determine the depth sampling.

Local patch-based algorithms for image registration fail to accurately match points in areas not discriminative enough, mainly textureless regions. These methods normally involve a validation process and provide a non-completely dense solution. In this paper, we propose a novel refinement and completion approach for registration. The proposed model combines single image nonlocal densification with classical variational image registration. We associate a total variation regularization with a nonlocal term to provide a smooth solution leveraging the image geometry. We show experiments on public stereo and optical flow datasets to filter and densify incomplete depth maps and motion fields. Extensive comparisons against existing and state-of-the-art depth/motion fields densification approaches demonstrate the competitive performance of the introduced method. Additionally, we illustrate how our method can deal with other tasks, such as filtering and interpolation of depth maps from RGBD data and depth upsampling.

Local algorithms for stereo fail to match accurately points in areas not discriminative enough, mainly texture-less regions. We propose a method for filtering incorrect depth estimates and filling in those areas which have not been matched. The proposed model combines total variation regularization with a non local term taking advantage of image self similarity. The method can be easily generalized to deal with other tasks, as for example stereo upsampling. The method is compared with state-of-the-art stereo methods, showing competitive results on the Middlebury stereo database.

We present a novel aggregation method based on adaptive support weights for local stereo matching. In order to correctly match each block, the adaptive weight distribution should favor pixels sharing the same disparity. State of the art algorithms make this configuration depend only on spatial and color differences, identifying disparity discontinuities with color ones. Compared to these algorithms, we introduce a weight support depending on each tested disparity and not only on the image configuration around the reference pixel. For each tested disparity, we favor pixels in the block matching with smaller cost, which are supposed to be more likely to be correctly represented by this disparity. Besides, we use a multiscale strategy with invalidation criteria to reduce match ambiguity and computational time. Results on the Middlebury stereo benchmark show the performance improvement of the proposed approach in comparison with state of the art.

We propose a depth estimation method for light field images. Light field images can be considered as a collection of 2D images taken from different viewpoints arranged in a regular grid. We exploit this configuration and compute the disparity maps between specific pairs of views. This computation is carried out by a state of the art two-view stereo method providing a non dense disparity estimation. We propose a disparity interpolation method increasing the density and improving the accuracy of this initial estimate. Disparities obtained from several pairs of views are fused to obtain a unique and robust estimation. Finally, different experiments on synthetic and real images show how the proposed method outperforms state of the art results.

In this work we propose a depth estimation method for light field images. Light field images can be considered as a collection of 2D images taken from different viewpoints arranged in a regular grid. This means that disparity is the same for any pair of consecutive views, in both vertical and horizontal directions. We exploit this fact by computing disparity maps between specific pairs of views. This computation is done by a state of the art two-view stereo method which also provides a mask of incorrect matches. Then, disparity maps and masks are used to obtain a unique and robust estimation. Finally, we show the performance of the algorithm by means of different experiments.