We propose a method to compute scale-invariant features in omnidirectional images. We present a formulation based on the Riemannian geometry for the definition of differential operators on non-Euclidian manifolds that adapt to the mirror and lens structures in omnidirectional imaging. These operators lead to a scale-space analysis that preserves the geometry of the visual information in omnidirectional images. We then build a novel scale-invariant feature detection framework for omnidirectional images that can be mapped on the sphere. We further present a new descriptor and feature matching solution for these omnidirectional images. The descriptor builds on the log-polar planar descriptors and adapts the descriptor computation to the specific geometry and the nonuniform sampling density of omnidirectional images. We also propose a rotation-invariant matching method that eliminates the orientation computation during the feature detection phase and thus decreases the computational complexity. Experimental results demonstrate that the new feature computation method combined with the adapted descriptors offers promising detection and matching performance, i.e., it improves on the common scale-invariant feature transform (SIFT) features computed on the unwrapped omnidirectional images, as well as spherical SIFT features. Finally, we show that the proposed framework also permits to match features between images with different native geometry.

This paper addresses the reconstruction of high-resolution omnidirectional images from multiple low-resolution images with inexact registration. When omnidirectional images from low-resolution vision sensors can be uniquely mapped on the 2-sphere, such a reconstruction can be described as a transform-domain super-resolution problem in a spherical imaging framework. We describe how several spherical images with arbitrary rotations in the SO(3) rotation group contribute to the reconstruction of a high-resolution image with help of the spherical Fourier transform (SFT). As low-resolution images might not be perfectly registered in practice, the impact of inaccurate alignment on the transform coefficients is analyzed. We then cast the joint registration and super-resolution problem as a total least-squares norm minimization problem in the SFT domain. A l(1)-regularized total least-squares problem is considered and solved efficiently by interior point methods. Experiments with synthetic and natural images show that the proposed methods lead to effective reconstruction of high-resolution images even when large registration errors exist in the low-resolution images. The quality of the reconstructed images also increases rapidly with the number of low-resolution images, which demonstrates the benefits of the proposed solution in super-resolution schemes. Finally, we highlight the benefit of the additional regularization constraint that clearly leads to reduced noise and improved reconstruction quality.

This paper presents a new approach for estimating the disparity search range in stereo video that enforces temporal consistency. Reliable search range estimation is very important since an incorrect estimate causes most stereo matching methods to get trapped in local minima or produce unstable results over time. In this work, the search range is estimated based on a disparity histogram that is generated with sparse feature matching algorithms such as SURF. To achieve more stable results over time, we further propose to enforce temporal consistency by calculating a weighted sum of temporally-neighboring histograms, where the weights are determined by the similarity of depth distribution between frames. Experimental results show that this proposed method yields accurate disparity search ranges for several challenging stereo videos and is robust to various forms of noise, scene complexity and camera configurations.

We propose a method to compute scale invariant features in omnidirectional images. We present a formulation based on Riemannian geometry for the definition of differential operators on non-Euclidian manifolds that correspond to the particular form of the mirrors in omnidirectional imaging. These operators lead to a scale-space analysis that preserves the geometry of the visual information in omnidirectional images. We eventually build novel scale-invariant omniSIFT features inspired by the planar SIFT framework. We apply our generic solution to omnidirectional images captured with parabolic mirrors. Simple descriptors that use omniSIFT characteristics offer promising performance in the case of image rotation or translation where visual features can be preserved due to the proper handling of the implicit image geometry.

We present a new descriptor and feature matching solution for omnidirectional images. The descriptor builds on the log-polar planar descriptors, but adapts to the specific geometry and non-uniform sampling density of spherical images. We further propose a rotation-invariant matching method for the proposed descriptor that is particularly interesting for mobile devices. It permits to reduce the computational complexity in the detection phase by eliminating the orientation assignment and moving it to the feature matching step. We then use a criteria based on the Kullback-Leibler divergence in order to improve the feature matching performance. Experimental results with spherical images show that the new descriptors offer promising performance and improve on SIFT descriptors computed on the sphere or on tangent planes.

In this paper, we address the problem of super-resolution from multiple low-resolution omnidirectional images with inexact registration. Such a problem is typically encountered in omnidirectional vision scenarios with reduced resolution sensors in imperfect settings. Several spherical images with arbitrary rotations in the SO(3) rotation group are used for the reconstruction of higher resolution images. We propose an l1regularized total least squares normminimization method for joint registration and reconstruction with better stabilization and denoising. Experimental results show that regularization offers a quality improvement of up to 1dB. In addition, it reduces the number of low resolution images that are necessary to reconstruct a high resolution image at a target quality.

This paper addresses the problem of super-resolution from low resolution spherical images that are not perfectly registered. Such a problem is typically encountered in omnidirectional vision scenarios with reduced resolution sensors in imperfect settings. Several spherical images with arbitrary rotations in the SO(3) rotation group are used for the reconstruction of higher resolution images. We first describe the impact of the registration error on the spherical Fourier transform coefficients. Then, we formulate the joint registration and reconstruction problem as a least squares norm minimization problem in the transform domain. Experimental results show that the proposed scheme leads to effective approximations of the high resolution images, even with large registration errors. The quality of the reconstructed images also increases rapidly with the number of low resolution images, which demonstrates the benefits of the proposed solution in super-resolution schemes.

This paper addresses the problem of dense estimation of disparities between omnidirectional images, in a spherical framework. Omnidirectional imaging certainly represents important advantages for the representation and processing of the plenoptic function in 3D scenes for applications in localization, or depth estimation for example. In this context, we propose to perform disparity estimation directly in a spherical framework, in order to avoid discrepancies due to inexact projections of omnidirectional images onto planes. We first perform rectification of the omnidirectional images in the spherical domain. Then we develop a global energy minimization algorithm based on the graph-cut algorithm, in order to perform disparity estimation on the sphere. Experimental results show that the proposed algorithm outperforms typical methods as the ones based on block matching, for both a simple synthetic scene, and complex natural scenes. The proposed method shows promising performances for dense disparity estimation and can be extended efficiently to networks of several camera sensors.