In this paper, we propose a novel image representation approach to tackle illumination variations in stereo matching problems. Images are mapped using their Fourier transforms which are convolved with a set of monogenic filters. Frequency analysis is carried out at different scales to account for most image content. The phase congruency and the local weighted mean phase angle are then computed over all the scales. The original image is transformed into a new representation using these two mappings. This representation is invariant to illumination and contrast variations. More importantly, it is generic and can be used with most sparse as well as dense stereo matching algorithms. In addition, sequential feature matching or tracking can also benefit from our approach in varying radiometric conditions. We demonstrate the improvements introduced with our image mappings on well-established data sets in the literature as well as on our own experimental scenarios that include high dynamic range imagery. The experiments include both dense and sparse stereo and sequential matching algorithms where the latter is considered in the very challenging visual odometry framework.

In this paper, we investigate the problem of visual odometry for ground vehicles based on the simultaneous utilization of multispectral cameras. It encompasses a stereo rig composed of an optical (visible) and thermal sensors. The novelty resides in the localization of the cameras as a stereo setup rather than two monocular cameras of different spectrums. To the best of our knowledge, this is the first time such task is attempted. Log-Gabor wavelets at different orientations and scales are used to extract interest points from both images. These are then described using a combination of frequency and spatial information within the local neighborhood. Matches between the pairs of multimodal images are computed using the cosine similarity function based on the descriptors. Pyramidal Lucas-Kanade tracker is also introduced to tackle temporal feature matching within challenging sequences of the data sets. The vehicle egomotion is computed from the triangulated 3-D points corresponding to the matched features. A windowed version of bundle adjustment incorporating Gauss-Newton optimization is utilized for motion estimation. An outlier removal scheme is also included within the framework to deal with outliers. Multispectral data sets were generated and used as test bed. They correspond to real outdoor scenarios captured using our multimodal setup. Finally, detailed results validating the proposed strategy are illustrated.

Recent developments in the infrared industry and the availability of relatively cheaper infrared cameras attracted more attention towards the simultaneous utilization of visible and infrared images and solving the correspondence problem between them. The Microsoft Kinect sensor represents one of the most widespread examples of low cost multimodal setups. In this context, we investigate the feasibility of matching features extracted from cross-spectral stereo cameras using a sparse approach. First, a set of stable features are extracted from both images using the Scale Invariant Feature Transform (SIFT). Then, descriptors are computed from local self-similarities around the selected key points. The combination of image moments is also investigated and expected to improve the matching process. Experimental results show the performance of this approach to the multimodal correspondence challenge.