In this paper, we propose a feature-tracking algorithm for sea ice drift retrieval from a pair of sequential satellite synthetic aperture radar (SAR) images. The method is based on feature tracking comprising feature detection, description, and matching steps. The approach exploits the benefits of nonlinear multiscale image representations using accelerated-KAZE (A-KAZE) features, a method that detects and describes image features in an anisotropic scale space. We evaluated several state-of-the-art feature-based algorithms, including A-KAZE, Scale Invariant Feature Transform (SIFT), and a very fast feature extractor that computes binary descriptors known as Oriented FAST and Rotated BRIEF (ORB) on dual polarized Sentinel-1A C-SAR extra wide swath mode data over the Arctic. The A-KAZE approach outperforms both ORB and SIFT up to an order of magnitude in ice drift. The experimental results showed high relevance of the proposed algorithm for retrieval of ice drift at subkilometre resolution from a pair of SAR images with 100-m pixel size. From this paper, we found that feature tracking using nonlinear scale-spaces is preferable due to its high efficiency against noise with respect to image features compared with other existing feature tracking alternatives that make use of Gaussian or linear scale spaces.

In this paper we provide novel insights about the performance and design of popular pairwise tests-based local binary descriptors with the aim of answering the question: How many bits are needed for matching local binary descriptors? We use the interpretation of binary descriptors as a Locality Sensitive Hashing (LSH) scheme for approximating Kendall's tau rank distance between image patches. Based on this understanding we compare local binary descriptors in terms of the number of bits that are required to achieve a certain performance in feature-based matching problems. Furthermore, we introduce a calibration method to automatically determine a suitable number of bits required in an image matching scenario. We provide a performance analysis in image matching and structure from motion benchmarks, showing calibration results in visual odometry and object recognition problems. Our results show that excellent performance can be achieved using a small fraction of the total number of bits from the whole descriptor, speeding-up matching and reducing storage requirements.

The extreme variability in the appearance of a place across the four seasons of the year is one of the most challenging problems in life-long visual topological localization for mobile robotic systems and intelligent vehicles. Traditional solutions to this problem are based on the description of images using hand-crafted features, which have been shown to offer moderate invariance against seasonal changes. In this paper, we present a new proposal focused on automatically learned descriptors, which are processed by means of a technique recently popularized in the computer vision community: Convolutional Neural Networks (CNNs). The novelty of our approach relies on fusing the image information from multiple convolutional layers at several levels and granularities. In addition, we compress the redundant data of CNN features into a tractable number of bits for efficient and robust place recognition. The final descriptor is reduced by applying simple compression and binarization techniques for fast matching using the Hamming distance. An exhaustive experimental evaluation confirms the improved performance of our proposal (CNN-VTL) with respect to state-of-the-art methods over varied long-term datasets recorded across seasons.

Life-long visual localization is one of the most challenging topics in robotics over the last few years. The difficulty of this task is in the strong appearance changes that a place suffers due to dynamic elements, illumination, weather or seasons. In this paper, we propose a novel method (ABLE-M) to cope with the main problems of carrying out a robust visual topological localization along time. The novelty of our approach resides in the description of sequences of monocular images as binary codes, which are extracted from a global LDB descriptor and efficiently matched using FLANN for fast nearest neighbor search. Besides, an illumination invariant technique is applied. The usage of the proposed binary description and matching method provides a reduction of memory and computational costs, which is necessary for long-term performance. Our proposal is evaluated in different life-long navigation scenarios, where ABLE-M outperforms some of the main state-of-the-art algorithms, such as WI-SURF, BRIEF-Gist, FAB-MAP or SeqSLAM. Tests are presented for four public datasets where a same route is traversed at different times of day or night, along the months or across all four seasons.

We present a novel approach for place recognition and loop closure detection based on binary codes and disparity information using stereo images. Our method (ABLE-S) applies the Local Difference Binary (LDB) descriptor in a global framework to obtain a robust global image description, which is initially based on intensity and gradient pairwise comparisons. LDB has a higher descriptiveness power than other popular alternatives such as BRIEF, which only relies on intensity. In addition, we integrate disparity information into the binary descriptor (D-LDB). Disparity provides valuable information which decreases the effect of some typical problems in place recognition such as perceptual aliasing. The KITTI Odometry dataset is mainly used to test our approach due to its varied environments, challenging situations and length. Additionally, a loop closure ground-truth is introduced in this work for the KITTI Odometry benchmark with the aim of standardizing a robust evaluation methodology for comparing different previous algorithms against our method and for future benchmarking of new proposals. Attending to the presented results, our method allows a fast and more effective visual loop closure detection compared to state-of-the-art algorithms such as FAB-MAP, WI-SURF and BRIEF-Gist.

Visual loop closure detection plays a key role in navigation systems for intelligent vehicles. Nowadays, state-of-the-art algorithms are focused on unidirectional loop closures, but there are situations where they are not sufficient for identifying previously visited places. Therefore, the detection of bidirectional loop closures when a place is revisited in a different direction provides a more robust visual navigation. We propose a novel approach for identifying bidirectional loop closures on panoramic image sequences. Our proposal combines global binary descriptors and a matching strategy based on cross-correlation of sub-panoramas, which are defined as the different parts of a panorama. A set of experiments considering several binary descriptors (ORB, BRISK, FREAK, LDB) is provided, where LDB excels as the most suitable. The proposed matching proffers a reliable bidirectional loop closure detection, which is not efficiently solved in any other previous research. Our method is successfully validated and compared against FAB-MAP and BRIEF-Gist. The Ford Campus and the Oxford New College datasets are considered for evaluation.

We propose a novel and fast multiscale feature detection and description approach that exploits the benefits of nonlinear scale spaces. Previous attempts to detect and describe features in nonlinear scale spaces such as KAZE [1] and BFSIFT [6] are highly time consuming due to the computational burden of creating the nonlinear scale space. In this paper we propose to use recent numerical schemes called Fast Explicit Diffusion (FED) [3, 4] embedded in a pyramidal framework to dramatically speed-up feature detection in nonlinear scale spaces. In addition, we introduce a Modified-Local Difference Binary (M-LDB) descriptor that is highly efficient, exploits gradient information from the nonlinear scale space, is scale and rotation invariant and has low storage requirements. Our features are called Accelerated-KAZE (A-KAZE) due to the dramatic speed-up introduced by FED schemes embedded in a pyramidal framework.

Deformable 3D reconstruction from 2D images requires prior knowledge on the scene structure. Template-free methods use generic prior knowledge such as piecewise smoothness but require multiple images with significant baseline. Template-based methods require only one image but handle only one object for which they need specific prior knowledge, namely a 3D template. We here propose a novel method that alleviates the strong assumptions of both the template-free and template-based methods: our method uses multiple templates to achieve deformable 3D reconstruction from only one image and for multiple objects. It uses object recognition to automatically discover what objects are visible in the input image and to select the appropriate templates for deformable 3D reconstruction. The object database is built offline. Crucially, this database does not only contain appearance descriptors as in existing object recognition frameworks, but also material properties to facilitate deformable 3D reconstruction. We show successful experimental results with objects made of various materials such as paper, cloth and plastic.

In this paper, we introduce the concept of dense scene flow for visual SLAM applications. Traditional visual SLAM methods assume static features in the environment and that a dominant part of the scene changes only due to camera egomotion. These assumptions make traditional visual SLAM methods prone to failure in crowded real-world dynamic environments with many independently moving objects, such as the typical environments for the visually impaired. By means of a dense scene flow representation, moving objects can be detected. In this way, the visual SLAM process can be improved considerably, by not adding erroneous measurements into the estimation, yielding more consistent and improved localization and mapping results. We show large-scale visual SLAM results in challenging indoor and outdoor crowded environments with real visually impaired users. In particular, we performed experiments inside the Atocha railway station and in the city-center of Alcalá de Henares, both in Madrid, Spain. Our results show that the combination of visual SLAM and dense scene flow allows to obtain an accurate localization, improving considerably the results of traditional visual SLAM methods and GPS-based approaches.

A crucial step in many vision based applications, such as localization and structure from motion, is the data association between a large map of known 3D points and 2D features perceived by a new camera. In this paper, we propose a novel approach to predict the visibility of known 3D points with respect to a query camera in large-scale environments. In our approach, we model the visibility of each 3D point with respect to a camera pose using a memory-based learning algorithm, in which a distance metric between cameras is learned in an entirely non-parametric way. We show that by fully exploiting the geometric relationships between the 3D map and the camera poses, as well as the related appearance information, the resulting prediction is much more robust and efficient than conventional approaches. We demonstrate the performance of our algorithm on a large urban 3D model in terms of both speed and accuracy.

This paper presents the basics of Bag of visual words method, which will be used for an occupant monitoring system that integrates a small onboard camera inside vehicles. It is intended to detect passengers' faces because it is the most appealing characteristic of occupants in a vehicle. This work proposes the implementation of visual categorization by means of two classification methods (Naïve Bayes and Multi-class SVM) that build multi-category image models using the invariant descriptors (SIFT and SURF) extracted from the images under analysis. Bag of visual words approach requires training in order to cluster invariant descriptors and learn the data distribution depending on the classification algorithm. Once the model is created, the category of every test image can be determined by querying a visual dictionary like searching a word in a text dictionary. The performance of the classifiers will be evaluated doing several comparative tests and using standard multi-category image databases. Experimental results and the conclusions are presented.

Computer vision techniques applied to systems used on road maintenance, which are related either to traffic signs or to the road itself, are playing a major role in many countries because of the higher investment on public works of this kind. These systems are able to collect a wide range of information automatically and quickly, with the aim of improving road safety. In this context, the correct visibility of traffic signs and panels is vital for the safety of drivers. This paper describes an approach to the VISUAL Inspection of Signs and panEls (“VISUALISE”), which is an automatic inspection system, mounted onboard a vehicle, which performs inspection tasks at conventional driving speeds. VISUALISE allows for an improvement in the awareness of the road signaling state, supporting planning and decision making on the administration's and infrastructure operators' side. A description of the main computer vision techniques and some experimental results obtained from thousands of kilometers are presented. Finally, the conclusions of the system are described.

One of the main drawbacks of standard visual EKF-SLAM techniques is the assumption of a general camera motion model. Usually this motion model has been implemented in the literature as a constant linear and angular velocity model. Because of this, most approaches cannot deal with sudden camera movements, causing them to lose accurate camera pose and leading to a corrupted 3D scene map. In this work we propose increasing the robustness of EKF-SLAM techniques by replacing this general motion model with a visual odometry prior, which provides a real-time relative pose prior by tracking many hundreds of features from frame to frame. We perform fast pose estimation using the two-stage RANSAC-based approach from [1]: a two-point algorithm for rotation followed by a one-point algorithm for translation. Then we integrate the estimated relative pose into the prediction step of the EKF. In the measurement update step, we only incorporate a much smaller number of landmarks into the 3D map to maintain real-time operation. Incorporating the visual odometry prior in the EKF process yields better and more robust localization and mapping results when compared to the constant linear and angular velocity model case. Our experimental results, using a handheld stereo camera as the only sensor, clearly show the benefits of our method against the standard constant velocity model.

We aim to perform robust and fast vision-based localization using a pre-existing large map of the scene. A key step in localization is associating the features extracted from the image with the map elements at the current location. Although the problem of data association has greatly benefited from recent advances in appearance-based matching methods, less attention has been paid to the effective use of the geometric relations between the 3D map and the camera in the matching process. In this paper we propose to exploit the geometric relationship between the 3D map and the camera pose to determine the visibility of the features. In our approach, we model the visibility of every map feature with respect to the camera pose using a non-parametric distribution model. We learn these non-parametric distributions during the 3D reconstruction process, and develop efficient algorithms to predict the visibility of features during localization. With this approach, the matching process only uses those map features with the highest visibility score, yielding a much faster algorithm and superior localization results. We demonstrate an integrated system based on the proposed idea and highlight its potential benefits for the localization in large and cluttered environments.