Map-based localization and sensing are one of the key components in autonomous driving technologies, where high quality 3D map reconstruction is fundamentally utmost important. However, due to the highly dynamic and uncontrollable properties of real world environment, building a high quality 3D map is not straightforward and requires several strong assumptions. To address this challenge, we present a complete framework, which detects and extracts the moving objects from a sequence of unordered and texture-less point clouds, to build high quality static maps. To accurately detect the moving objects from data acquired with a possibly fast moving platform, we propose a novel 3D Flow Field Analysis approach in which we inspect the motion behaviour of the registered point sets. The proposed algorithm elegantly models the temporal and spatial displacement of the moving objects. Thus, both small moving objects (e.g. walking pedestrians) and large moving objects (e.g. moving trucks) can be detected effectively. Further, by incorporating the Sparse Subspace Clustering framework, we propose a Sparse Flow Clustering algorithm to group the 3D motion flows under both the constraints of motion similarity and spatial closeness. To this end, the static scene parts and the moving objects can be independently processed to achieve photo-realistic 3D reconstructions. Finally, we show that the proposed 3D Flow Field Analysis algorithm and the Sparse Flow Clustering approach are highly effective for motion detection and segmentation, as exemplified on the KITTI benchmark, and yield high quality reconstructed static-maps as well as rigidly moving objects.

Many fields of application could benefit from an accurate registration of measurements of different modalities over a known 3D model. However, aligning a 2D image to a 3D model is a challenging task and is even more complex when the two have a different modality. Most of the 2D/3D registration methods are based on either geometric or dense visual features. Both have their own advantages and their own drawbacks. We propose, in this paper, to mutually exploit the advantages of one feature type to reduce the drawbacks of the other one. For this, an hybrid registration framework has been designed to mutually align geometrical and dense visual features in order to obtain an accurate final 2D/3D alignment. We evaluate and compare the proposed registration method on real data acquired by a robot equipped with several visual sensors. The results highlights the robustness of the method and its ability to produce wide convergence domain and a high registration accuracy.

In the context of scene modelling, understanding, and landmark-based robot navigation, the knowledge of static scene parts and moving objects with their motion behaviours plays a vital role. We present a complete framework to detect and extract the moving objects to reconstruct a high quality static map. For a moving 3D camera setup, we propose a novel 3D Flow Field Analysis approach which accurately detects the moving objects using only 3D point cloud information. Further, we introduce a Sparse Flow Clustering approach to effectively and robustly group the motion flow vectors. Experiments show that the proposed Flow Field Analysis algorithm and Sparse Flow Clustering approach are highly effective for motion detection and segmentation, and yield high quality reconstructed static maps as well as rigidly moving objects of real-world scenarios.

The knowledge of the static scene parts and the moving objects in a dynamic scene plays a vital role for scene modelling, understanding, and landmark-based robot navigation. The key information for these tasks lies on semantic labels of the scene parts and the motion trajectories of the dynamic objects. In this work, we propose a method that segments the 3D feature trajectories based on their motion behaviours, and assigns them semantic labels using 2D-to-3D label transfer. These feature trajectories are constructed by using the proposed trajectory recovery algorithm which takes the loss of feature tracking into account. We introduce a complete framework for static-map and dynamic objects' reconstruction, as well as semantic scene understanding for a calibrated and moving 2D-3D camera setup. Our motion segmentation approach is faster by two orders of magnitude, while performing better than the state-of-the-art 3D motion segmentation methods, and successfully handles the previously discarded incomplete trajectory scenarios.

This paper aims to optimize the coverage of a given area from a set of views to allow a complete mosaicing. Among the investigated methods to find the best camera positions, two of them are studied, namely the Particle Swarm Optimization (PSO) and the Genetic Algorithms (GA). After having performed experiments to compare the algorithms, the hybridization of GA and PSO is investigated. To validate the proposed method, it is simulated area of irregular shapes with the cameras mounted on a Unmanned Aerial Vehicles (UAVs). V-REP is used to simulate the UAVs in an indoor environment and satellite images are used for a large outdoor area. The simulation validates the efficiency of the proposed method to find the optimal position of cameras. Then by using the images acquired it is possible to monitor the area and to compute a full mosaic of it.

Earth mover's distance is one of the most effective metric for comparing histograms in various image retrieval applications. The main drawback is its computational complexity which hinders its usage in various comparison tasks. We propose fast earth mover's distance computation by providing better initialization to the transportation simplex algorithm. The new approach enables faster EMD computation in Visual Memory (VM) compared to the state of the art methods. The new proposed strategy computes earth mover distance without compromising its accuracy.

We present a new globally optimal algorithm for self-calibrating a moving camera with constant parameters. Our method aims at estimating the Dual Absolute Quadric (DAQ) under the rank-3 and, optionally, camera centers chirality constraints. We employ the Branch-and-Prune paradigm and explore the space of only 5 parameters. Pruning in our method relies on solving Linear Matrix Inequality (LMI) feasibility and Generalized Eigenvalue (GEV) problems that solely depend upon the entries of the DAQ. These LMI and GEV problems are used to rule out branches in the search tree in which a quadric not satisfying the rank and chirality conditions on camera centers is guaranteed not to exist. The chirality LMI conditions are obtained by relying on the mild assumption that the camera undergoes a rotation of no more than 90 between consecutive views. Furthermore, our method does not rely on calculating bounds on any particular cost function and hence can virtually optimize any objective while achieving global optimality in a very competitive running-time.

We present an algorithm for reconstruction of piece-wise planar scenes from only two views and based on minimum line correspondences. We first recover camera rotation by matching vanishing points based on the methods already exist in the literature and then recover the camera translation by searching among a family of hypothesized planes passing through one line. Unlike algorithms based on line segments, the presented algorithm does not require an overlap between two line segments or more that one line correspondence across more than two views to recover the translation and achieves the goal by exploiting photometric constraints of the surface around the line. Experimental results on real images prove the functionality of the algorithm.

In this paper, we present a novel approach exploiting fluorescence imaging to estimate the shape of transparent objects. Classical inspection systems require users to coat transparent objects with some powder before measurement. Methods suggested in literature through non contact measurement do not effectively deal with the refraction problem, thus, providing inaccuracies. The proposed method handles the scanning of transparent objects without using any powder and solving the refraction problem using UV environment. A classical triangulation method based on stereovision scheme using fixed stereoscopic visible range cameras with a fixed UV (Ultra Violet) laser source is implemented. Transparent object surface irradiated by UV laser emits fluorescence i.e. a visible white light in a diffused manner. Images consisting of fluorescent points are thereafter analyzed to extract 3D information of the transparent object surface with a stereovision scheme. The object is moved for scanning different parts and the above procedure is repeated.

This paper introduces a novel 3D measurement scheme based on UV laser triangulation to ascertain the shape of transparent objects. Transparent objects are extremely difficult to scan with traditional 3D scanners because of the refraction problem observed in the visible range. Therefore, the object surface needs to be preliminary powdered before being digitized with commercial scanners. Our approach consists of using non contact measurement scheme while dealing with the refraction problem in visible environment. The object shape is computed by classical triangulation method based on stereovision constraint. The proposed acquisition system is composed of two classical visible range cameras and a UV laser source. The exploitation of the UV laser for triangulation system characterizes the novelty of the proposed approach. The fluorescence generated by the UV radiation enables to acquire 3D data of transparent surface with a classical stereovision scheme.

A non parametric method that defines a pixel neighborhood within catadioptric images is presented in this paper. It is based on an accurate modeling of the mirror shape by using polarization imaging. Unlike the most of current processing methods in the literature, this method is non-parametric and can deal with the deformation of catadioptric images. This paper demonstrates how an appropriate neighborhood can be derived from the polarization parameters by estimation of the degree of polarization and the angle of polarization which in return directly provide an adapted neighborhood of each pixel that can be used to perform image derivation, edge detection, interest point detection and namely image matching, all in the catadioptric image plane.

Images produced with catadioptric sensors contain a significant amount of radial distortions and variation in inherent scale. Direct application of conventional methods of image processing without considering the mirror anamorphosis yields erroneous issues. Examples of such applications are those based on the pixel neighborhood: one can mention image derivation, image convolution or image matching. The inefficiency of classical algorithms in catadioptric images processing leads in general to biased results. In this paper, we propose a new method to match a pair of catadioptric images taking into account their distorted topology. This method is based on an adapted window resulting from a convenient parameterization of the mirror. Experiments have been performed on image sequences acquired with a parabolic catadioptric sensor.

We propose a full automatic technique to project virtual texture on a real textureless 3D object. Our system is composed of cameras and projector and are used to determine the pose of the object in the real world with the projector as reference and then estimate the image seen by the projector if it would be a camera.

A new efficient method of calibration for catadioptric sensors is presented in this paper. It is based on an accurate reconstruction of the mirror by means of polarization imaging and it permits to relax most of the constraints related to the calibration of the catadioptric systems. No image processing and no calibration pattern are required so that the calibration can be performed in "one click". As a result, the coordinates of the projection rays and a mapping between projection rays and image pixels is obtained. Besides, two methods of triangulation have been adapted to catadioptric sensors and tested. Experiments prove the validity of the method with some preliminary results on 3D reconstruction.

The recovery of the three-dimensional structure of the environment is a pre-requisite for many tasks in mobile robotics. Unfortunately, calibration acts as a brake upon visual adaptation and robotic autonomy. We provide tools and constraints based on structured light for a complete and efficient uncalibrated Euclidean reconstruction of the environment. Experimental results achieved both on simulated and real data validate the method.