Ego-motion estimation and localization in large environments are key components in any assistive technology for real-time user orientation and navigation. We consider the case where a large known environment is explored without a priori assumptions on the initial location. In particular we propose a framework that uses a single portable 3D sensor to solve the place recognition problem and continuously tracks its position even when leaving the known area or when significant changes occur in the observed environment. We cast the place recognition step as a classification problem and propose an efficient search space reduction considering only navigable areas where the user can be localized. Classification hypotheses are then discarded exploiting temporal consistency w.r.t. a relative tracker that exploits only the sensor input data. The solution uses a compact classifier whose representation scales well with the map size. After being localized, the user is continuously tracked exploiting the known environment using an efficient data structure that provides constant access time for nearest neighbor searches and that can be streamed to keep only the local region close to the last known position in memory. Robust results are achieved by performing a geometrically stable selection of points, efficiently filtering outliers and integrating the relative tracker based on previous observations. We experimentally show that such a framework provides good localization results and that it scales well with the environment map size yielding real-time performance for both place recognition and tracking.

Precise 3D mapping and 6DOF trajectory estimation using exteroceptive sensors are key problems in many fields. Real-time moving laser sensors gained popularity due to their precise depth measurements, high frame rate and large field of view. We propose an optimization framework for Simultaneous Localization And Mapping that properly models the acquisition process in a scanning-while-moving scenario. Each measurement is correctly reprojected in the map reference frame by considering a continuous time trajectory which is defined as the linear interpolation of a discrete set of control poses in SE3. The trajectory estimation is performed using the sensor readings only, i.e., no external motion measurement units are used. An efficient data structure that makes use of a hybrid sparse voxelized representation for large map management allows to perform global optimization over trajectories, resetting the accumulated drift when loops are detected. We experimentally show that such framework improves localization and mapping w.r.t. solutions that compensate the distortion effects without including them in the optimization step. Moreover, we show that the proposed map structure provides linear or constant time operations w.r.t. the map size in order to perform real time SLAM and it can handle very large maps.

We describe a method for non-invasive, accurate and efficient 3D reconstruction of occluded scenes, from a minimal number of X-ray and range scan image acquisitions. The residuals of generalised epipolar constraints (GEC) are incorporated in a highly efficient bundle adjustment minimization, to obtain maximum likelihood estimations of the X-ray image calibration parameters from correspondences between scene points, image points and apparent contours of scene objects. Furthermore, we propose a multimodal template adequate for accurate joint calibration of X-ray and range scan images. It offers crucial advantages for security applications, such as minimal scene occlusion and an agile data acquisition. Finally we describe a shape-from-silhouettes method based on the state of the art, able to reconstruct scene objects with general 3D shapes. We combine these proposals in a full system for 3D reconstruction of occluded scenes, and use it to demonstrate the practical and computational advantages of the methods herein described, with respect to previous proposals, using both synthetic and real data experiments.

Closed receptacle contents can be reconstructed without invasive approaches by coupling X-ray and 3D data acquisitions. We present a solution to the reconstruction problem that exploits 3D data acquisitions of the environment to initialize the X-ray projection calibration and to align the final reconstruction to a global frame. In particular we show that the problem can be solved, without employing calibration patterns, by precisely locating the X-ray emitters and receiver plates w.r.t. a global reference frame. We then exploit matching contours extracted from the X-ray images to calibrate the projection system and to perform the reconstruction of parametric objects. We provide experimental results to measure the reconstruction precision of the overall system using both synthetic data and real experiments.

As microscopes have a very shallow depth of field, Z-stacks (i.e. sets of images shot at different focal planes) are often acquired to fully capture a thick sample. Such stacks are viewed by users by navigating them through the mouse wheel. We propose a new technique of visualizing 3D point, line or area markers in such focus stacks, by displaying them with a depth-dependent defocus, simulating the microscope's optics; this leverages on the microscopists' ability to continuously twiddle focus, while implicitly performing a shape-from-focus reconstruction of the 3D structure of the sample. User studies confirm that the approach is effective, and can complement more traditional techniques such as color-based cues. We provide two implementations, one of which computes defocus in real time on the GPU, and examples of their application.

We present an efficient method for accurate automatic registration of two geometrically complex 3D range scans by using 3D lines. We first detect edges from the associated 2D reflectance images and collect 3D edge contours by only taking into account valid foreground points. Then we use an efficient split-and-merge line fitting algorithm to detect 3D lines. We build a fast search codebook to efficiently match the two sets of 3D lines. This is done by computing the orientation angle and distance of pairs of 3D lines in each set, both of which are invariant under rigid transformations. Finally we recover the rigid transformation between two scans using an efficient RANSAC algorithm with robust transformation estimation that exploits two sets of corresponding 3D lines. We conclude presenting experimental results that demonstrate efficiency and accuracy of our proposed method.

Line localization from a single image of a central camera is an ill-posed problem unless other constraints or apriori knowledge are exploited. Recently, it has been proved that noncentral catadioptric cameras allow space lines to be localized from a single image. In this paper we propose two novel localization algorithms. The first method exploits a pair of coplanar viewing rays to localize the space line. The second method follows a constrained non-linear minimization procedure using a suitable parametrization to represent space lines. We compare the accuracy of the proposed method w.r.t. the classical line localization algorithm and two robust variants of it. We carried out both synthetic and real experiments and evaluated the performance in localizing a set of space lines. We also propose a quality index for the viewing surfaces associated to space lines in order to better evaluate the quality of the localization. The experimental results showed the effectiveness and the accuracy of both proposed methods.

We present a novel calibration method for off-axis catadioptric cameras, i.e. standard perspective cameras placed in a generic position w.r.t. an axial-symmetric mirror of unknown shape. The proposed method estimates the intrinsic parameters of the natural perspective camera, the 3D shape of the mirror and its pose w.r.t. the camera. The peculiarity of our approach is that, unlike several other calibration methods, we do not require any cross section of the mirror to be visible in the image. Instead, we require that the catadioptric image contains at least the image of one generic space line. We then derive some constraints that, combined with the harmonic homology relating the apparent contours of the mirror, allow us to calibrate the off-axis camera. We provide experimental results both on synthetic and camera images that prove the validity of the technique.