In this article, we present a data-driven approach to obtain the static image of a scene, eliminating dynamic objects that might have been present at the time of traversing the scene with a camera. The general objective is to improve vision-based localization and mapping tasks in dynamic environments, where the presence (or absence) of different dynamic objects in different moments makes these tasks less robust. We introduce an end-to-end deep learning framework to turn images of an urban environment that include dynamic content, such as vehicles or pedestrians, into realistic static frames suitable for localization and mapping. This objective faces two main challenges: detecting the dynamic objects, and inpainting the static occluded background. The first challenge is addressed by the use of a convolutional network that learns a multiclass semantic segmentation of the image. The second challenge is approached with a generative adversarial model that, taking as input the original dynamic image and the computed dynamic/static binary mask, is capable of generating the final static image. This framework makes use of two new losses, one based on image steganalysis techniques, useful to improve the inpainting quality, and another one based on ORB features, designed to enhance feature matching between real and hallucinated image regions. To validate our approach, we perform an extensive evaluation on different tasks that are affected by dynamic entities, i.e.,visual odometry, place recognition, and multiview stereo, with the hallucinated images. Code has been made available on https://github.com/bertabescos/EmptyCities_SLAM.

In this paper we present an end-to-end deep learning framework to turn images that show dynamic content, such as vehicles or pedestrians, into realistic static frames. This objective encounters two main challenges: detecting all the dynamic objects, and inpainting the static occluded background with plausible imagery. The former challenge is addressed by the use of a convolutional network that learns a multiclass semantic segmentation of the image. The second problem is approached with a conditional generative adversarial model that, taking as input the original dynamic image and its dynamic/static binary mask, is capable of generating the final static image. These generated images can be used for applications such as augmented reality or vision-based robot localization purposes. To validate our approach, we show both qualitative and quantitative comparisons against other state-of-the-art inpainting methods by removing the dynamic objects and hallucinating the static structure behind them. Furthermore, to demonstrate the potential of our results, we carry out pilot experiments that show the benefits of our proposal for visual place recognition.

The purpose of this work is to highlight the paramount importance of representing and quantifying uncertainty to correctly report the associated confidence of the robot's location estimate at each time step along its trajectory and therefore decide the correct course of action in an active SLAM mission. We analyze the monotonicity property of different decision-making criteria, both in 2-D and 3-D, with respect to the representation of uncertainty and of the orientation of the robot's pose. Monotonicity, the property that uncertainty increases as the robot moves, is essential for adequate decision making. We analytically show that, by using differential representations to propagate spatial uncertainties, monotonicity is preserved for all optimality criteria, A-opt, D-opt, and E-opt, and for Shannon's entropy. We also show that monotonicity does not hold for any criteria in absolute representations using Roll-Pitch-Yaw and Euler angles. Finally, using unit quaternions in absolute representations, the only criteria that preserve monotonicity are D-opt and Shannon's entropy.

We propose an alternative to the common approaches to the topological segmentation in structured or unstructured environments, Contour-Based Segmentation. It is faster and equally accurate, without the need of fine tuning parameters or heuristics. During robotic exploration, we propose an incremental version that reduces the processing time by reusing the previous segmentation. Tests demonstrate the velocity and quality in room segmentation in both batch and incremental mode. Tests also demonstrate the incremental version outperforms the state of the art in incremental topological segmentation.

In this work we address the problem of trajectory planning in Graph SLAM. We propose the use of Expected Value of the Final Uncertainty, which summarizes all the possible uncertainties that should be considered. In fully explored environments, this is used to determine the most reliable path to the final position. In partially explored environments, this criteria quantifies the reliability of the path planned in the free space. Tests demonstrate its ability to avoid unreliable paths in fully explored environments as compared to other uncertainty based criteria. In the exploration scenario, potential paths not present in the original graph are proposed using Voronoi Diagram of the space ahead observed by the sensors. A cost function is proposed considering the length of the path as well as the expected final uncertainty, thus including potentially shorter, but still reliable paths. Tests demonstrate the shortest path is preferred as long as it contains loop closures with low uncertainty.

Simultaneous localization and mapping (SLAM) consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications and witnessing a steady transition of this technology to industry. We survey the current state of SLAM and consider future directions. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

In this paper we investigate the monotonicity of various optimality criteria during the exploration phase of an active SLAM algorithm. Optimality criteria such as A-opt, D-opt or E-opt are used in active SLAM to account for uncertainty in the map or the robot's pose, and these criteria are usually part of utility functions which help active SLAM algorithms decide where the robot should move next. The monotonicity of the optimality criteria is of utmost importance. During the exploration phase, i.e. when the robot is traversing new territory or cannot perform a loop closure, the most common way of estimating the pose of the robot is through dead-reckoning. Correctly accounting for the uncertainty is important for an active SLAM algorithm and in particular for a dead-reckoning scenario, where by definition the uncertainty in the robot's pose grows. If monotonicity does not hold in this scenario, active SLAM algorithms can execute actions under the false belief that the uncertainty has reduced. We show analytically and experimentally some conditions in which the A-opt and E-opt criteria lose monotonicity in a dead-reckoning scenario, where the propagation of the robot's pose is done using a linearized framework. We also show analytically and experimentally that under the same conditions the D-opt does not lose monotonicity and, in general for the linearized framework under consideration, D-opt does not break monotonicity.

Place categorization addresses the problem of determining the semantic label of the current position of a robot, given a snapshot of the environment as well as previously labeled information about different places that the robot has already seen. State-of-the-art approaches use machine learning techniques that require extensive and often time consuming training. This work proposes a novel formulation by posing place categorization as an efficient ℓ1-minimization problem, leading to both a faster training phase and to performance comparable to state-of-the-art methods. The formulation allows online robot operation particularly in the case when the training phase has to be learned on-the-fly and in an active manner. To validate the performance of the proposed method, extensive experimental results carried out on real data under different lighting conditions as well as structural changes in the environment are provided.

In this work, we provide an in-depth analysis of several recent robust Simultaneous Localization And Mapping (SLAM) back-end techniques that aim to recover the correct graph estimate in the presence of outliers in loop closure constraints. We present a benchmark dataset for evaluation of such methods by augmenting the KITTI Vision Benchmark with ground truth as well as generated loop closure hypotheses and present a detailed analysis of recently proposed robust SLAM methods using this benchmark. We also look into how these methods achieve the desired robustness and what are the implications for the SLAM problem. We discuss the issues involved in using the output of these robust back-ends for tasks such as path planning and how they can be addressed. The problem of robustness needs to be addressed adequately in order to have a complete and reliable solution to the SLAM problem.

This paper addresses the problem of path planning considering uncertainty criteria over the belief space. Specifically, we propose a path planning algorithm that uses a novel determinant-based measure of uncertainty and a reduced representation of the environment, in order to obtain the minimum uncertainty path from a roadmap. Our proposal does not require a priori knowledge of the environment due to the construction of the roadmap via a graph-based SLAM algorithm. We report experimental results of our proposal in four datasets that show its feasibility to obtain the minimum uncertainty path towards an autonomous navigation framework and we also show an improvement in the computation time with respect to the state of the art.

The ability to reconsider information over time allows to detect failures and is crucial for long term robust autonomous robot applications. This applies to loop closure decisions in localization and mapping systems. This paper describes a method to analyze all available information up to date in order to robustly remove past incorrect loop closures from the optimization process. The main novelties of our algorithm are: 1. incrementally reconsidering loop closures and 2. handling multi-session, spatially related or unrelated experiments. We validate our proposal in real multi-session experiments showing better results than those obtained by state of the art methods.

We propose a place recognition algorithm for simultaneous localization and mapping (SLAM) systems using stereo cameras that considers both appearance and geometric information of points of interest in the images. Both near and far scene points provide information for the recognition process. Hypotheses about loop closings are generated using a fast appearance-only technique based on the bag-of-words (BoW) method. We propose several important improvements to BoWs that profit from the fact that, in this problem, images are provided in sequence. Loop closing candidates are evaluated using a novel normalized similarity score that measures similarity in the context of recent images in the sequence. In cases where similarity is not sufficiently clear, loop closing verification is carried out using a method based on conditional random fields (CRFs). We build on CRF matching with two main novelties: We use both image and 3-D geometric information, and we carry out inference on a minimum spanning tree (MST), instead of a densely connected graph. Our results show that MSTs provide an adequate representation of the problem, with the additional advantages that exact inference is possible and that the computational cost of the inference process is limited. We compare our system with the state of the art using visual indoor and outdoor data from three different locations and show that our system can attain at least full precision (no false positives) for a higher recall (fewer false negatives).

Place recognition is a challenging task in any SLAM system. Algorithms based on visual appearance are becoming popular to detect locations already visited, also known as loop closures, because cameras are easily available and provide rich scene detail. These algorithms typically result in pairs of images considered depicting the same location. To avoid mismatches, most of them rely on epipolar geometry to check spatial consistency. In this paper we present an alternative system that makes use of stereo vision and combines two complementary techniques: bag-of-words to detect loop closing candidate images, and conditional random fields to discard those which are not geometrically consistent. We evaluate this system in public indoor and outdoor datasets from the Rawseeds project, with hundred-metre long trajectories. Our system achieves more robust results than using spatial consistency based on epipolar geometry.

In this paper we describe an algorithm to compute cycle constraints that can be used in many graph-based SLAM algorithms; we exemplify it in hierarchical SLAM. Our algorithm incrementally computes the minimum cycle basis of constraints from which any other cycle can be derived. Cycles in this basis are local and of minimum length, so that the associated cycle constraints have less linearization problems. This also permits to construct regional maps, that is, it makes possible efficient and accurate intermediate mapping levels between local maps and the whole global map. We have extended our algorithm to the multi-robot case. We have tested our methodology using the Victoria Park data set with satisfactory results.

In this paper we show that SLAM can be executed in as low as O(log n) per step. Our algorithm, the Combined Filter SLAM, uses a combination of Extended Kalman and Extended Information filters in such a way that the total cost of building a map can be reduced to O(n log n), as compared with O(n3) for standard EKF SLAM, and O(n2) for Divide and Conquer (D&C) SLAM and the Sparse Local Submap Joining Filter (SLSJF). We discuss the computational improvements that have been proposed for Kalman and Information filters, discuss the advantages and limitations of each, and how a judicious combination results in the possibility of reducing the computational cost per step to O(log n).We use simulations and real datasets to show the advantages of the proposed algorithm.

In this paper we present a loop closure method for a handheld single-camera SLAM system based on our previous work on relocalization. By finding correspondences between the current image and the map, our system is able to reliably detect loop closures. We compare our algorithm to existing techniques for loop closure in single-camera SLAM based on both image-to-image and map-to-map correspondences and discuss both the reliability and suitability of each algorithm in the context of monocular SLAM.

In this paper, we describe a system that can carry out simultaneous localization and mapping (SLAM) in large indoor and outdoor environments using a stereo pair moving with 6 DOF as the only sensor. Unlike current visual SLAM systems that use either bearing-only monocular information or 3-D stereo information, our system accommodates both monocular and stereo. Textured point features are extracted from the images and stored as 3-D points if seen in both images with sufficient disparity, or stored as inverse depth points otherwise. This allows the system to map both near and far features: the first provide distance and orientation, and the second provide orientation information. Unlike other vision-only SLAM systems, stereo does not suffer from “scale drift” because of unobservability problems, and thus, no other information such as gyroscopes or accelerometers is required in our system. Our SLAM algorithm generates sequences of conditionally independent local maps that can share information related to the camera motion and common features being tracked. The system computes the full map using the novel conditionally independent divide and conquer algorithm, which allows constant time operation most of the time, with linear time updates to compute the full map. To demonstrate the robustness and scalability of our system, we show experimental results in indoor andoutdoor urban environments of 210 m and 140 m loop trajectories, with the stereo camera being carried in hand by a person walking at normal walking speeds of 4--5 km/h.

In this paper, we show thatallprocesses associated with the move-sense-update cycle of extended Kalman filter (EKF) Simultaneous Localization and Mapping (SLAM) can be carried out in timelinearwith the number of map features. We describeDivide and ConquerSLAM, which is an EKF SLAM algorithm in which the computational complexity per step is reduced from$O(n^2)$to$O(n)$, and the total cost of SLAM is reduced from$O(n^3)$to$O(n^2)$. Unlike many current large-scale EKF SLAM techniques, this algorithm computes a solution without relying on approximations or simplifications (other than linearizations) to reduce computational complexity. Also, estimates and covariances are available when needed by data association without any further computation. Furthermore, as the method works most of the time in local maps, where angular errors remain small, the effect of linearization errors is limited. The resulting vehicle and map estimates are more precise than those obtained with standard EKF SLAM. The errors with respect to the true value are smaller, and the computed state covariance is consistent with the real error in the estimation. Both simulated experiments and the Victoria Park dataset are used to provide evidence of the advantages of this algorithm.

In this paper we describe divide and conquer SLAM (D&C SLAM), an algorithm for performing simultaneous localization and mapping using the extended Kalman filter. D&C SLAM overcomes the two fundamental limitations of standard EKF SLAM: 1.) the computational cost per step is reduced from O(n2) to O(n) (the cost full SLAM is reduced from O(n3) to O(n2)); 2.) the resulting vehicle and map estimates have better consistency properties than standard EKF SLAM in the sense that the computed state covariance adequately represents the real error in the estimation. Unlike many current large scale EKF SLAM techniques, this algorithm computes an exact solution, without relying on approximations or simplifications to reduce computational complexity. Also, estimates and covariances are available when needed by data association without any further computation. Empirical results show that, as a bi-product of reduced computations, and without losing precision because of approximations, D&C SLAM has better consistency properties than standard EKF SLAM. Both characteristics allow to extend the range of environments that can be mapped in real time using EKF. We describe the algorithm and study its computational cost and consistency properties.

This paper describes a navigation system for autonomous underwater vehicles (AUVs) in partially structured environments, such as dams, harbors, marinas or marine platforms. A mechanical scanning imaging sonar is used to obtain information about the location of planar structures present in such environments. A modified version of the Hough transform has been developed to extract line features, together with their uncertainty, from the continuous sonar datafiow. The information obtained is incorporated into a feature-based SLAM algorithm running an Extended Kalman Filter (EKF). Simultaneously, the AUV’s position estimate is provided to the feature extraction algorithm to correct the distortions that the vehicle motion produces in the acoustic images. Experiments carried out in a marina located in the Costa Brava (Spain) with the Ictineu AUV show the viability of the proposed approach.

In this paper we show how to optimize the computational cost and maximize consistency in EKF-based SLAM for large environments. We combine local mapping with map joining in a way that the total cost of computing the final map is minimized compared to full global EKF-SLAM. This solution is not now only shown to be (1) computationally optimal, but in addition, it is empirically shown that (2) it also produces the most consistent environment map. For a given environment size and sensor range, we can determine the optimal size of the local maps required to minimize the total computational cost and maximize map consistency. The motivation of this work is described in a map building experiment in our lab, and the statistical significance of the proposed method is validated using Monte Carlo simulations

A person buried by a snow avalanche can be found by measuring the magnetic field generated by an avalanche beacon or ARVA carried by the victim. However, the signals received are difficult to interpret and require people with good training on the actual searching techniques. In this paper we show that the search can be automated using SLAM techniques. The rescuer is equipped with an inertial sensor to estimate its own motion and a triple antenna to obtain 3D measurements of the magnetic field generated by the victim's ARVA. Both measurements are used to build a "robocentric" map that contains the location of the victim relative to the rescuer. To solve this highly non-linear SLAM problem we propose and compare two alternative solutions based on a sum of Gaussians (SOGs) filter and a particle filter. We present simulation results showing that, for comparable computing times, the SOGs solution gives more accurate results

In this paper we describe a system for underwater navigation with AUVs in partially structured environments, such as dams, ports or marine platforms. An imaging sonar is used to obtain information about the location of planar structures present in such environments. This information is incorporated into a feature-based SLAM algorithm in a two step process: (I) the full 360deg sonar scan is undistorted (to compensate for vehicle motion), thresholded and segmented to determine which measurements correspond to planar environment features and which should be ignored; and (2) SLAM proceeds once the data association is obtained: both the vehicle motion and the measurements whose correct association has been previously determined are incorporated in the SLAM algorithm. This two step delayed SLAM process allows to robustly determine the feature and vehicle locations in the presence of large amounts of spurious or unrelated measurements that might correspond to boats, rocks, etc. Preliminary experiments show the viability of the proposed approach

In this paper we study the global localization problem in SLAM: the determination of the vehicle location in a previously mapped environment with no other prior information. We show that, using a grid sampling representation of the configuration space, it is possible to evaluate all vehicle location hypotheses in the environment (up to a certain resolution) with a computational cost that is bilinear: linear both in the number of map features and in the number of sensor measurements. We propose a pairing-driven algorithm that considers only individual measurement-feature pairings and thus, in contrast with current correspondence space algorithms, it avoids searching in the exponential correspondence space. It uses a voting strategy that accumulates evidence for each vehicle location hypothesis, assuring robustness to noise in the sensor measurements and environment models. The general nature of the proposed strategy allows the consideration of different types of features and sensor measurements. Using the popular Victoria Park dataset, we compare its performance with location-driven algorithms where the solution space is usually randomly sampled. We show that the proposed pairing-driven technique is computationally more efficient in proportion to the density of features in the environment.

In this paper, we present a hierarchical mapping method that allows us to obtain accurate metric maps of large environments in real time. The lower (or local) map level is composed of a set of local maps that are guaranteed to be statistically independent. The upper (or global) level is an adjacency graph whose arcs are labeled with the relative location between local maps. An estimation of these relative locations is maintained at this level in a relative stochastic map. We propose a close to optimal loop closing method that, while maintaining independence at the local level, imposes consistency at the global level at a computational cost that is linear with the size of the loop. Experimental results demonstrate the efficiency and precision of the proposed method by mapping the Ada Byron building at our campus. We also analyze, using simulations, the precision and convergence of our method for larger loops.

We present a method for solving the first location problem using 2D laser and vision. Our observation is a two-dimensional laser scan together with its corresponding image. The observation is segmented into textured vertical planes; each vertical plane contains geometrical information about its location given by the laser scan, plus the gray level image obtained by the camera. The rich plane texture allows a safe plane recognition. Once two planes are recognized as correspondent, the computer vision geometry allows to compute the relative camera motion. The proposed algorithm outperforms both laser-only and vision-only algorithms. This is shown in the experimental results where a map composed of 8 observations of a 20/spl times/3 meter corridor is used to successfully locate the robot (without any other prior) in 163 out of 192 initial test robot locations.

Abstract — In this paper we propose an algorithm to determine the location of a vehicle in an environment represented by a stochastic map, given a set of environment measurements obtained by a sensor mounted on the vehicle. We show that the combined use of (1) geometric constraints considering feature correlation, (2) joint compatibility, (3) random sampling and (4) locality, make this algorithm linear with both the size of the stochastic map and the number of measurements. We demonstrate the practicality and robustness of our approach with experiments in an outdoor environment. I.

This paper describes a real-time implementation of feature-based concurrent mapping and localization (CML) running on a mobile robot in a dynamic indoor environment. Novel characteristics of this work include: (1) a hierarchical representation of uncertain geometric relationships that extends the SPMap framework, (2) use of robust statistics to perform extraction of line segments from laser data in real-time, and (3) the integration of CML with a "roadmap" path planning method for autonomous trajectory execution. These Innovations are combined to demonstrate the ability for a mobile robot to autonomously return back to its starting position within a few centimeters of precision, despite the presence of numerous people walking through the environment.

This paper describes how multisensor fusion increases both reliability and precision of the environmental observations used for the simultaneous localization and map-building problem for mobile robots. Multisensor fusion is performed at the level of landmarks, which represent sets of related and possibly correlated sensor observations. The work emphasizes the idea of partial redundancy due to the different nature of the information provided by different sensors. Experimentation with a mobile robot equipped with a multisensor system composed of a 2D laser rangefinder and a charge coupled device camera is reported.

In this paper, we address the problem of robust data association for simultaneous vehicle localization and map building. We show that the classical gated nearest neighbor approach, which considers each matching between sensor observations and features independently, ignores the fact that measurement prediction errors are correlated. This leads to easily accepting incorrect matchings when clutter or vehicle errors increase. We propose a new measurement of the joint compatibility of a set of pairings that successfully rejects spurious matchings. We show experimentally that this restrictive criterion can be used to efficiently search for the best solution to data association. Unlike the nearest neighbor, this method provides a robust solution in complex situations, such as cluttered environments or when revisiting previously mapped regions.

We present a comparative study of the performance of map-based robot localisation processes based on diverse sensing devices such as monocular and trinocular vision systems and laser rangefinders. We study both the precision (error with respect to the true values) and robustness (sensor measurements correctly paired with map features) of each localisation process. The experiment design we used allows one to compare these processes under exactly the same conditions. We conclude that comparable precision levels can be attained with each of the three sensors. With respect to robustness, monocular and trinocular vision pose more complex matching problems than laser, requiring more elaborate solutions to make the process robust.

This article describes a rigorous and complete framework for the simultaneous localization and map building problem for mobile robots: the symmetries and perturbation map (SPmap), which is based on a general probabilistic representation of uncertain geometric information. We present a complete experiment with a LabMate/sup TM/ mobile robot navigating in a human-made indoor environment and equipped with a rotating 2D laser rangefinder. Experiments validate the appropriateness of our approach and provide a real measurement of the precision of the algorithms.

We present the two-dimensional (2-D) version of the symmetries and perturbation model (SPmodel), a probabilistic representation model and an extended Kalman filter integration mechanism for uncertain geometric information that is suitable for sensor fusion and integration in multisensor systems. We apply the SPmodel to the problem of location estimation in indoor mobile robotics, experimenting with the mobile robot MACROBE. We have chosen two types of complementary sensory information: (1) range images; (2) intensity images; obtained from a laser sensor. Results of these experiments show that fusing simple and computationally inexpensive sensory information can allow a mobile robot to precisely locate itself. They also demonstrate the generality of the proposed fusion and integration mechanism.

During mobile robot navigation, position estimates obtained by odometry drift with time, therefore becoming unrealistic and useless. This work enhances the use of external mechanisms by considering a multisensor system, composed of a 2D laser rangefinder and an off-the-shelf CCD camera, which provides redundancy and assures reliability and precision of the observed features. We simultaneously consider both the map building and the localization problems using a state vector approach, which is related to the location estimations of both the robot and the map features, whilst its covariance matrix reflects the relationships between them. Relevance and importance of its off-diagonal elements is demonstrated by their contributions to "backwards estimations" whenever the vehicle returns to places in the navigation area which have been already visited and learned. Real experiments are presented, considering a LabMate mobile robot navigating in an static indoor environment.

Presents the 2D version of the symmetries and perturbation model (SPmodel), a probabilistic representation model and an EKF integration mechanism for uncertain geometric information that is suitable for sensor fusion and integration in multisensor systems. The authors apply the SPmodel to the problem of location estimation in mobile robotics, experimenting with the mobile robot MACROBE. The authors have chosen two types of sensorial information whose complementary nature allows a mobile robot to localize itself precisely in a known environment: range images and intensity images obtained from a laser sensor. By measuring distance to objects, range basically provides position information. On the other hand, the intensity image provided by the laser sensor (equivalent to a monocular vision system) can be used to detect vertical edges, corresponding to corners, frames, etc., which mainly provide angular information. Results of these experiments show that fusing simple and computationally inexpensive sensorial information can allow a mobile robot to precisely locate itself. They also demonstrate the generality of the proposed fusion and integration mechanism.