Multi-sensor fusion is a mainstream method for localization of unmanned systems. How to achieve 6-degrees of freedom (DOF) pose estimation of the system is challenging in GPS-denied environments. Although map-aided localization methods normally perform well on intelligent transportation systems, prior maps are unavailable in some GPS-denied scenes (e.g., dense forests, tunnels, and underground parking lots). In this paper, we present a robust and consistent monocular visual-inertial-depth odometry (VIDO) to perform 6-DOF pose estimation without the need of prior information. The system contains a visual-inertial subsystem (VIS) based on tightly coupled optimization in a sliding window and a depth subsystem (DS) based on the iterative closest point (ICP) estimation using 3D point clouds obtained by a LiDAR or depth camera. The uncertainties of the estimation results in VIS and DS are rigorously calculated to consider measurement noises of the sensors. The obtained uncertainty estimates are fed into a covariance intersection (CI) filter for pose fusion, and the fused pose is further refined in the mapping process. We perform experiments on public datasets, as well as in various real-world outdoor and indoor scenes to verify the performance on localization and mapping in urban areas with buildings and cars, off-road environments with rugged terrains, as well as indoor structured environments. The results show that the proposed method can provide both a robust 6-DOF pose estimate and a precise 3D map for fully autonomous navigation in different scenes without a prior map, which presents an attractive complement to map-aided automated driving.

Planes and edges are attractive features for simultaneous localization and mapping (SLAM) in indoor environments because they can be reliably extracted and are robust to illumination changes. However, it remains a challenging problem to seamlessly fuse two different kinds of features to avoid degeneracy and accurately estimate the camera motion. In this article, a plane-edge-SLAM system using an RGB-D sensor is developed to address the seamless fusion of planes and edges. Constraint analysis is first performed to obtain a quantitative measure of how the planes constrain the camera motion estimation. Then, using the results of the constraint analysis, an adaptive weighting algorithm is elaborately designed to achieve seamless fusion. Through the fusion of planes and edges, the solution to motion estimation is fully constrained, and the problem remains well-posed in all circumstances. In addition, a probabilistic plane fitting algorithm is proposed to fit a plane model to the noisy 3-D points. By exploiting the error model of the depth sensor, the proposed plane fitting is adaptive to various measurement noises corresponding to different depth measurements. As a result, the estimated plane parameters are more accurate and robust to the points with large uncertainties. Compared with the existing plane fitting methods, the proposed method definitely benefits the performance of motion estimation. The results of extensive experiments on public data sets and in real-world indoor scenes demonstrate that the plane-edge-SLAM system can achieve high accuracy and robustness.Note to Practitioners—This article is motivated by the robust localization and mapping for mobile robots. We suggest a novel simultaneous localization and mapping (SLAM) approach fusing the plane and edge features in indoor scenes (plane-edge-SLAM). This newly proposed approach works well in the textureless or dark scenes and is robust to the sensor noise. The experiments are carried out in various indoor scenes for mobile robots, and the results demonstrate the robustness and effectiveness of the proposed framework. In future work, we will address the fusion of other high-level features (for example, 3-D lines) and the active exploration of the environments.

Target recognition is a challenging task for human following of mobile service robots. In this paper, we combine the principal-component-analysis (PCA)-based face recognition with the tracking-learning-detection applied to the human face (Face-TLD) to obtain an improvement, named as IFace-TLD. The proposed IFace-TLD can significantly improve the discrimination ability of the Face-TLD for ambiguous facial appearances. To further deal with motion uncertainties of the human head, especially the sudden motion change, which makes face-based target recognition methods unstable or even loses the target, a skeleton-based model is introduced to improve the accuracy and robustness of the target recognition. Specifically, within a walk half-cycle, the skeleton features are extracted from the upper-body three-dimensional skeleton coordinates. Then, the extracted skeleton features are fed into the support vector data description (SVDD) to identify the target person when the IFace-TLD becomes invalid. The seamless fusion of the skeleton recognition and the IFace-TLD, named as the SIFace-TLD, significantly enhances the robustness in complex scenarios, especially for people tracking from both front and behind. To achieve a complete human following system, the particle filter (PF) is adopted for estimating the state of the human motion. And then, a controller is designed to maintain the relative position between the robot and the target. Experimental results demonstrate that the proposed IFace-TLD is more accurate and flexible than the original Face-TLD. And the SIFace-TLD shows a robust performance to human motion uncertainties. Moreover, the developed controller can achieve a satisfactory human following performance.

Visual place recognition is a challenging problem in simultaneous localization and mapping (SLAM) due to a large variability of the scene appearance. A place is usually described by a single-frame image in conventional place recognition algorithms. However, it is unlikely to completely describe the place appearance using a single frame image. Moreover, it is more sensitive to the change of environments. In this article, a novel image-sequence-based framework for place detection and recognition is proposed. Rather than a single frame image, a place is represented by an image sequence in this article. Position invariant robust feature (PIRF) descriptors are extracted from images and processed by the incremental bag-of-words (BoWs) for feature extraction. The robot automatically partitions the sequentially acquired images into different image sequences according to the change of the environmental appearance. Then, the echo state network (ESN) is applied to model each image sequence. The resultant states of the ESN are used as features of the corresponding image sequence for place recognition. The proposed method is evaluated on two public datasets. Experimental comparisons with the FAB-MAP 2.0 and SeqSLAM are conducted. Finally, a real-world experiment on place recognition with a mobile robot is performed to further verify the proposed method.

Human following in structured indoor environments has to face the challenge of full occlusion caused by the walls when the target person makes a turn at the corridor intersections. This may result in short-term, and even permanent loss of the target from the field of view of the robot. In this paper, human following with a mobile robot in presence of potential occlusions occurring at corridor intersections is addressed. The robot detects four different types of corridor intersections using the on-board laser scanner. Then, a potential-field-based human tracker is designed by integrating the intersection information into the potential function, in order to increase the visibility of the target, while maintaining the relative distance and orientation between the target and the robot. Simultaneously, the robot builds a multihypothesis topological map of the environment based on an improved generalized Voronoi graph, where the edges represent the corridors and the vertices are the virtual meet points extracted from the intersections. In such a way, simultaneous human following and topological mapping are achieved. Simulation and experimental results show that the proposed method can largely avoid occlusion of the target and obtain good performance of human following and topological mapping.