Accurate and reliable online real-time localization and mapping are crucial for autonomous navigation of robot. Dynamic objects within the perception field can affect the accuracy of registration and localization, and also introduce ghost trail artifacts in the map, hindering robot planning and decision-making. While semantic segmentation can assist in perceiving object categories, it struggles to accurately segment moving objects. In this paper, we present SOLO-SMap, a real-time localization and static map construction framework based solely on LiDAR point cloud. We leverage semantic inference to identify potential dynamic points. And then, our instance-level true dynamic points removal is achieved by utilizing geometric rules based on moving point occlusion relationships and multi-object tracking (MOT) within a nearby temporal window in the pre-alignment stage. This design preserves stable static constraints while adhering to the static world model assumption of SLAM systems, benefiting accuracy and reducing drift, particularly in busy intersections. We evaluated the performance of SOLO-SMap in dynamic scenes on KITTI datasets and our self-made datasets, and conducted a comprehensive comparison with other methods, validating the effectiveness and robustness of the proposed method. A supplementary video can be accessed at https://www.youtube.com/watch?v=x-VKr3ag03M.

To address the issues of inaccurate feature association and insufficient utilization of the complementarity between the visual and inertial information in visual-inertial navigation systems, this paper proposes a feature radiance field (FeRF)-based bidirectionally enhanced visual-inertial odometry (FeRF-BEVIO). FeRF is a novel method for describing environment features, incorporating both the grayscale and position of point and line features. It provides a unified framework for representing and storing features, regardless of whether their depth has converged. Then, a bidirectional visual-inertial enhancement method is designed based on FeRFs. Specifically, on one hand, the information of the inertial measurement unit (IMU) is utilized to aid visual feature association within FeRFs. On the other hand, the visual information is employed to refine the IMU parameter estimation. This bidirectional enhancement process is iterated to improve the integration of the visual and inertial data. At last, the system jointly optimizes the robot poses and point-line feature parameters within a sliding window and updates FeRFs accordingly. Comparative experiments on public datasets and in the real-world environments demonstrate that FeRF-BEVIO outperforms state-of-the-art visual-inertial odometry (VIO) systems in both accuracy and robustness. Therefore, FeRF-BEVIO is highly suitable for navigation and simultaneous localization and mapping (SLAM) of micro aerial vehicles (MAVs).

In this paper, an efficient motion planning approach with grid-based generalized Voronoi diagrams (G$^2$VD) is newly proposed for mobile robots. Different from existing approaches, the novelty of this work is twofold: 1) a new state lattice-based path searching approach is proposed, in which the search space is reduced to a novel Voronoi corridor to further improve the search efficiency; 2) an efficient quadratic programming-based path smoothing approach is presented, wherein the clearance to obstacles is considered to improve the path clearance of hard-constrained path smoothing approaches. We validate the efficiency and smoothness of our approach in various challenging simulation scenarios and outdoor environments. It is shown that the computational efficiency is improved by 17.1% in the path searching stage, and path smoothing with the proposed approach is 6.6 times faster than an advanced sparse-banded structure-based path smoothing approach and 53.3 times faster than the popular timed-elastic-band planner. A video showing outdoor navigation on our campus is available at https://youtu.be/iMXGthgvp58.Note to Practitioners—This paper is motivated by the challenges of motion planning problems of mobile robots. An efficient motion planning approach called G$^2$VD planner is proposed by combining path searching, path smoothing, and time-optimal velocity planning. Extensive simulation and experimental results show the effectiveness of the proposed motion planning approach. However, the prediction information of dynamic obstacles is not incorporated in the proposed motion planner, thus the motion planner may be a bit sluggish in response to dynamic obstacles. Furthermore, we plan to integrate the intention/trajectory prediction of pedestrians/vehicles into the proposed framework to enhance the foreseeability of the motion planner.

Existing object-level SLAM methods often overlook the correspondence between semantic information and geometric features, resulting in a significant gap between them within SLAM frameworks. To tackle this issue, this paper proposes TiMoSLAM, a semantic-geometric tight-coupling monocular visual object SLAM system, which considers a rigorous correspondence between semantics and geometry across all steps of SLAM. Initially, a general Semantic Relation Graph (SRG) is developed to consistently represent semantic information alongside geometric features. Detailed analyses on complete constraints of the geometric feature combinations on estimation of 3D cuboid model are performed. Subsequently, a Compound Hypothesis Tree (CHT) is proposed to incrementally construct the object-specific SRG and concurrently estimate the 3D cuboid model of an object, ensuing semantic-geometric consistency in object representation and estimation. Special attention is given to the matching errors between geometric features and objects during the optimization of camera poses and object parameters. The effectiveness of this method is validated on various datasets, as well as in real-world environments.

In this paper, a novel multi-robot autonomous exploration approach CURE is proposed based on dynamic Voronoi diagrams and centroids of unknown connected regions. Compared with existing approaches, the novelty of this work is twofold: 1) Dynamic Voronoi diagram is used for partition of the space being explored to improve the efficiency of multi-robot exploration, and then a new parameter-insensitive utility function is elaborately designed to evaluate the information of centroids, which helps guide the robot to explore unknown regions. 2) A hierarchical framework consisting of global and local exploration windows for detecting centroids is designed, wherein the global exploration window is activated to find centroids to guide the robot exploration when there are no centroids in any one local exploration window. We validate the feasibility and exploration efficiency of the proposed approach in various complex simulation scenarios and challenging real-world tasks. All test results show that the exploration time consumption and path cost are reduced by up to 50.7% and 34.4%, respectively, compared with an advanced RRT-based multi-robot exploration approach. (Supplementary video link: https://youtu.be/P5jXKlGQOec)Note to Practitioners—This paper is motivated by the efficient multi-robot autonomous exploration problem. In some applications such as target search and disaster rescue, the information about the environment is totally unknown to the robots, and thus they are required to explore unknown environments autonomously. In this case, it is necessary to improve the efficiency of multi-robot exploration due to the time limitation of the task and the battery capacity. In this paper, a hierarchical framework is proposed to improve the efficiency of multi-robot autonomous exploration. Each robot only needs to explore the Voronoi partition it is responsible for and is guided to the unknown region by the centroid detected in the global and local exploration windows. Overall, the proposed approach can dramatically reduce the exploration time and path cost.

Multi-sensor fusion is a mainstream method for localization of unmanned systems. How to achieve 6-degrees of freedom (DOF) pose estimation of the system is challenging in GPS-denied environments. Although map-aided localization methods normally perform well on intelligent transportation systems, prior maps are unavailable in some GPS-denied scenes (e.g., dense forests, tunnels, and underground parking lots). In this paper, we present a robust and consistent monocular visual-inertial-depth odometry (VIDO) to perform 6-DOF pose estimation without the need of prior information. The system contains a visual-inertial subsystem (VIS) based on tightly coupled optimization in a sliding window and a depth subsystem (DS) based on the iterative closest point (ICP) estimation using 3D point clouds obtained by a LiDAR or depth camera. The uncertainties of the estimation results in VIS and DS are rigorously calculated to consider measurement noises of the sensors. The obtained uncertainty estimates are fed into a covariance intersection (CI) filter for pose fusion, and the fused pose is further refined in the mapping process. We perform experiments on public datasets, as well as in various real-world outdoor and indoor scenes to verify the performance on localization and mapping in urban areas with buildings and cars, off-road environments with rugged terrains, as well as indoor structured environments. The results show that the proposed method can provide both a robust 6-DOF pose estimate and a precise 3D map for fully autonomous navigation in different scenes without a prior map, which presents an attractive complement to map-aided automated driving.

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach is newly presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and time-optimal velocity planning. Compared with existing approaches, the novelty of this work is twofold: 1) a novel heuristic-guided pruning strategy of motion primitives is proposed and fully integrated into the state lattice-based global path planner to further improve the computational efficiency of graph search, and 2) a new soft-constrained local path optimization approach is proposed, wherein the sparse-banded system structure of the underlying optimization problem is fully exploited to efficiently solve the problem. We validate the safety, smoothness, flexibility, and efficiency of our approach in various complex simulation scenarios and challenging real-world tasks. It is shown that the computational efficiency is improved by 66.21% in the global planning stage and the motion efficiency of the robot is improved by 22.87% compared with the recent quintic Bézier curve-based state space sampling approach. We name the proposed motion planning framework E$\mathbf {^{3}} $MoP, where the number 3 not only means our approach is a three-layer framework but also means the proposed approach is efficient in three stages. Note to Practitioners—This paper is motivated by the challenges of motion planning problems of mobile robots. A three-layer motion planning framework is proposed by combining global path planning, local path optimization, and time-optimal velocity planning. For mobile robot navigation applications in semi-structured environments, optimization-based local planners are recommended. Extensive simulation and experimental results show the effectiveness of the proposed motion planning framework. However, due to the non-convexity of the path optimization formulation, the proposed local planner may get stuck in local optima. In future research, we will concentrate on extending the proposed local path optimization approach with the theory of homology classes to maintain several homotopically distinct local paths and seek global optima.

Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning researchers who want to compare the performance of a new local planner relative to many other state-of-the-art approaches as well as end users in the mobile robotics industry who want to select a local planner that performs best on some problems of interest. We elaborately design various simulation scenarios to challenge the applicability of local planners, including large-scale, partially unknown, and dynamic complex environments. Furthermore, three types of principled evaluation metrics are carefully designed to quantitatively evaluate the performance of local planners, wherein the safety, efficiency, and smoothness of motions are comprehensively considered. We present the application of the proposed benchmark in two popular open-source local planners to show the practicality of the benchmark. In addition, some insights and guidelines about the design and selection of local planners are also provided. The benchmark website [1] contains all data of the designed simulation scenarios, detailed descriptions of these scenarios, and example code.

Planes and edges are attractive features for simultaneous localization and mapping (SLAM) in indoor environments because they can be reliably extracted and are robust to illumination changes. However, it remains a challenging problem to seamlessly fuse two different kinds of features to avoid degeneracy and accurately estimate the camera motion. In this article, a plane-edge-SLAM system using an RGB-D sensor is developed to address the seamless fusion of planes and edges. Constraint analysis is first performed to obtain a quantitative measure of how the planes constrain the camera motion estimation. Then, using the results of the constraint analysis, an adaptive weighting algorithm is elaborately designed to achieve seamless fusion. Through the fusion of planes and edges, the solution to motion estimation is fully constrained, and the problem remains well-posed in all circumstances. In addition, a probabilistic plane fitting algorithm is proposed to fit a plane model to the noisy 3-D points. By exploiting the error model of the depth sensor, the proposed plane fitting is adaptive to various measurement noises corresponding to different depth measurements. As a result, the estimated plane parameters are more accurate and robust to the points with large uncertainties. Compared with the existing plane fitting methods, the proposed method definitely benefits the performance of motion estimation. The results of extensive experiments on public data sets and in real-world indoor scenes demonstrate that the plane-edge-SLAM system can achieve high accuracy and robustness.Note to Practitioners—This article is motivated by the robust localization and mapping for mobile robots. We suggest a novel simultaneous localization and mapping (SLAM) approach fusing the plane and edge features in indoor scenes (plane-edge-SLAM). This newly proposed approach works well in the textureless or dark scenes and is robust to the sensor noise. The experiments are carried out in various indoor scenes for mobile robots, and the results demonstrate the robustness and effectiveness of the proposed framework. In future work, we will address the fusion of other high-level features (for example, 3-D lines) and the active exploration of the environments.

Target recognition is a challenging task for human following of mobile service robots. In this paper, we combine the principal-component-analysis (PCA)-based face recognition with the tracking-learning-detection applied to the human face (Face-TLD) to obtain an improvement, named as IFace-TLD. The proposed IFace-TLD can significantly improve the discrimination ability of the Face-TLD for ambiguous facial appearances. To further deal with motion uncertainties of the human head, especially the sudden motion change, which makes face-based target recognition methods unstable or even loses the target, a skeleton-based model is introduced to improve the accuracy and robustness of the target recognition. Specifically, within a walk half-cycle, the skeleton features are extracted from the upper-body three-dimensional skeleton coordinates. Then, the extracted skeleton features are fed into the support vector data description (SVDD) to identify the target person when the IFace-TLD becomes invalid. The seamless fusion of the skeleton recognition and the IFace-TLD, named as the SIFace-TLD, significantly enhances the robustness in complex scenarios, especially for people tracking from both front and behind. To achieve a complete human following system, the particle filter (PF) is adopted for estimating the state of the human motion. And then, a controller is designed to maintain the relative position between the robot and the target. Experimental results demonstrate that the proposed IFace-TLD is more accurate and flexible than the original Face-TLD. And the SIFace-TLD shows a robust performance to human motion uncertainties. Moreover, the developed controller can achieve a satisfactory human following performance.

Visual place recognition is a challenging problem in simultaneous localization and mapping (SLAM) due to a large variability of the scene appearance. A place is usually described by a single-frame image in conventional place recognition algorithms. However, it is unlikely to completely describe the place appearance using a single frame image. Moreover, it is more sensitive to the change of environments. In this article, a novel image-sequence-based framework for place detection and recognition is proposed. Rather than a single frame image, a place is represented by an image sequence in this article. Position invariant robust feature (PIRF) descriptors are extracted from images and processed by the incremental bag-of-words (BoWs) for feature extraction. The robot automatically partitions the sequentially acquired images into different image sequences according to the change of the environmental appearance. Then, the echo state network (ESN) is applied to model each image sequence. The resultant states of the ESN are used as features of the corresponding image sequence for place recognition. The proposed method is evaluated on two public datasets. Experimental comparisons with the FAB-MAP 2.0 and SeqSLAM are conducted. Finally, a real-world experiment on place recognition with a mobile robot is performed to further verify the proposed method.

Human following in structured indoor environments has to face the challenge of full occlusion caused by the walls when the target person makes a turn at the corridor intersections. This may result in short-term, and even permanent loss of the target from the field of view of the robot. In this paper, human following with a mobile robot in presence of potential occlusions occurring at corridor intersections is addressed. The robot detects four different types of corridor intersections using the on-board laser scanner. Then, a potential-field-based human tracker is designed by integrating the intersection information into the potential function, in order to increase the visibility of the target, while maintaining the relative distance and orientation between the target and the robot. Simultaneously, the robot builds a multihypothesis topological map of the environment based on an improved generalized Voronoi graph, where the edges represent the corridors and the vertices are the virtual meet points extracted from the intersections. In such a way, simultaneous human following and topological mapping are achieved. Simulation and experimental results show that the proposed method can largely avoid occlusion of the target and obtain good performance of human following and topological mapping.

In this paper, a novel 2-D laser-based autonomous exploration approach for mobile robots is proposed, which is based on a novel polygon map construction approach and graph-based simultaneous localization and mapping (SLAM) with directional endpoint features. This approach is composed of three modules: graph-based SLAM using directional endpoint features, polygon map construction, and exploration. Different from existing approaches in the field of 2-D SLAM, the newly proposed 2-D graph-SLAM is based on 3-D “directional endpoint” features; on this basis, a well-known data structure “circular-doubly linked list” is applied to construct a novel polygon map for navigation. Note that it is efficient for circular-doubly linked list to initialize and update the polygon map. In addition, we propose a new information entropy calculation approach to quantify the entropy of the polygon map. Then for each candidate goal, we could obtain corresponding information gain and make next decision through collision detection. Comparative experimental results with respect to the well-known Gmapping and Karto SLAM are presented to show superior performance of the proposed graph-based SLAM. The autonomous exploration experiments in the office and hallway environments show the effectiveness of the proposed approach for robotic mapping and exploration tasks.

In this paper, a multilevel humanlike motion planning approach is proposed for indoor mobile robots. Compared with existing approaches, the novelty of this paper is twofold: 1) the proposed path planning framework is multilevel and humanlike to ensure both foreseeability and flexibility, wherein functions of human brain, eyes, and legs are corresponding to global path planning, sensor-level path planning, and action-level path planning, respectively, and 2) along the planned path, a new velocity-adjustable trajectory planning algorithm is put forward which is provably complete and time optimal considering multiple constraints from both the robot and the environment. Experimental results show that the proposed approach has a better performance in terms of efficiency, smoothness, foreseeability, and flexibility, and autonomous navigation is realized in large-scale, dynamic, partially unknown, and unstructured indoor environments.

A cooperation approach with consideration of communication limit is proposed for multi-robot area exploration, in which all the robots select local destinations satisfying the constraints on communication range and reach their destinations at the same time to communicate and fuse their map information. Firstly, the robots compute the frontier between the explored region and the unexplored one. The robots choose the optimal frontier points, which maximize information gain, minimize navigation cost and satisfy communication limit as their local destinations. Then the problem of global exploration in unknown environment is converted into that of multi-stage trajectory planning in local known environment. Collision-free, synchronous and separate trajectories are planned for all the robots to realize the limited communication at their destinations. In such a way, efficient and distributed exploration can be achieved. Simulation results are presented to show the effectiveness of our method.

This paper investigates the on-line autonomous distribution for landmarks and the active exploration in environment without or lack of landmarks/features, such as disaster conditions and polar region. In such situation, the robot enters the environment carrying some landmarks and distributes them according to the rules given in this paper. The utility of the landmark distribution is analyzed. Then, based on the extended Kalman filter (EKF), the active exploration is converted into a problem of multi-objective optimization, in which the objective function includes three aspects, i.e. the accuracy of localization and mapping, the predictive area of the unknown environment that will be explored in next step and the information gain provided by the distributed landmarks respectively. The robot chooses the control input that optimizes the objective function such that accurate localization, high-quality mapping and complete exploration will be realized. And then, the supplementation and the redundancy elimination for landmarks are implemented. At last, a set of simulations is presented to show the effectiveness of our approach.

This paper addresses the problems of the forward and backward path following control for tractor-trailer mobile robots (TTMR) with connection structures of on-axle hitching and off-axle hitching. First, the kinematics is described and the motion characteristics are analyzed. Then, by Lyapunov method we design a global path following controller for single-body mobile robot and extend it to the forward path following control of the TTMR. Furthermore, by the kinematics transformation and the backstepping technique respectively, we propose approaches to the backward path following control for TTMR with two kinds of connection structures based on the above controller. Finally, a set of simulations is presented to show the validity of our approach

Path planned for tractor-trailer mobile robot (TTMR) does not seem to be easily adapted to the case when the number of the trailers increases, thus the repetitive path planning has to be introduced. For this problem, this paper investigates the connection relation of TTMR with variable structure, i.e. the number of the trailers is variable, and deals with the optimization design for it to avoid the repetitive path planning when the number of the trailers increases. After analyzing the motion trajectories of TTMR, we establish the quantitative relationship for the motion characteristics between the transient course and the steady state. Based on which, a new design method to optimize the connection relation for variable-structure TTMR is proposed. In our scheme, the length of each connecting rod between two adjacent bodies is adjusted properly such that the transient deviations of the trailers from the path followed by the tractor will be less than the steady ones. In such a way, the path planned for TTMR can also be appropriate for the case when the number of the trailers varies. The simulations and experiments show the validity of our method