Long-horizon composite task planning for multi-robot systems in cross-regional complex scenarios faces dual challenges: spatial-semantic comprehension of natural language described tasks and collaborative optimization of subtask al-location. To address these challenges, this paper proposes a progressive three-stage task planning framework. First, an augmented scene graph is constructed to enable large language models (LLMs) to comprehend environmental structures, thereby generating simplified Linear Temporal Logic (LTL) task sequences. Subsequently, a novel heuristic function is employed to select optimal task allocation plans. Finally, LLMs are used to generate low-level executable robot instructions based on robotic system instruction templates. We establish a long-horizon composite task dataset for experimental validation on real-world quadrupedal multi-robot systems. Experimental results demonstrate the effectiveness of our approach in resolving cross-regional composite tasks.

It is a challenging task for ground robots to autonomously navigate in harsh environments due to the presence of non-trivial obstacles and uneven terrain. This requires trajectory planning that balances safety and efficiency. The primary challenge is to generate a feasible trajectory that prevents robot from tip-over while ensuring effective navigation. In this paper, we propose a capsizing-aware trajectory planner (CAP) to achieve trajectory planning on the uneven terrain. The tip-over stability of the robot on rough terrain is analyzed. Based on the tip-over stability, we define the traversable orientation, which indicates the safe range of robot orientations. This orientation is then incorporated into a capsizing-safety constraint for trajectory optimization. We employ a graph-based solver to compute a robust and feasible trajectory while adhering to the capsizing-safety constraint. Extensive simulation and real-world experiments validate the effectiveness and robustness of the proposed method. The results demonstrate that CAP outperforms existing state-of-the-art approaches, providing enhanced navigation performance on uneven terrains.

Remote sensing images processing and analysis face significant challenges due to varying object scales and complex backgrounds. Existing detection algorithms often suffer from high computational complexity and suboptimal performance. A lightweight algorithm SCC-YOLO was proposed for remote sensing objects detection. It incorporates three key innovations: (1) Slimneck-V feature fusion architecture to enhance multi-scale adaptability while reducing computational load. (2) Cross Stage Partial with Context Anchor Attention (C2CAA) module to improve feature representation of key object regions. (3) Cross Stage Partial with Ghost (CSPGhost) module that optimizes feature extraction efficiency. The algorithm is validated on DOTA and RSOD datasets. Experimental results demonstrate that, compared to baseline algorithms, SCC-YOLO reduces model parameters by 15.3% and computational complexity by 26%. On the DOTA dataset, detection accuracy and inference speed are improved by 3.9% and 6.5%, respectively.

Visual localization plays a critical role in the functionality of low-cost autonomous mobile robots. Contemporary leading methods for precise visual localization are predominantly 3D scene-specific, necessitating extra computational and memory overhead to construct a 3D scene model in novel environments. An alternative approach of directly using a database of 2D images for visual localization offers more flexibility. However, such methods currently suffer from limited localization accuracy. In this paper, we propose an accurate and robust multiple checking-based 3D model-free visual localization system to address the aforementioned issues. To ensure high accuracy, our focus is on estimating the pose of a query image relative to the retrieved database images using 2D-2D feature matches. Theoretically, by incorporating the local planar motion constraint into both the estimation of the essential matrix and the triangulation stages, we reduce the minimum required feature matches for absolute pose estimation, thereby enhancing the robustness of outlier rejection. Additionally, we introduce a multiple-checking mechanism to ensure the correctness of the solution throughout the solving process. The efficacy of our approach is substantiated through both qualitative and quantitative assessments on simulated and two real-world datasets evidencing significant improvements in accuracy and robustness provided by our 3D model-free visual localization system.Note to Practitioners—The motivation of this article stems from the need to develop an accurate visual localization system with simplicity and flexibility of map construction and easy adaption to new environments. Such a system holds great practical value for a range of applications, including warehouse robots, service robots, and countless others. Existing visual localization systems that achieve high accuracy are dependent on a pre-built accurate 3D scene map, which pose challenges in terms of map construction and consume significant storage resources onboard, particularly for large scenes. And the aforementioned efforts need to be repeated when changing to a new scene. In this article, an accurate and robust 3D model-free visual localization system is proposed to handle this problem. The map construction is simplified to build a set of database images with associated camera poses, which is trivial as it amounts to adding posed images to a database. The core idea for achieving high accuracy and robustness is to model the local planar motion characteristic of general ground-moving robots into both essential matrix estimation and triangulation stages to obtain two minimal solutions. The proposed localization system simplifies the task of switching between different application scenarios for the robot, reducing additional workload and lowering the difficulty of use.

Trajectory planning for quadrotors in cluttered environments has been challenging in recent years. While many trajectory planning frameworks have been successful, there still exists potential for improvements, particularly in enhancing the speed of generating efficient trajectories. In this paper, we present a novel hierarchical trajectory planning framework to reduce computational time and memory usage called MINER-RRT*, which consists of two main components. First, we propose a sampling-based path planning method boosted by neural networks, where the predicted heuristic region accelerates the convergence of rapidly-exploring random trees. Second, we utilize the optimal conditions derived from the quadrotor’s differential flatness properties to construct polynomial trajectories that minimize control effort in multiple stages. Extensive simulation and real-world experimental results demonstrate that, compared to several state-of-the-art (SOTA) approaches, our method can generate high-quality trajectories with better performance in 3D cluttered environments (https://youtu.be/fXuuMRX19q0). Note to Practitioners—The motivation is the problem of planning trajectories for quadrotor autonomous flight in 3D cluttered and complex scenarios such as wild forest exploration and subterranean environment search-and-rescue. Sampling-based path planning methods are suitable for dealing with the complexity of the physical environment but are not convenient for computing dynamics and their differentials. Optimization-based trajectory generation methods are appropriate for handling various high-order constraints but rely on high-quality initial path solutions. Therefore, this paper combines the advantages of the two methods to propose a novel trajectory planning framework that can generate high-quality trajectories for quadrotors faster than many previous algorithms. We conduct numerous simulations and real-world experiments to verify that our method can be effectively deployed in real scenarios and empower quadrotors for complex autonomous tasks in the future.

Endowing robots with the ability to understand natural language and execute grasping is a challenging task in a human-centric environment. Existing works on language-conditioned grasping achieve end-to-end grasping detection based on language. However, these works lack fine-grained visual grounding, resulting in cognitive deficits for robots. Moreover, they ignore the correlation between visual attributes of objects and grasping, leading to coarse grasp poses. To this end, we propose a CLIP-driven aTtribute-aware network (CTNet) for language-conditioned visual segmentation and grasping, enabling the robots to listen, perceive, and grasp the referred object in real-world applications. Specifically, we first employ Listen stage to understand basic linguistic and visual concepts. Subsequently, we introduce Perceive stage to mine multi-modal features and visual attribute cues (e.g., boundary and spatial location), then yield a language-conditioned segmentation mask. Further, we design Grasp stage to aggregate the perceived attribute information and refine the spatial location and grasping rectangle, generating a high-quality grasp pose. Lastly, we provide an extended large dataset Ref-OCID-Grasp to train and test our method, achieving a grasping accuracy of 97.76% and segmentation OIoU of 91.82%. The real-world robotic applications demonstrate the effectiveness of our proposed approach. The project, video, and dataset can be found athttps://ctnetgrasp.github.io. Note to Practitioners—Most of the existing grasping methods focus on clearing all objects in the workspace. However, as robots integrate into human society, robots should learn to grasp the desired object by understanding human language. Therefore, language-conditioned grasping is a significant skill for human-robot collaboration. The prior works directly complete the grasp detection through the language-grasp paradigm, but they ignore the discussion on whether the robot understands the concept of vision and language expression of the object. Therefore, this paper proposed the Listen-Perceive-Grasp paradigm, in which the Listen-Perceive stage is responsible for the conception alignment of the object in language expression and visual pixels, and the Perceive-Grasp stage achieves the constraining and refining the grasp detection by the perceived visual attributes such as boundary and shape. Experiments show that this method can obtain a refiner grasp pose in cluttered environments and perform language-conditioned grasping well in the real world. In future research, we will work on 6-DoF grasping and multi-object disambiguation conditioned on language.

Path planning in robotics often involves solving continuously valued, high-dimensional problems. Popular informed approaches include graph-based searches, such as A*, and sampling-based methods, such as Informed RRT*, which utilize informed set and anytime strategies to expedite path optimization incrementally. Informed sampling-based planners define informed sets as subsets of the problem domain based on the current best solution cost. However, when no solution is found, these planners re-sample and explore the entire configuration space, which is time-consuming and computationally expensive. This article introduces Multi-Informed Trees (MIT*), a novel planner that constructs estimated informed sets based on prior admissible solution costs before finding the initial solution, thereby accelerating the initial convergence rate. Moreover, MIT* employs an adaptive sampler that dynamically adjusts the sampling strategy based on the exploration process. Furthermore, MIT* utilizes length-related adaptive sparse collision checks to guide lazy reverse search. These features enhance path cost efficiency and computation times while ensuring high success rates in confined scenarios. Through a series of simulations and real-world experiments, it is confirmed that MIT* outperforms existing single-query, sampling-based planners for problems in$\mathbb {R}^{4}$to$\mathbb {R}^{16}$and has been successfully applied to real-world robot manipulation tasks. A video showcasing our experimental results is available at:https://youtu.be/30RsBIdexTUNote to Practitioners—The motivation for this work stems from the challenges faced by existing informed path planners in high-dimensional, continuously valued environments, particularly when an initial feasible solution is difficult to find. Traditional asymmetric bidirectional planners rely on the best current solution to define problem subsets. When a lazy path has been found through lazy reverse search, these planners tend to re-sample and explore the entire problem space, which could hinder the path planning process. Our proposed MIT* algorithm addresses this issue by constructing an estimated informed set based on prior admissible solution costs before finding the initial solution. This estimated set helps to narrow the search area, thereby accelerating the initial convergence rate. MIT* also integrates an adaptive sampling strategy that dynamically adjusts based on the ongoing exploration process, enhancing the planner’s ability to efficiently navigate through challenging spaces. Furthermore, MIT* employs adaptive sparse collision checks, which guide the lazy reverse search that balances computational efficiency with accuracy in pathfinding. The proposed algorithm can be applied to industrial robots, humanoid robots, or service robots to achieve efficient path planning.

Traditional assembly tasks often require robots to transfer the acquired skills to new tasks. However, previous transfer reinforcement learning methods typically ignore the inherent relationship between the source and the target domain tasks. This requires a substantial amount of interaction data to compensate for this deficiency, and generally results in poor transfer effects. To address this issue, a strategy transfer method that establishes a shared feature space between the source domain and the target domain is proposed to enhance the efficiency of strategy learning on peg-in-hole assembly. Initially, by calculating the distance between each feature in the source and target domains, the features with small distance are selected as shared features. Subsequently, in order to determine the successful search state, this paper uses the jump state of contact force and the relative position between the peg and the hole as the judgment criterion. Lastly, search and insertion peg-in-hole assembly experiments are conducted to validate the generalization of the proposed strategy, demonstrating its capability to transfer from simulation to the real world.

This paper presents an effective and reliable pose tracking solution, termed ERPoT, for mobile robots operating in large-scale outdoor and challenging indoor environments, underpinned by an innovative prior polygon map. Especially, to overcome the challenge that arises as the map size grows with the expansion of the environment, the novel form of a prior map composed of multiple polygons is proposed. Benefiting from the use of polygons to concisely and accurately depict environmental occupancy, the prior polygon map achieves long-term reliable pose tracking while ensuring a compact form. More importantly, pose tracking is carried out under pure LiDAR mode, and the dense 3D point cloud is transformed into a sparse 2D scan through ground removal and obstacle selection. On this basis, a novel cost function for pose estimation through point-polygon matching is introduced, encompassing two distinct constraint forms: point-to-vertex and point-to-edge. In this study, our primary focus lies on two crucial aspects: lightweight and compact prior map construction, as well as effective and reliable robot pose tracking. Both aspects serve as the foundational pillars for future navigation across diverse mobile platforms equipped with different LiDAR sensors in varied environments. Comparative experiments based on the publicly available datasets and our self-recorded datasets are conducted, and evaluation results show the superior performance of ERPoT on reliability, prior map size, pose estimation error, and runtime over the other six approaches. The corresponding code can be accessed athttps://github.com/ghm0819/ERPoT, and the supplementary video is athttps://youtu.be/6XdcXyUrLKw.

It is challenging for the mobile robot to achieve autonomous and mapless navigation in the unknown environment with uneven terrain. In this study, we present a layered and systematic pipeline. At the local level, we maintain a tree structure that is dynamically extended with the navigation. This structure unifies the planning with the terrain identification. Besides, it contributes to explicitly identifying the hazardous areas on uneven terrain. In particular, certain nodes of the tree are consistently kept to form a sparse graph at the global level, which records the history of the exploration. A series of subgoals that can be obtained in the tree and the graph are utilized for leading the navigation. To determine a subgoal, we develop an evaluation method whose input elements can be efficiently obtained on the layered structure. We conduct both simulation and real-world experiments to evaluate the developed method and its key modules. The experimental results demonstrate the effectiveness and efficiency of our method. The robot can travel through the unknown uneven region safely and reach the target rapidly without a preconstructed map.

Global trajectory planning is crucial for long-range formation navigation tasks of multi-robot systems in efficiency improvement and energy saving, whose main challenges are the joint space constraints of the whole team and the long-range deployment. To overcome the above difficulties, we reformulate the original problem into an affine formation planning problem in parameter space. Further, we propose a front-end & back-end framework for global trajectory planning of Multi-Robot Systems (MRS) with affinely deformable formation. For the front-end, an RL-steering affine formation RRT* method is designed to search a global formation-level trajectory in affine parameter space, combining the efficient BVP-solving capability of RL and the global guidance and generalizing ability of RRT*. For the back-end, we propose a formationlevel affine parameter trajectory optimization method to refine the front-end trajectory, and further transform it into peragent trajectories for execution. Extensive benchmarks and ablation experiments in simulation show the effectiveness of our framework for the global trajectory generation of a multiUAV system with affinely deformable formation. The appendix can be seen here3.

Human-aware path planner is essential for achieving harmonious coexistence between humans and robots in highly dynamic environments. In this paper, we propose an integrated framework to find the optimal path in the complex environment with considering collision risk, social norms, and crowded areas. In the proposed framework, a general dynamic group model (g-space) based on the Gaussian Mixed Model (GMM) is proposed as the social norms of dynamic groups, which not only considers the factors of humans (e.g., pose, quantity, distribution, psychology) but also establishes the proximity and human interacting constraints of dynamic groups. An integrated Collision Risk and Human Space (CR&HS) model is applied to achieve human-acceptable behaviors, in which both collision avoidance, human comfort, and interference-free constraints have been involved. Moreover, an Improved Virtual Doppler Method (IVDM) has been used to realize safety navigation to avoid the robot falling into the crowded area. Finally, the proposed framework has been utilized with the sampling-based rapidly-exploring random tree. Experimental results demonstrate that the proposed method can generate the optimal human-aware collision-free path in complex environments. Note to Practitioners—This paper aims to plan an optimal trajectory for the robot in highly dynamic environments. In this field, it is still a challenging task to plan a trajectory with collision-free, human-aware, and crowd-aware. To do that, we present an integrated framework to generate the optimal trajectory by involving the collision risk, social norms, and human density. First, the g-space model is adopted as interference-free constraints of dynamic groups. The integrated knowledge fusion model (CR&HS) then penalizes the manners which have higher collision risk and adverse effects on human interaction or human comfortable. Besides, human motion and density are provided to a robot by IVDM. The proposed framework is utilized in the sampling-based rapidly-exploring random tree as the evaluation module. Finally, the feasibility and reliability of the proposed method have been verified by experiments in different simulated environments. The proposed framework can be applied in most mobile service robots to achieve human-friendly manners.

This study focuses on localization and mapping for UGVs when they are deployed in environments with non-level roads. In these scenarios, the vehicles need to travel through flat but not necessarily level grounds, i.e., ascent or descent, which may cause drifts of the robot pose and distortion of the map. We develop a low-drift LiDAR odometry and mapping approach for the UGV with LiDAR as the only exteroceptive sensor. A factor-graph based pose optimization method is developed with a specifically designed factor named slope factor. This factor includes the slope information that is estimated from a real-time LiDAR data stream. The slope information is also used to enhance the loop-closure detection procedure. Moreover, an incremental pitch estimation mechanism is designed to achieve further pose estimation refinement. We demonstrate the effectiveness of the developed framework in real-world environments. The odometry drift is lower and the map is more precise than experiments with the state-of-the-arts. Notably, on the Kitti dataset, our method also exhibits convincing performance, demonstrating its strength in more general application scenarios.

In environmental monitoring programs, mobile robots have been widely deployed for remote sensing, with the end objective of monitoring and mapping out environmental fields. Complex characteristics and correlations in natural phenomena make it challenging to establish a reliable framework for mobile sensing and field mapping. Furthermore, constraints of onboard resources will limit the ability of mobile robots to cover a large area. This article focuses on the active sensing problem in environmental field mapping and particularly exploits the use of intrinsic interactions among multivariate spatiotemporal data. A novel deep neural network of a hybrid CNN-RNN model is employed to learn the monitored multivariate spatiotemporal field. Specifically, a set of attention mechanisms is designed and embedded in the network, which is able to adaptively capture parameterwise dependencies among the monitored heterogeneous parameters and spatial correlations in geolocations of a surveyed field. The weights of inferred attention facilitate explicit interpretation of the driving parameters and geolocations. Some subregions of interest in the surveyed field are specified by their spatial attention distribution and are actively sensed by following the proposed coverage path planner. Experiments are carried out using a real-world dataset with multisource environmental imagery from a remote sensing program. Experimental results are obtained, which demonstrate the superior mapping performance of the proposed systematical methodology compared to baseline methods. Furthermore, the proposed model is able to quantitatively reveal the driving monitored parameters and geolocations in a regression process.Note to Practitioners—This article was motivated by the need for a practical and systematic approach for reconstruction and planning to execute robotic active sensing (AS) in environmental field mapping. Field robotic applications are not maneuverable in comparison with indoor scenarios due to severe conflict between the need for long execution endurance in the field and the very limited onboard resources. Traditional AS planners normally use statistical model-based informative metrics, which may lead to model misspecification in real-world phenomena. The developed framework in this article yields a novel attention-driven metric to guide AS and mapping. It relies on an attention-based hybrid neural network that reveals the driving variables in terms of the heterogeneities and complexities in a natural environment. The high-priority regions are maximized in a coverage path depending on the inferred spatial attention distribution while maintaining the travel cost of the sensing robots within an available energy budget. Experiments using a remote sensing dataset validate the reliable performance of the proposed framework, in environmental field mapping.

In this article, a novel informative path planning (IPP) framework is proposed for efficient robotic object search. We innovatively reformulate the object search into an IPP problem, which takes account of the knowledge of possible target object locations. To model the target object distribution knowledge, the semantic information of the focused environment is utilized to obtain the probabilities of finding the target object at possible locations. Then, the probability distribution is modeled by Gaussian mixture model (GMM) to generate an information map. Based on the map, a sampling-based IPP method is proposed to minimize the object search cost. It is worth noting that the object search path is planned with a tree structure and evaluated by a utility function that concerns both search information gain and path cost. Moreover, to improve the quality of the search path, a novel informative sampling strategy and a rewire mechanism are conceived. The performance of the proposed object search framework is fully evaluated through both simulation experiments and real-world tests with a mobile robot platform. Results demonstrated that our method can find the target object efficiently and robustly with shorter path length than three comparative methods in the literature and the mobile robot shows human-like behavior when searching for the target object.

As one of the most promising areas, mobile robots draw much attention these years. Current work in this field is often evaluated in a few manually designed scenarios, due to the lack of a common experimental platform. Meanwhile, with the recent development of deep learning techniques, some researchers attempt to apply learning-based methods to mobile robot tasks, which requires a substantial amount of data. To satisfy the underlying demand, in this paper we build HouseExpo, a large-scale indoor layout dataset containing 35, 126 2D floor plans including 252, 550 rooms in total. Together we develop PseudoSLAM, a lightweight and efficient simulation platform to accelerate the data generation procedure, thereby speeding up the training process. In our experiments, we build models to tackle obstacle avoidance and autonomous exploration from a learning perspective in simulation as well as real-world experiments to verify the effectiveness of our simulator and dataset. All the data and codes are available online and we hope HouseExpo and PseudoSLAM can feed the need for data and benefit the whole community.

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.Note to Practitioners-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

Rapidly random-exploring tree (RRT) and its variants are very popular due to their ability to quickly and efficiently explore the state space. However, they suffer sensitivity to the initial solution and slow convergence to the optimal solution, which means that they consume a lot of memory and time to find the optimal path. It is critical to quickly find a short path in many applications such as the autonomous vehicle with limited power/fuel. To overcome these limitations, we propose a novel optimal path planning algorithm based on the convolutional neural network (CNN), namely the neural RRT* (NRRT*). The NRRT* utilizes a nonuniform sampling distribution generated from a CNN model. The model is trained using quantities of successful path planning cases. In this article, we use the A* algorithm to generate the training data set consisting of the map information and the optimal path. For a given task, the proposed CNN model can predict the probability distribution of the optimal path on the map, which is used to guide the sampling process. The time cost and memory usage of the planned path are selected as the metric to demonstrate the effectiveness and efficiency of the NRRT*. The simulation results reveal that the NRRT* can achieve convincing performance compared with the state-of-the-art path planning algorithms. Note to Practitioners-The motivation of this article stems from the need to develop a fast and efficient path planning algorithm for practical applications such as autonomous driving, warehouse robot, and countless others. Sampling-based algorithms are widely used in these areas due to their good scalability and high efficiency. However, the quality of the initial path is not guaranteed and it takes much time to converge to the optimal path. To quickly obtain a high-quality initial path and accelerate the convergence speed, we propose the NRRT*. It utilizes a nonuniform sampling distribution and achieves better performance. The NRRT* can be also applied to other sampling-based algorithms for improved results in different applications.

Unmanned Aerial Vehicles (UAVs) have been implemented for environmental monitoring by using their capabilities of mobile sensing, autonomous navigation, and remote operation. However, in real-world applications, the limitations of on-board resources (e.g., power supply) of UAVs will constrain the coverage of the monitored area and the number of the acquired samples, which will hinder the performance of field estimation and mapping. Therefore, the issue of constrained resources calls for an efficient sampling planner to schedule UAV-based sensing tasks in environmental monitoring. This paper presents a mission planner of coverage sampling and path planning for a UAV-enabled mobile sensor to effectively explore and map an unknown environment that is modeled as a random field. The proposed planner can generate a coverage path with an optimal coverage density for exploratory sampling, and the associated energy cost is subjected to a power supply constraint. The performance of the developed framework is evaluated and compared with the existing state-of-the-art algorithms, using a real-world dataset that is collected from an environmental monitoring program as well as physical field experiments. The experimental results illustrate the reliability and accuracy of the presented coverage sampling planner in a prior survey for environmental exploration and field mapping.

In a human-robot coexisting environment, reaching the target place efficiently and safely is pivotal for a mobile service robot. In this paper, a Risk-based Dual-Tree Rapidly exploring Random Tree (Risk-DTRRT) algorithm is proposed for the robot motion planning in a dynamic environment, which provides a homotopy optimal trajectory on the basis of a heuristic trajectory. A dual-tree framework consisting of an RRT tree and a rewired tree is proposed for the trajectory searching. The RRT tree is a time-based tree, considering the future trajectory predictions of the pedestrians, and this tree is utilized to generate a heuristic trajectory. However, the heuristic trajectory is usually nonoptimal. Then, a line-of-sight (LoS) control checking algorithm is proposed to detect whether two time-based nodes can be rewired with the least cost. On the basis of the LoS control checking algorithm, a tree rewiring algorithm is proposed to optimize the heuristic trajectory. The tree generated in the tree rewiring process is called the rewired tree. The trajectory generated by the Risk-DTRRT algorithm proves to be optimal in the homotopy class of the heuristic trajectory. The navigation run time and the lengths of the planned trajectories are selected to demonstrate the effectiveness of the proposed algorithm. The experimental results in both simulation studies and real-world implementations reveal that our proposed method achieves convincing performance in both static and dynamic environments. Note to Practitioners-This paper is motivated by planning optimized trajectories for the mobile service robots in dynamic environments with pedestrians. In this area, the sampling-based motion planning algorithms have been widely used for their high efficiency and robustness. However, the real-time optimality of the motion planning cannot be guaranteed due to the challenges caused by the moving pedestrians. In this paper, we propose a dual-tree framework to solve this problem. First, a classic Rapidly exploring Random Tree (RRT) is constructed to generate a heuristic trajectory. Then, instead of reconnecting the nodes on the heuristic trajectory directly, a rewired tree is built to optimize the heuristic trajectory. This proposed dual-tree framework can fully exploit the information of the RRT tree and ensure the completeness of the motion planning. The proposed motion planning algorithm also considers the constraints of the nonholonomic mobile robots, and it can be applied in most mobile service robots to improve their motion planning quality.

In this paper, we propose a novel path planning framework for autonomous exploration in unknown environments using a mobile robot. A graph structure is incrementally constructed along with the exploration process. The structure is the road map that represents the topology of the explored environment. To construct the road map, we design a sampling strategy to get random points in the explored environment uniformly. A global path from the current location of the robot to the target area can be found on this road map efficiently. We utilize a lazy collision checking method that only checks the feasibility of the generated global path to improve the planning efficiency. The feasible global path is further optimized with our proposed trajectory optimization method considering the motion constraints of the robot. This mechanism can facilitate the path cost evaluation for the next best view selection. In order to select the next best target region, we propose a utility function that takes into account both the path cost and the information gain of a candidate target region. Moreover, we present a target reselection mechanism to evaluate the target region and reduce the extra path cost. The efficiency and effectiveness of our approach are demonstrated using a mobile robot in both simulation and real experimental studies.

In this paper, we study the problem of autonomous exploration in unknown indoor environments using mobile robot. We use mutual information (MI) to evaluate the information the robot would get at a certain location. In order to get the most informative sensing location, we first propose a sampling method that can get random sensing patches in free space. Each sensing patch is extended to informative locations to collect information with true values. Then we use Gaussian Markov Random Fields (GMRF) to model the distribution of MI in environment. Compared with the traditional methods that employ Gaussian Process (GP) model, GMRF is more efficient. MI of every sensing location can be estimated using the training sample patches and the established GMRF model. We utilize an efficient computation algorithm to estimate the GMRF model hyperparameters so as to speed up the computation. Besides the information gain of the candidates regions, the path cost is also considered in this work. We propose a utility function that can balance the path cost and the information gain the robot would collect. We tested our algorithm in both simulated and real experiment. The experiment results demonstrate that our proposed method can explore the environment efficiently with relatively shorter path length.

Exploration region selection is an essential decision making process in autonomous robot exploration task. While a majority of greedy methods are proposed to deal with this problem, few efforts are made to investigate the importance of predicting long-term planning. In this paper, we present an algorithm that utilizes deep reinforcement learning (DRL) to learn exploration knowledge over office blueprints, which enables the agent to predict a long-term visiting order for unexplored subregions. On the basis of this algorithm, we propose an exploration architecture that integrates a DRL model, a next-best-view (NBV) selection approach and a structural integrity measurement to further improve the exploration performance. At the end of this paper, we evaluate the proposed architecture against other methods on several new office maps, showing that the agent can efficiently explore uncertain regions with a shorter path and smarter behaviors.

Robots are increasingly operating in indoor environments designed for and shared with people. However, robots working safely and autonomously in uneven and unstructured environments still face great challenges. Many modern indoor environments are designed with wheelchair accessibility in mind. This presents an opportunity for wheeled robots to navigate through sloped areas while avoiding staircases. In this paper, we present an integrated software and hardware system for autonomous mobile robot navigation in uneven and unstructured indoor environments. This modular and reusable software framework incorporates capabilities of perception and navigation. Our robot first builds a 3D OctoMap representation for the uneven environment with the 3D mapping using wheel odometry, 2D laser and RGB-D data. Then we project multilayer 2D occupancy maps from OctoMap to generate the the traversable map based on layer differences. The safe traversable map serves as the input for efficient autonomous navigation. Furthermore, we employ a variable step size Rapidly Exploring Random Trees that could adjust the step size automatically, eliminating tuning step sizes according to environments. We conduct extensive experiments in simulation and real-world, demonstrating the efficacy and efficiency of our system. (Supplemented video link: https://youtu.be/6XJWcsH1fk0).

This paper introduces a generic framework for field surveillance using consumer rotorcrafts and ground vehicles. Building such an autonomous system comes with two key challenges in persistent perception and obstacle avoidance. We begin with explaining two core algorithms to solve the challenges: an auto-landing algorithm that enables a quadrotor to land on a moving ground vehicle at a speed of 6.00 m/s, and an obstacle avoidance algorithm that ensures the safety of the quadrotor during searching process. On the basis of these algorithms, the architecture and infrastructure of Hawkeye framework are presented as well. Hawkeye is designed to be a generic platform with extensibility that allows integration of other domain applications. We demonstrate the potential of Hawkeye framework in a simulated agriculture monitoring mission and report its performance at the end of the paper.