This article presents a novel curriculum framework named symmetric-response guided reinforcement learning (RL) for autopilot control of fixed-wing aircraft, driven by an adaptive bidirectional learning curve and a dynamic target scheduling mechanism. Unlike traditional methods with static or overly smoothed learning progressions, the proposed method dynamically adjusts the learning curve's slope in both directions based on historical reward trends, allowing the learning intensity to increase or decrease as needed. This bidirectional adjustment ensures that the agent is neither overwhelmed by excessively difficult tasks nor stagnated by too-easy ones, leading to better stability and faster convergence. Furthermore, dynamic target generation within an episode from static target constraints enables both reward amplification and implicit enforcement of maneuver rate constraints, improving learning efficiency without manual reward shaping. Experiments on trajectory tracking tasks show that the proposed controller achieves faster convergence, reduced overshoot, and more accurate tracking under turbulence. Further tests on waypoint navigation and dynamic pursuit demonstrate its superiority over the baseline, achieving more precise and timely interception. These results highlight the robustness and applicability of the controller to complex aerial missions such as autonomous air combat.

Focusing on a distributed aircraft manufacturing scenario that supports cloud-edge collaboration, this study formulates an extended Flexible Job Shop Scheduling Problem (EFJSP) incorporating task area partitioning and cross-region machine travel time constraints. The inclusion of additional constraints increases the complexity of the scheduling problem. To ensure both solution quality and computational efficiency, we propose an end-to-end deep reinforcement learning (DRL)-based approach. First, the EFJSP is represented as a disjunctive graph with node and edge features capturing area partitioning and machine travel time. A dual graph attention network (DGAT) is then employed to extract node embeddings from the operation and machine subgraphs. Machine processing and travel times are unified into a single temporal dimension and incorporated into the operation-machine features, which, together with the node embeddings, support high-quality decision-making. Finally, under the Proximal Policy Optimization (PPO) framework, a scale-invariant policy and value network interact iteratively with a simulation environment for training. Experiments on synthetic datasets show that the proposed method outperforms priority dispatching rules (PDRs), matches or exceeds exact methods within time limits. Additionally, it exhibits superior computational efficiency over exact methods and strong generalization capabilities for large-scale problems.

Task-oriented grasping (TOG) involves grasping specific parts of an object based on a task instruction. Existing methods generally integrate semantic analysis with grasp detection, resulting in low efficiency and poor generalization when dealing with new tasks and hardware. To address these problems, we propose SemSegGrasp, which reformulates TOG as a semantic segmentation problem based on point cloud and text matching. By decomposing TOG into semantic segmentation and grasp detection, SemSegGrasp can significantly enhance both the efficiency and generalization performance of TOG. Moreover, it can be combined with any off-the-shelf grasp detection algorithms in a plug-and-play manner. For semantic segmentation, SemSegGrasp first utilizes a Vision-Language Model (VLM) to generate local geometric descriptions of the target object. These descriptions are then fed into a Large Language Model (LLM) along with user instructions to obtain operational guidance. Subsequently, we separately encode the input point cloud and the operational guidance and obtain their features. Leveraging multi-head cross-attention, we conduct a matching process between these two types of features to predict the probability of each point serving as a TOG grasp point, i.e. semantic segmentation. Finally, the grasp pose is determined by fusing the segmentation results with the candidate poses generated by an existing grasp detection algorithm. Experimental results on the publicly available TaskGrasp dataset and a real-world setting show that our SemSegGrasp method achieves state-of-the-art performance, outperforming existing methods by at least 5% and 10% on new tasks, respectively.

Accurate and reliable online real-time localization and mapping are crucial for autonomous navigation of robot. Dynamic objects within the perception field can affect the accuracy of registration and localization, and also introduce ghost trail artifacts in the map, hindering robot planning and decision-making. While semantic segmentation can assist in perceiving object categories, it struggles to accurately segment moving objects. In this paper, we present SOLO-SMap, a real-time localization and static map construction framework based solely on LiDAR point cloud. We leverage semantic inference to identify potential dynamic points. And then, our instance-level true dynamic points removal is achieved by utilizing geometric rules based on moving point occlusion relationships and multi-object tracking (MOT) within a nearby temporal window in the pre-alignment stage. This design preserves stable static constraints while adhering to the static world model assumption of SLAM systems, benefiting accuracy and reducing drift, particularly in busy intersections. We evaluated the performance of SOLO-SMap in dynamic scenes on KITTI datasets and our self-made datasets, and conducted a comprehensive comparison with other methods, validating the effectiveness and robustness of the proposed method. A supplementary video can be accessed at https://www.youtube.com/watch?v=x-VKr3ag03M.

The tremendous potential has been demonstrated by large language models (LLMs) in intelligent decision-making problems, with unprecedented capabilities shown across diverse applications ranging from gaming AI systems to complex strategic planning frameworks. However, the StarCraft II platform, which has been widely adopted for validating decision-making algorithms in the past decade, has not yet provided substantial support for this emerging domain. To address issues that LLMs cannot interface with the hundreds of actions of the pysc2 backend and the lack of native support for multi-agent (MA) collaboration, we propose the LLM-PySC2 environment. This is the first environment that offers LLMs the complete pysc2 action space with sufficient multi-modal information and game Wiki knowledge. With an asynchronous query architecture, the environment efficiently interacts with LLMs that maintain a constant latency regardless of the scale of the agents' population. In the experiments, we evaluated LLMs' decision-making performance in both the macro-decision and micro-operation scenarios, with traditional StarCraft II Multi-Agent Challenge (SMAC) tasks and a series of new proposed. Results indicate that LLMs possess the potential to achieve victories in complex scenarios but cannot constantly generate correct decisions, especially in the recovered pysc2 action space and MA settings. Without task-relevant instructions, the pre-trained models suffer from issues such as hallucinations and inefficient collaboration. Our findings suggest that StarCraft II still challenges in the era of large models, revealing that there is a lot to do to develop an advanced LLM decision-making system, and the proposed LLM-PySC2 environment will support future development of LLM-based decision-making solutions.

Unmanned aerial vehicles (UAVs) are widely used in autonomous exploration, but their motion speed is underutilized due to inaccurate motion time cost evaluation and high computational cost. Existing methods either fail to consider UAV’s motion tendency and environment simultaneously or can’t ensure real-time planning in large 3-D environments. This paper presents a consistent, high-speed, and efficient online autonomous UAV exploration method. First, a motion primitive activated graph search method is proposed to fully take advantage of the UAV’s current velocity and acceleration. It improves motion time cost evaluation by simulating short-term motion tendencies with motion primitives and reduces the computational cost by searching on a voxel graph with a dynamic upper bound. Then, a minimum time trajectory to the optimal viewpoint with a non-zero terminal velocity constraint in a convex hull is optimized. Finally, an SE(3) coverage trajectory for unknown space around the exploration path is further optimized. Simulations in various environments with different speed settings show that the proposed method’s average UAV velocity is 18.6%−116.1% faster and its exploration efficiency is 13.8%−49.5% higher than state-of-the-art methods. Real-world tests verify its effectiveness. The source code of our method will be released at: https://github.com/NKU-MobFly-Robotics/HighStar.

To address the issues of inaccurate feature association and insufficient utilization of the complementarity between the visual and inertial information in visual-inertial navigation systems, this paper proposes a feature radiance field (FeRF)-based bidirectionally enhanced visual-inertial odometry (FeRF-BEVIO). FeRF is a novel method for describing environment features, incorporating both the grayscale and position of point and line features. It provides a unified framework for representing and storing features, regardless of whether their depth has converged. Then, a bidirectional visual-inertial enhancement method is designed based on FeRFs. Specifically, on one hand, the information of the inertial measurement unit (IMU) is utilized to aid visual feature association within FeRFs. On the other hand, the visual information is employed to refine the IMU parameter estimation. This bidirectional enhancement process is iterated to improve the integration of the visual and inertial data. At last, the system jointly optimizes the robot poses and point-line feature parameters within a sliding window and updates FeRFs accordingly. Comparative experiments on public datasets and in the real-world environments demonstrate that FeRF-BEVIO outperforms state-of-the-art visual-inertial odometry (VIO) systems in both accuracy and robustness. Therefore, FeRF-BEVIO is highly suitable for navigation and simultaneous localization and mapping (SLAM) of micro aerial vehicles (MAVs).

This article proposes a perception-complementarity-driven trajectory generating method for multiple unmanned aerial vehicles (UAVs), which can effectively enhance the visibility of the target for UAVs in unknown environments. rgb0,0,0Traditional methods often rely on prior maps or additional sensors to assist with obstacle avoidance. Nevertheless, these methods are not only costly but also fail to effectively avoid occlusions caused by obstacles. rgb0,0,0Different from existing methods, the trajectory planned by the proposed method not only enables the vision-based UAVs to maintain the perception of obstacles and the target on the one hand, but also preserves topological equivalence with the predicted target trajectory on the other hand. Specifically, a vision-based mutual observation approach among UAVs is proposed to enhance the overall perception capability of the UAVs system. On this basis, a target-guided collaborative trajectory planning method is proposed to ensure the planned collision-free trajectory for other UAV in the formation maintains target visibility. In addition, a trajectory feasibility assessment method is proposed to obtain the collaborative trajectory planned by the UAV at the optimal observation location. Finally, comparative simulations are conducted with three state-of-the-art methods, demonstrating the advantages of the proposed method in maintaining target visibility and tracking efficiency during the vision-based aerial tracking. The real-world experiment demonstrates the feasibility of the proposed method.Note to Practitioners—Most existing vision-based multi-UAV target tracking methods require additional prior maps or laser sensors to assist in obstacle avoidance. In practical tracking scenarios, the limited perception range of cameras poses significant challenges for UAVs in synchronously observing moving target and environmental obstacles. The article proposes a trajectory generation method based on perceptual complementarity to ensure that the generated trajectory effectively perceives surrounding obstacles while enhancing the observation capability of UAVs towards moving target. The key insight of this work is to utilize mutual observation among multiple UAVs to assist in planning collision-free tracking trajectories, ensuring safety during the tracking process. Building upon this, by preserving topological equivalence with the predicted target trajectory, there is an improvement in the target visibility ratio during the tracking process. Furthermore, a trajectory feasibility assessment method is proposed to obtain the optimal collaborative trajectory from the UAV positioned at the optimal location in the formation. The effectiveness of the proposed method in improving flight safety and target visibility is validated through comparative experiments.

In this paper, an efficient motion planning approach with grid-based generalized Voronoi diagrams (G$^2$VD) is newly proposed for mobile robots. Different from existing approaches, the novelty of this work is twofold: 1) a new state lattice-based path searching approach is proposed, in which the search space is reduced to a novel Voronoi corridor to further improve the search efficiency; 2) an efficient quadratic programming-based path smoothing approach is presented, wherein the clearance to obstacles is considered to improve the path clearance of hard-constrained path smoothing approaches. We validate the efficiency and smoothness of our approach in various challenging simulation scenarios and outdoor environments. It is shown that the computational efficiency is improved by 17.1% in the path searching stage, and path smoothing with the proposed approach is 6.6 times faster than an advanced sparse-banded structure-based path smoothing approach and 53.3 times faster than the popular timed-elastic-band planner. A video showing outdoor navigation on our campus is available at https://youtu.be/iMXGthgvp58.Note to Practitioners—This paper is motivated by the challenges of motion planning problems of mobile robots. An efficient motion planning approach called G$^2$VD planner is proposed by combining path searching, path smoothing, and time-optimal velocity planning. Extensive simulation and experimental results show the effectiveness of the proposed motion planning approach. However, the prediction information of dynamic obstacles is not incorporated in the proposed motion planner, thus the motion planner may be a bit sluggish in response to dynamic obstacles. Furthermore, we plan to integrate the intention/trajectory prediction of pedestrians/vehicles into the proposed framework to enhance the foreseeability of the motion planner.

This paper proposes a novel environment-adaptive motion planning framework for mobile robots, which utilizes deep reinforcement learning to dynamically adjust optimization objectives according to various environmental and robot-ego characteristics, greatly enhancing the adaptability and robustness compared to existing motion planning strategies. Our approach features a two-stage trajectory optimization algorithm that optimizes for smoothness, safety, and efficiency—elements critical in practical applications. Firstly, we propose a reinforcement learning algorithm that dynamically adjusts optimization objectives based on the environmental context, which carefully encodes the environment, coarse initial path and robot information into the observation space. Additionally, two techniques are designed to reduce the sim-to-real gap: 1) integrating classical optimization framework as the motion planning backbone; 2) using low-dimensional input in the learning component, which minimizes discrepancies between simulated and real-world conditions. With the hybrid strategy, not only the interpretability and stability of the classical motion planning pipeline is preserved, but also the adaptive capability of emerging DRL techniques is fully utilized. The efficacy of the proposed solution is demonstrated through extensive simulations and real-world experiments, showcasing superior performance in terms of safety and efficiency across various testing scenarios. Especially in unknown and cluttered real office environments, our approach significantly improve safety (17% increase in success rate) and efficiency (29% reduction in time cost), compared to the popular traditional methods. (Supplementary video link: https://youtu.be/ph0pDGpI864.).

Designing intelligent game strategies for autonomous air combat has suffered from the vast exploration space, lengthy decision-making process, and sparse rewards. Some existing approaches adopt the hierarchical framework to improve the exploration efficiency. However, in these methods, agents in different layers are typically trained independently and operate at fixed frequencies, which limits their performance and hampers their ability to respond to highly dynamic combat situations. In view of this, we present PJOH-TED2, a partial-joint-optimization-based hierarchical (PJOH) learning framework with a time-event dual-driven (TED2) mechanism, for one-on-one beyond-visual-range (BVR) air combat. Specifically, the PJOH learning framework embeds the partial joint optimization mechanism into hierarchical reinforcement learning (HRL), thus improving the exploration efficiency dramatically while enhancing the integration across hierarchical levels. Moreover, the TED2 mechanism combines the advantages of event-driven and time-driven methods, which promote the dynamic response speed of agents as well as avoid redundant actions. In addition, we evaluated this work through a series of games against the state-of-the-art (SOTA) methods in a high-fidelity air combat simulation environment. The results empirically demonstrate that the proposed approach outperforms four SOTA methods with a win rate of at least 71%. Finally, this approach achieved the 1st place in learning methods in the intelligent air game algorithm challenge (IAGAC) by the Chinese Institute of Command and Control among 43 teams.

VSLAM is one of the key technologies for indoor mobile robots, used to perceive the surrounding environment, achieve accurate positioning and mapping. However, traditional VSLAM algorithms based on the assumption of a static environment still face certain challenges. The movement, occlusion, and appearance changes of dynamic objects can lead to feature point-matching errors, making data association difficult and causing biases in motion estimation. In order to address this challenge, this paper proposes a dynamic feature point removal method and a closed-loop detection method for high dynamic scenes, aiming to effectively improve the robustness and positioning accuracy in dynamic environments. First, the YOLOv7-tiny object detection network and LK optical flow algorithm are combined to detect the dynamic area, and the adaptive threshold keyframe selection method is adopted to solve the problem of poor quality of keyframe caused by the existing heuristic threshold selection method. Then, this paper proposes a dynamic keyframe sequence creation method based on the angle difference between keyframes, which reduces the workload of loop back detection and accelerates the efficiency of loop back detection in the system. Next, the ParC_NetVLAD image matching algorithm is proposed. In this paper, ConvNeXt-Tiny network is used for feature extraction of images, and ParC-Net network and CBAM attention mechanism are added to the feature extraction network. Finally, NetVLAD is used to cluster the extracted local features to obtain global features that can represent images. Experiments are conducted on public TUM RGB-D datasets and in real-world situations. The proposed algorithm reduces the ATE (Absolute Trajectory Error) by 96.4% and the RPE (Relative Trajectory Error) by 82.8% on average in highly dynamic scenarios. In the Pittsburgh30k dataset, the average accuracy of loop closure detection has been improved by 2.6%.

This paper presents an effective and reliable pose tracking solution, termed ERPoT, for mobile robots operating in large-scale outdoor and challenging indoor environments, underpinned by an innovative prior polygon map. Especially, to overcome the challenge that arises as the map size grows with the expansion of the environment, the novel form of a prior map composed of multiple polygons is proposed. Benefiting from the use of polygons to concisely and accurately depict environmental occupancy, the prior polygon map achieves long-term reliable pose tracking while ensuring a compact form. More importantly, pose tracking is carried out under pure LiDAR mode, and the dense 3D point cloud is transformed into a sparse 2D scan through ground removal and obstacle selection. On this basis, a novel cost function for pose estimation through point-polygon matching is introduced, encompassing two distinct constraint forms: point-to-vertex and point-to-edge. In this study, our primary focus lies on two crucial aspects: lightweight and compact prior map construction, as well as effective and reliable robot pose tracking. Both aspects serve as the foundational pillars for future navigation across diverse mobile platforms equipped with different LiDAR sensors in varied environments. Comparative experiments based on the publicly available datasets and our self-recorded datasets are conducted, and evaluation results show the superior performance of ERPoT on reliability, prior map size, pose estimation error, and runtime over the other six approaches. The corresponding code can be accessed athttps://github.com/ghm0819/ERPoT, and the supplementary video is athttps://youtu.be/6XdcXyUrLKw.

Existing object-level SLAM methods often overlook the correspondence between semantic information and geometric features, resulting in a significant gap between them within SLAM frameworks. To tackle this issue, this paper proposes TiMoSLAM, a semantic-geometric tight-coupling monocular visual object SLAM system, which considers a rigorous correspondence between semantics and geometry across all steps of SLAM. Initially, a general Semantic Relation Graph (SRG) is developed to consistently represent semantic information alongside geometric features. Detailed analyses on complete constraints of the geometric feature combinations on estimation of 3D cuboid model are performed. Subsequently, a Compound Hypothesis Tree (CHT) is proposed to incrementally construct the object-specific SRG and concurrently estimate the 3D cuboid model of an object, ensuing semantic-geometric consistency in object representation and estimation. Special attention is given to the matching errors between geometric features and objects during the optimization of camera poses and object parameters. The effectiveness of this method is validated on various datasets, as well as in real-world environments.

Visual servoing (VS) is a widely used technique in industries where there are hundreds of robots, but it requires accurate camera calibration including camera intrinsic and extrinsic parameters. However, it is labour-intensive to calibrate robots one-by-one in practical use. In this paper, we propose a neural uncalibrated VS policy (NUVS) that can adapt to calibration disturbances with an adaption mechanism and a control-oriented guidance. It bridges the disturbance adaption of classical VS methods and the large convergence of learning-based VS methods. NUVS estimates the calibration embedding from past observations and servos to the desired pose under the supervision of a PBVS that can access the ground truth in simulation. With this adaption mechanism, NUVS outperforms the classical IBUVS algorithm when facing large initial camera pose offsets under the calibration disturbance. Supplementary material in: https://sites.google.com/view/neural-uncalibrated-vs

Efficient data transmission and reasonable task allocation are important to improve multi-robot exploration efficiency. However, most communication data types typically contain redundant information and thus require massive communication volume. Moreover, exploration-oriented task allocation is far from trivial and becomes even more challenging for resource-limited unmanned aerial vehicles (UAVs). In this paper, we propose a fast and communication-efficient multi-UAV exploration method for exploring large environments. We first design a multi-robot dynamic topological graph (MR-DTG) consisting of nodes representing the explored and exploring regions and edges connecting nodes. Supported by MR-DTG, our method achieves efficient communication by only transferring the necessary information required by exploration planning. To further improve the exploration efficiency, a hierarchical multi-UAV exploration method is devised using MR-DTG. Specifically, the graph Voronoi partition is used to allocate MR-DTG’s nodes to the closest UAVs, considering the actual motion cost, thus achieving reasonable task allocation. To our knowledge, this is the first work to address multi-UAV exploration using graph Voronoi partition. The proposed method is compared with a state-of-the-art method in simulations. The results show that the proposed method is able to reduce the exploration time and communication volume by up to 38.3% and 95.5%, respectively. Finally, the effectiveness of our method is validated in the real-world experiment with 6 UAVs. We will release the source code to benefit the community.

This paper proposes a mapping uncertainty-aware point-wise Lidar Inertial Odometry (LIO), which synthesizes the point-wise point-to-plane match and map refreshment into a probabilistic model. As a result, it can address the issue of mismatching during point registration and remove in-frame motion distortion of Lidar sensors. Specifically, the uncertainty-aware map is designed to embody the uncertainty of map geometric features (points and planes), which comes from the Lidar point measurement and pose estimation. Then the map can be modeled in a probabilistic form. In addition, the proposed framework refreshes map at each Lidar point measurement to timely revise geometric features and provide non-delayed map. On the basis, the probabilistic point-to-plane match method is designed to seek a corresponding plane for each Lidar point in point registration, which can enhance the effectiveness of match and provide adaptive observation noises for more accurate state estimation. Comparative experiments on various public datasets are conducted to demonstrate the superior performance of the proposed framework in terms of higher accuracy and better robustness.

In this paper, a novel multi-robot autonomous exploration approach CURE is proposed based on dynamic Voronoi diagrams and centroids of unknown connected regions. Compared with existing approaches, the novelty of this work is twofold: 1) Dynamic Voronoi diagram is used for partition of the space being explored to improve the efficiency of multi-robot exploration, and then a new parameter-insensitive utility function is elaborately designed to evaluate the information of centroids, which helps guide the robot to explore unknown regions. 2) A hierarchical framework consisting of global and local exploration windows for detecting centroids is designed, wherein the global exploration window is activated to find centroids to guide the robot exploration when there are no centroids in any one local exploration window. We validate the feasibility and exploration efficiency of the proposed approach in various complex simulation scenarios and challenging real-world tasks. All test results show that the exploration time consumption and path cost are reduced by up to 50.7% and 34.4%, respectively, compared with an advanced RRT-based multi-robot exploration approach. (Supplementary video link: https://youtu.be/P5jXKlGQOec)Note to Practitioners—This paper is motivated by the efficient multi-robot autonomous exploration problem. In some applications such as target search and disaster rescue, the information about the environment is totally unknown to the robots, and thus they are required to explore unknown environments autonomously. In this case, it is necessary to improve the efficiency of multi-robot exploration due to the time limitation of the task and the battery capacity. In this paper, a hierarchical framework is proposed to improve the efficiency of multi-robot autonomous exploration. Each robot only needs to explore the Voronoi partition it is responsible for and is guided to the unknown region by the centroid detected in the global and local exploration windows. Overall, the proposed approach can dramatically reduce the exploration time and path cost.

This article proposes a frontier-guided informative planner for unmanned aerial vehicle volumetric exploration and 3-D reconstruction, which can explore a complex unknown environment and provide the accurate truncated signed distance function reconstruction simultaneously. Different from the existing methods, the key insight of the proposed method is that the hybrid surface frontier is proposed to guide both the tree expansion of dynamic rapidly exploring random tree star and the informative trajectory generation. As a result, the proposed planner can achieve more efficient volumetric exploration with higher reconstruction quality. Specifically, hybrid global–local surface frontiers are designed to guide the potential viewpoints sampling and tree expansion, which results in directional exploration. Then, the hybrid surface frontiers are further leveraged to guide the candidate paths generation. On the basis, the path maximizing the new comprehensive gain is selected for the following B-spline trajectory optimization, which can further improve the reconstruction quality. Comparative simulation and real-world experiments are conducted to demonstrate the superior performance of the proposed method including the exploration efficiency and reconstruction quality.

Recent advances in trajectory planning have enabled quadrotor unmanned aerial vehicles (UAVs) to navigate autonomously in complex environments. However, most of the existing methods do not consider the perception quality and the safety simultaneously. This article proposes a perception-aware trajectory planning strategy for quadrotors, which can ensure the safety and localization accuracy. In contrast to the existing methods, the main idea of the proposed method lies in that the yaw angle trajectory is planned to actively obtain more information in the environment to improve the localization accuracy and keep the safe flight simultaneously. Following the mainstream two-stage motion planning framework, a coarse-to-fine graph search strategy is proposed to search for a safe and perception-aware yaw angle path in the first stage. Specifically, a Yaw Safety Corridor (YSC) is proposed to guarantee the safety, which can observe the obstacles directly along the tangent direction of the position trajectory. In addition, a dedicated map Fisher Information Field (FIF) is employed to evaluate the perception quality. In the second stage, a path-guided optimization method is proposed to quickly generate a safe and perception-aware trajectory. Finally, comparative simulation and real-world experiments are conducted to verify the superior performance in terms of the perception quality and the safety of the proposed method.

The perception-aware motion planning method based on the localization uncertainty has the potential to improve the localization accuracy for robot navigation. How-ever, most of the existing perception-aware methods pre-build a global feature map and can not generate the perception- aware trajectory in real time. This paper proposes a topology- guided perception-aware receding horizon trajectory generation method, which contains a topology-guided position trajectory generation and a perception-aware yaw angle trajectory generation. Specifically, a memorable active map is built by selectively storing the visual landmarks. After that, a library of candidate topological trajectories are generated, which are then evaluated in terms of the perception quality based on the active map, smoothness, collision possibility and feasibility. In addition, the yaw angle trajectory is obtained through a front-end multiple refined path search and a back-end path- guided trajectory optimization. Comparative simulation and real-world experiments are carried out to confirm that the proposed method can keep more visual features in view and reduce the localization error.

This article proposes a novel asymptotic robust control approach for rotorcraft unmanned aerial vehicles (UAVs), which can effectively eliminate the impact of external disturbances and the model uncertainties. Different from existing works, the proposed method alleviates the assumption that disturbances should have no variations in the existing observers for uncertainties. In addition, the equilibrium point of the entire observer-controller system is asymptotically stable without the assumption of the boundness of the outer-loop signals or the time-scale separation assumption. Specifically, two observer-based estimators are designed to estimate the model uncertainty and the external disturbance for the force and torque, respectively. On this basis, a nonlinear hierarchical tracking controller is then proposed with the feedforward compensated disturbance term. Despite the nonlinear coupled dynamics and the disturbances, a generic framework for the stability analysis is proposed to yield the asymptotic stability of the equilibrium point of the entire controller-observer system. Comparative experiments are conducted to show the superior performance of the proposed approach in terms of higher tracking accuracy and stronger robustness. Note to Practitioners—Most of the existing observer-based control approaches can only govern the rotorcraft closed-loop system to be ultimately uniformly bounded. The highly coupled dynamics and mismatched uncertainties in practice make the effective asymptotic robust control of rotorcrafts very challenging. A novel robust control approach for rotorcraft UAVs is proposed to yield the asymptotic stability of the equilibrium point despite the nonlinear coupled dynamics and the disturbances. The key insight of this work to guarantee the asymptotic stability of the system is that the nominal signals (i.e., the output of the nominal auxiliary dynamics) are fed back to the controller. In addition, the attitude error signal is proved to be exponentially convergent, which can further help prove the asymptotic stability of the entire controller-observer system. Comparative experiments are conducted to show the applicability of the proposed approach.

Multi-sensor fusion is a mainstream method for localization of unmanned systems. How to achieve 6-degrees of freedom (DOF) pose estimation of the system is challenging in GPS-denied environments. Although map-aided localization methods normally perform well on intelligent transportation systems, prior maps are unavailable in some GPS-denied scenes (e.g., dense forests, tunnels, and underground parking lots). In this paper, we present a robust and consistent monocular visual-inertial-depth odometry (VIDO) to perform 6-DOF pose estimation without the need of prior information. The system contains a visual-inertial subsystem (VIS) based on tightly coupled optimization in a sliding window and a depth subsystem (DS) based on the iterative closest point (ICP) estimation using 3D point clouds obtained by a LiDAR or depth camera. The uncertainties of the estimation results in VIS and DS are rigorously calculated to consider measurement noises of the sensors. The obtained uncertainty estimates are fed into a covariance intersection (CI) filter for pose fusion, and the fused pose is further refined in the mapping process. We perform experiments on public datasets, as well as in various real-world outdoor and indoor scenes to verify the performance on localization and mapping in urban areas with buildings and cars, off-road environments with rugged terrains, as well as indoor structured environments. The results show that the proposed method can provide both a robust 6-DOF pose estimate and a precise 3D map for fully autonomous navigation in different scenes without a prior map, which presents an attractive complement to map-aided automated driving.

For pose stabilization task of nonholonomic mobile robots, this article proposes a novel integrated interactive framework, bridging the gap between visual servoing and simultaneous localization and mapping (SLAM). The framework consists of two cooperative components, control module for servoing task and SLAM module for feedback signals estimation. In most visual servoing methods, feedback signals for the servoing controller are estimated by means of multiple-view geometry assuming the target scene being always within the camera field of view (FOV). To handle the challenge that the target scene gets out of view during servoing process, the desired image is associated with the initial map by a two-step strategy, and an incremental map is constructed to guarantee available feedback signals estimation. In addition, on the basis of the kinematic model of the mobile robot and velocities designed by the servo controller, the predicted pose is exploited to discard moving objects in the camera FOV, thus making the proposed framework effective in dynamic scenes. Experimental results operated in different scenes without prior information demonstrate the effectiveness of the proposed approach to handle the FOV problem and dynamic scenes.Note to Practitioners—Traditional visual servoing stabilization approaches usually require that the feature points in the target scene remain within the FOV of the camera for feedback signals calculation, which is often neglected. Motivated by the requirement of continuous feedback signals to the servo controller, the SLAM technique is introduced to relax the FOV constraint during the servoing process. A novel integrated interactive framework is proposed in this article to further increase the applicability of the servoing system in practice, in which the SLAM module is also redesigned for the flexibility in dynamic scenes. The SLAM module provides feedback signals for the servo controller; meanwhile, velocities designed by the servo controller are utilized for the prediction mechanism in the SLAM module to discard features on moving objects. Experiments validate the applicability of the proposed framework in different scenarios.

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach is newly presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and time-optimal velocity planning. Compared with existing approaches, the novelty of this work is twofold: 1) a novel heuristic-guided pruning strategy of motion primitives is proposed and fully integrated into the state lattice-based global path planner to further improve the computational efficiency of graph search, and 2) a new soft-constrained local path optimization approach is proposed, wherein the sparse-banded system structure of the underlying optimization problem is fully exploited to efficiently solve the problem. We validate the safety, smoothness, flexibility, and efficiency of our approach in various complex simulation scenarios and challenging real-world tasks. It is shown that the computational efficiency is improved by 66.21% in the global planning stage and the motion efficiency of the robot is improved by 22.87% compared with the recent quintic Bézier curve-based state space sampling approach. We name the proposed motion planning framework E$\mathbf {^{3}} $MoP, where the number 3 not only means our approach is a three-layer framework but also means the proposed approach is efficient in three stages. Note to Practitioners—This paper is motivated by the challenges of motion planning problems of mobile robots. A three-layer motion planning framework is proposed by combining global path planning, local path optimization, and time-optimal velocity planning. For mobile robot navigation applications in semi-structured environments, optimization-based local planners are recommended. Extensive simulation and experimental results show the effectiveness of the proposed motion planning framework. However, due to the non-convexity of the path optimization formulation, the proposed local planner may get stuck in local optima. In future research, we will concentrate on extending the proposed local path optimization approach with the theory of homology classes to maintain several homotopically distinct local paths and seek global optima.

In this article, a virtual-goal-guided rapidly exploring random tree (RRT)-based visual servoing approach is proposed for nonholonomic mobile robots to simultaneously satisfy the field-of-view (FOV) constraint and the velocity constraints during the motion toward the desired pose. The presented approach contains two parts: 1) trajectory planning in the scaled Euclidean space and 2) trajectory tracking control. For the trajectory planning part, a new virtual-goal-guided RRT algorithm is designed to guarantee the FOV constraint and the velocity constraints by iteratively exploring the scaled Euclidean space in the presence of unknown image depth. Specifically, a virtual goal directly behind the desired pose is set to guide the tree to extend laterally into the area wherein the robot is easier to satisfy the FOV constraint. In addition, the lateral extension of the tree also helps decrease the lateral error of the robot as much as possible. Following each successful extension toward the virtual goal node, a greedy extension from the newly explored node to the desired pose is attempted using a polar stabilization controller, so that the planned trajectory can accurately arrive at the desired pose. Each newly explored edge in the scaled space is projected into the image space to check for the FOV limit. For the visual tracking part, the final searched trajectory in the scaled space is first transformed into image feature trajectories, which are then tracked by an image-based visual tracking controller. Experiments validate the effectiveness of the proposed approach.

Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning researchers who want to compare the performance of a new local planner relative to many other state-of-the-art approaches as well as end users in the mobile robotics industry who want to select a local planner that performs best on some problems of interest. We elaborately design various simulation scenarios to challenge the applicability of local planners, including large-scale, partially unknown, and dynamic complex environments. Furthermore, three types of principled evaluation metrics are carefully designed to quantitatively evaluate the performance of local planners, wherein the safety, efficiency, and smoothness of motions are comprehensively considered. We present the application of the proposed benchmark in two popular open-source local planners to show the practicality of the benchmark. In addition, some insights and guidelines about the design and selection of local planners are also provided. The benchmark website [1] contains all data of the designed simulation scenarios, detailed descriptions of these scenarios, and example code.

Planes and edges are attractive features for simultaneous localization and mapping (SLAM) in indoor environments because they can be reliably extracted and are robust to illumination changes. However, it remains a challenging problem to seamlessly fuse two different kinds of features to avoid degeneracy and accurately estimate the camera motion. In this article, a plane-edge-SLAM system using an RGB-D sensor is developed to address the seamless fusion of planes and edges. Constraint analysis is first performed to obtain a quantitative measure of how the planes constrain the camera motion estimation. Then, using the results of the constraint analysis, an adaptive weighting algorithm is elaborately designed to achieve seamless fusion. Through the fusion of planes and edges, the solution to motion estimation is fully constrained, and the problem remains well-posed in all circumstances. In addition, a probabilistic plane fitting algorithm is proposed to fit a plane model to the noisy 3-D points. By exploiting the error model of the depth sensor, the proposed plane fitting is adaptive to various measurement noises corresponding to different depth measurements. As a result, the estimated plane parameters are more accurate and robust to the points with large uncertainties. Compared with the existing plane fitting methods, the proposed method definitely benefits the performance of motion estimation. The results of extensive experiments on public data sets and in real-world indoor scenes demonstrate that the plane-edge-SLAM system can achieve high accuracy and robustness.Note to Practitioners—This article is motivated by the robust localization and mapping for mobile robots. We suggest a novel simultaneous localization and mapping (SLAM) approach fusing the plane and edge features in indoor scenes (plane-edge-SLAM). This newly proposed approach works well in the textureless or dark scenes and is robust to the sensor noise. The experiments are carried out in various indoor scenes for mobile robots, and the results demonstrate the robustness and effectiveness of the proposed framework. In future work, we will address the fusion of other high-level features (for example, 3-D lines) and the active exploration of the environments.

Target recognition is a challenging task for human following of mobile service robots. In this paper, we combine the principal-component-analysis (PCA)-based face recognition with the tracking-learning-detection applied to the human face (Face-TLD) to obtain an improvement, named as IFace-TLD. The proposed IFace-TLD can significantly improve the discrimination ability of the Face-TLD for ambiguous facial appearances. To further deal with motion uncertainties of the human head, especially the sudden motion change, which makes face-based target recognition methods unstable or even loses the target, a skeleton-based model is introduced to improve the accuracy and robustness of the target recognition. Specifically, within a walk half-cycle, the skeleton features are extracted from the upper-body three-dimensional skeleton coordinates. Then, the extracted skeleton features are fed into the support vector data description (SVDD) to identify the target person when the IFace-TLD becomes invalid. The seamless fusion of the skeleton recognition and the IFace-TLD, named as the SIFace-TLD, significantly enhances the robustness in complex scenarios, especially for people tracking from both front and behind. To achieve a complete human following system, the particle filter (PF) is adopted for estimating the state of the human motion. And then, a controller is designed to maintain the relative position between the robot and the target. Experimental results demonstrate that the proposed IFace-TLD is more accurate and flexible than the original Face-TLD. And the SIFace-TLD shows a robust performance to human motion uncertainties. Moreover, the developed controller can achieve a satisfactory human following performance.

Visual place recognition is a challenging problem in simultaneous localization and mapping (SLAM) due to a large variability of the scene appearance. A place is usually described by a single-frame image in conventional place recognition algorithms. However, it is unlikely to completely describe the place appearance using a single frame image. Moreover, it is more sensitive to the change of environments. In this article, a novel image-sequence-based framework for place detection and recognition is proposed. Rather than a single frame image, a place is represented by an image sequence in this article. Position invariant robust feature (PIRF) descriptors are extracted from images and processed by the incremental bag-of-words (BoWs) for feature extraction. The robot automatically partitions the sequentially acquired images into different image sequences according to the change of the environmental appearance. Then, the echo state network (ESN) is applied to model each image sequence. The resultant states of the ESN are used as features of the corresponding image sequence for place recognition. The proposed method is evaluated on two public datasets. Experimental comparisons with the FAB-MAP 2.0 and SeqSLAM are conducted. Finally, a real-world experiment on place recognition with a mobile robot is performed to further verify the proposed method.

In this article, a novel real-time acceleration-continuous path-constrained trajectory planning algorithm is proposed with an appealing built-in tradeoff mechanism between the cruise motion and time-optimal motion. Different from existing approaches, the proposed approach smoothens time-optimal trajectories with bang-bang input structures to generate acceleration-continuous trajectories while preserving the completeness property. More importantly, a novel built-in tradeoff mechanism is proposed and embedded into the trajectory planning framework so that the proportion of the cruise motion and time-optimal motion can be flexibly adjusted by changing a user-specified functional parameter. Thus, the user can easily apply the trajectory planning algorithm for various tasks with different requirements on motion efficiency and cruise proportion. Moreover, it is shown that feasible trajectories are computed more quickly than optimal trajectories. Rigorous mathematical analysis and proofs are presented for those aforementioned theoretical results. Comparative simulations and experimental results on an omnidirectional wheeled mobile robot demonstrate that flexible tunings between the cruise and time-optimal motions can be achieved in a higher computational efficiency manner by the proposed algorithm. Note to Practitioners-This article is motivated by the time-optimal and smooth motion planning problem for mobile robots along given paths. Existing approaches generally use the piecewise polynomial interpolations to smoothen and adjust feasible trajectories. This article proposes a novel path-constrained trajectory planning approach, which preserves properties of completeness and a high-efficient tradeoff mechanism between the optimal and cruise motions when achieving a globally optimal and acceleration-continuous trajectory. Comparative experimental results with other methods show the effectiveness of the proposed approach. In future research, we will attempt to integrate the proposed approach with typical path planning methods to achieve a complete and high-efficient motion planning framework.

This article proposes a novel adaptive image-based output feedback visual servoing approach to control a multirotor to the desired pose by using a minimum onboard sensor suite, which consists of an inertial measurement unit and a monocular camera. Different from “perspective moment,” a new type of image feature is designed as “rotated perspective moment,” whose dynamics is independent of roll, pitch, and yaw rates. On this basis, a nonlinear adaptive observer is designed to estimate the scaled linear velocity, which is more accurate, since the observer does not involve noisy angular velocity measurements. Then, a novel image-based output feedback controller is proposed with the designed image features and the observer, wherein the new saturated integral terms of linear and angular velocity errors are introduced into the controller design, respectively, to compensate system uncertainties. As a result, the steady-state error is decreased considerably. In addition, without the assumption of the separation principle between the observer and the controller, the small-angle approximation, or the time-scale separation assumption, the error signals of image features, attitude, velocity, and observer estimation can all converge to the origin asymptotically, which is proven by rigorous Lyapunov analysis. Comparative experiments are conducted to show the superior performance of the proposed approach in terms of more accurate velocity estimation, smaller steady-state errors, and stronger robustness.

In this paper, a novel 2-D laser-based autonomous exploration approach for mobile robots is proposed, which is based on a novel polygon map construction approach and graph-based simultaneous localization and mapping (SLAM) with directional endpoint features. This approach is composed of three modules: graph-based SLAM using directional endpoint features, polygon map construction, and exploration. Different from existing approaches in the field of 2-D SLAM, the newly proposed 2-D graph-SLAM is based on 3-D “directional endpoint” features; on this basis, a well-known data structure “circular-doubly linked list” is applied to construct a novel polygon map for navigation. Note that it is efficient for circular-doubly linked list to initialize and update the polygon map. In addition, we propose a new information entropy calculation approach to quantify the entropy of the polygon map. Then for each candidate goal, we could obtain corresponding information gain and make next decision through collision detection. Comparative experimental results with respect to the well-known Gmapping and Karto SLAM are presented to show superior performance of the proposed graph-based SLAM. The autonomous exploration experiments in the office and hallway environments show the effectiveness of the proposed approach for robotic mapping and exploration tasks.

In this paper, a multilevel humanlike motion planning approach is proposed for indoor mobile robots. Compared with existing approaches, the novelty of this paper is twofold: 1) the proposed path planning framework is multilevel and humanlike to ensure both foreseeability and flexibility, wherein functions of human brain, eyes, and legs are corresponding to global path planning, sensor-level path planning, and action-level path planning, respectively, and 2) along the planned path, a new velocity-adjustable trajectory planning algorithm is put forward which is provably complete and time optimal considering multiple constraints from both the robot and the environment. Experimental results show that the proposed approach has a better performance in terms of efficiency, smoothness, foreseeability, and flexibility, and autonomous navigation is realized in large-scale, dynamic, partially unknown, and unstructured indoor environments.

This paper proposes a novel monocular visual servoing strategy, which can drive a wheeled mobile robot to the desired pose without a prerecorded desired image. Compared with existing methods that adopt the teaching pattern for visual regulation, this scheme can still work well in the situation that the desired image has not been previously acquired. Thus, with the aid of this method, it is more convenient for mobile robots to execute visual servoing tasks. Specifically, to deal with nonexistence of the desired image, the reference frame is craftily defined by taking advantage of visual targets and the planar motion constraint, and the pose estimation algorithm is designed for the mobile robot with respect to the reference frame. Then, an adaptive visual regulation controller is developed to drive the mobile robot to the intermediate frame, where the parameter updating law is constructed for the unknown feature height based on the concurrent learning framework. Stability analysis shows that regulation errors and height identification error can converge simultaneously. Afterwards, the mobile robot is driven to the metric desired pose with the identified feature height. Both simulation and experimental results are provided to validate the performance of this strategy.

In this paper, we propose an acceleration-level pseudo-dynamic visual servoing structure for the nonholonomic mobile robots, based on which we design two different adaptive controllers-backstepping and dynamic surface control (DSC) in the presence of unknown depth information. Different from existing kinematic controllers, which directly regard linear and angular velocities as control inputs, this paper designs acceleration control that is integrated to easily obtain smooth velocity signals to be accurately executed by the robot. Two controllers are designed and analyzed with Lyapunov techniques: 1) a backstepping controller yielding asymptotical stability and 2) a dynamic surface controller ensuring system errors to be ultimately uniformly bounded. The unknown depth is handled by designing an adaptive parameter estimation law in both methods. Finally, a comparison between backstepping and DSC is given based on the experimental results and the design procedures.

This paper develops an image-based visual servoing (IBVS) control strategy using model predictive control (MPC) to stabilize a physically constrained mobile robot. In IBVS strategy, ambiguity, and degeneracy problems of the homography and fundamental matrix-based algorithms can be avoided. Moreover, a synthetic error vector incorporating the advantages of IBVS and position-based visual servoing is defined that includes both the robot angle and image coordinates. By using linear system control theory, the kinematics of nonholonomic chained robotic systems can be transformed into a skew-symmetric form, and through introducing an exponential decay phase, the uncontrollable problem can be solved. Then, an MPC strategy is developed and, thereafter, iteratively transformed into a constrained quadratic programming (QP) problem. Subsequently, we utilize a primal-dual neural network (PDNN) to solve this QP problem. By using PDNN optimization, the cost function of MPC effectively converges to the exact optimal values. Finally, experimental studies on the actual robotic systems have been conducted to demonstrate the performance of the proposed approach.

In this paper, a new background modeling method called stacked multilayer self-organizing map background model (SMSOM-BM) is proposed, which presents several merits such as strong representative ability for complex scenarios, easy to use, and so on. In order to enhance the representative ability of the background model and make the parameters learned automatically, the recently developed idea of representative learning (or deep learning) is elegantly employed to extend the existing single-layer self-organizing map background model to a multilayer one (namely, the proposed SMSOM-BM). As a consequence, the SMSOM-BM gains several merits including strong representative ability to learn background model of challenging scenarios, and automatic determination for most network parameters. More specifically, every pixel is modeled by a SMSOM, and spatial consistency is considered at each layer. By introducing a novel over-layer filtering process, we can train the background model layer by layer in an efficient manner. Furthermore, for real-time performance consideration, we have implemented the proposed method using NVIDIA CUDA platform. Comparative experimental results show superior performance of the proposed approach.

To generate local trajectory between initial states and target states for autonomous vehicles, a feasible trajectory generation algorithm based on quartic Bézier curve is proposed. The problem of trajectory generation is firstly separated into generating continuous and bounded curvature profile to shape the trajectory and generating linear velocity profile to execute the trajectory. The curvature profile generation is further converted to an optimization problem with only 3 parameters owing to the specific properties of quartic Bézier curve. Sequential quadratic programming is employed to find optimal solution with respect to specific objective function. To avoid sideslip and ensure velocity-continuity and acceleration limits, the framework of linear velocity profile generation is also proposed. A simple profile with constant acceleration is also provided as an example. Simulation results on lane keeping and changing and path following demonstrate the capability and the real-time performance of the proposed algorithm.

An automatic method for solving the problem of view planning in high-resolution industrial inspection is presented. The method's goal is to maximize the visual coverage, and to minimize the number of cameras used for inspection. Using a CAD model of the object of interest, we define the scene-points and the viewpoints, with the later being the solution space. The problem formulation accurately encapsulates all the vision- and task-related requirements of the design process for inspection systems. We use a graph-based approach to formulate a solution for the problem. The solution is implemented as a greedy algorithm, and the method is validated through experiments.

Many theoretic approaches for feedback stabilization control of nonholonomic mobile robots cannot be directly applied to practical robots since various kinematic constraints such as the velocity and acceleration limits are not considered in existing methods. To deal with this issue, we aim to propose a generic approach which first uses an (arbitrary) feedback stabilizer to generate the `path' and then rebuilt the corresponding `trajectory' along this `path' to meet various kinematic constraints, which ultimately gives a practical satisfactory solution for local trajectory planning. Specifically, a general framework is established to transform feedback stabilizers into a feasible and highly efficient trajectory planner by using path generation and optimal velocity planning techniques, considering both kinematic and differential constraints. Extensive simulation results are provided to validate the proposed approach.

The problem that using visual feedback to control the distance and orientation of the mobile robot with respect to a static sphere object is considered in this paper. Firstly, a unit virtual sphere is added on the classical camera model to obtain accurately the direction of the object. After measurable signal analysis, the kinematics model of the system is obtained. Then a switched controller and a continuous adaptive controller are developed to drive the mobile robot to the desired pose. Lastly, in order to estimate the distance between the camera and the object, a nonlinear observer is designed to give an exact estimation for the radius of the object, thus no metric information of the object is needed. Simulation results are collected to validate the effectiveness of the proposed method.

This paper designs an innovative prediction-based interception control strategy to enable a wheeled mobile robot to intercept a dynamic target with a specified angle, which can be potentially utilized in such applications as service robots. Specifically, visual information is collected and then utilized to estimate the state of the moving target, based on which, the follow-up pose of the target is calculated so as to improve the interception accuracy. A prediction-based controller is then proposed to drive the wheeled robot to efficiently intercept a dynamic target with a specified angle, whose stability is proven by Lyapunov techniques. Both simulation and experimental results are provided to demonstrate the superior performance of the proposed approach.

In this paper, an uncalibrated visual servo regulation strategy is designed for a nonholonomic mobile robot equipped with an eye-in-hand camera, which drives the mobile robot to the target pose with exponential convergence. Specifically, a novel fundamental matrix-based algorithm is firstly proposed to rotate the robot to point toward the desired position, with the camera intrinsic parameters estimated simultaneously by employing the fundamental matrix and a projection homography matrix. Subsequently, by utilizing the obtained camera intrinsic parameters, a straight-line motion controller is developed to drive the robot to the desired position, with the orientation of the robot always facing the target position. Another pure rotation controller is finally adopted to correct the orientation error. The exponentially convergent properties of the visual servo errors are proven with mathematical analysis. The performance of the proposed uncalibrated visual servo regulation method is further validated by simulation results.

Inspired by the desire to achieve fast payload transportation as well as sufficient swing suppression, a novel phase plane based motion planning method is proposed for underactuated overhead cranes. Specifically, the variation law of the underactuated system states in the phase plane is firstly derived via mathematical analysis for the phase portraits. Based on this, an analytical three-segment acceleration trajectory (namely, a trapezoid velocity trajectory) with the coupling behavior being taken into consideration is obtained under actual crane control constraints. To deal with the jerk (discontinuity) problem, we then present two modified acceleration trajectories by introducing some transition stages and performing some rigorous analysis. Moreover, the trajectories generated by the proposed method can evaluate the maximum payload swing and the arrival time for a given transportation task in advance, which provides essential control indexes for crane operation. Simulation results are provided to illustrate the superior performance of the proposed trajectory planning method.

A 2-1/2-D visual servoing strategy, which is based on a novel motion-estimation technique, is presented for the stabilization of a nonholonomic mobile robot (which is also called the “parking problem”). By taking into account the planar motion constraint of mobile robots, the proposed motion-estimation technique can be applied in both planar and nonplanar scenes. In addition, this approach requires no matrix estimation or decomposition, and it avoids ambiguity and degeneracy problems for the homography or fundamental matrix-based algorithms. Moreover, the field-of-view (FOV) constraint of the onboard camera is largely alleviated because the presented algorithm works well with few feature points. In order to incorporate the advantages of position-based visual servoing and image-based visual servoing, a composite error vector is defined that includes both image signals and the estimated rotational angle. Subsequently, a smooth time-varying feedback controller is adopted to cope with the nonholonomic constraints, which yields global exponential convergent rate for the closed-loop system. On the basis of the perturbed linear system theory, we show that practical exponential stability can be achieved, despite the lack of depth information, which is inherent for monocular camera systems. Both simulation and experiment results are collected to investigate the feasibility of the proposed approach.