This paper proposes an observer-based output feedback controller that enforces triply damped dynamics in servo drives. This makes transients softer as well as the model less dependent. The primary contributions are as follows: (a) The observer gain, structured by virtual damping (VD) and a convergence rate, enables the estimation loop to be governed by a first-order transfer function without requiring any system model information; (b) a high-order disturbance observer (HODOB) exponentially estimates high-frequency disturbances in accordance with the triple-damping transfer function; and (c) a proportional–integral-type controller based on the observer and HODOB, incorporating VD terms, guarantees exponential recovery of the desired triply damped performance. Experimental results on a commercial servo drive confirm the practical effectiveness of the proposed method across varying load conditions.

Particle image-based fluid measurement techniques are widely used to study complex flows in nature and industrial processes. Despite that particle tracking velocimetry (PTV) has shown potential in various experimental applications for quantitatively capturing unsteady flow characteristics, estimating fluid motion with long displacement and high particle density remains challenging. We propose an artificial-intelligence-enhanced PTV framework to track particle trajectories from consecutive images. The proposed framework, called GOTrack+ (a learning framework with graph optimal transport for particle tracking velocimetry), contains three components: a convolutional neural network-based particle detector for particle recognition and sub-pixel coordinate localization; a graph neural network-based initial displacement predictor for fluid motion estimation; and a graph-based optimal transport particle tracker for continuous particle trajectory linking. Each component of GOTrack+ can be extracted and used independently, not only to enhance classical PTV algorithms but also as a simple, fast, accurate, and robust alternative to traditional PTV programs. Comprehensive evaluations, including numerical simulations and real-world experiments, have shown that GOTrack+ achieves state-of-the-art performance compared to recent PTV approaches. All the codes are available at https://github.com/wuwuwuas/GOTrack.git.

This article proposes a multiloop feedback system that utilizes the virtual damping (VD) injection technique to ensure critically damped input–output behavior for each quadcopter in a multiagent topology. The proposed framework also considers system nonlinearity and model–plant mismatches to ensure robustness while maintaining a simple proportional–derivative (PD) controller design for convenient industrialization. The benefits of the proposed system are threefold. First, the observer allows both the outer and inner loops to feed back the time-derivative components of the position and attitude measurements, without requiring any system model information. Second, the PD-type adaptive controllers for each loop incorporate observer-based VD components into the feedforward and feedback loops to attenuate disturbances. Third, based on a nonlinear combination of two design parameters for VD and the desired convergence rate, the critically damped transfer function for the closed loop is determined using the order-reduction technique. A hardware testbed with the three quadcopters confirmed the practical advantages of the proposed solution in maintaining specified formations.

Transporting a heavy payload using multiple aerial robots (MARs) is an efficient manner to extend the load capacity of a single aerial robot. However, existing planning schemes for the multiple aerial robots transportation system (MARTS) still lack the capability to generate a collision-free and dynamically feasible trajectory in real-time. Therefore, they are limited to low-agility transportation in simple environments. To bridge the gap, we propose a complete planning scheme for the MARTS, achieving safe and agile aerial transportation (SAAT) of a cable-suspended payload in complex environments. Flatness map for the motor's revolutions per minute (RPM) of the aerial robot, considering the complete kinematic constraint and the dynamical coupling between each aerial robot and payload, is derived. To improve the responsiveness for the generation of the safe, dynamically feasible, and agile trajectory in complex environments, a real-time spatio-temporal trajectory planning scheme is proposed for the MARTS. Besides, we break away from the reliance on the state measurement for both the payload and cable, as well as the closed-loop control for the payload, and integrate a fully distributed control scheme to track the agile trajectory that is robust against imprecise payload mass, non-point mass payload, wind disturbances, and communication delays. The proposed schemes are extensively validated through benchmark comparisons, ablation studies, and simulations. Finally, extensive real-world experiments are conducted on practical MARTSs containing different numbers of aerial robots with onboard computers and sensors. The result validates the efficiency and robustness of our proposed schemes for the SAAT in complex environments.

Obtaining high-precision aerodynamics in the automotive industry relies on large-scale simulations with computational fluid dynamics, which are generally time-consuming and computationally expensive. Recent advances in operator learning for partial differential equations offer promising improvements in terms of efficiency. However, capturing intricate physical correlations from extensive and varying geometries while balancing large-scale discretization and computational costs remains a significant challenge. To address these issues, we propose **AeroGTO**, an efficient graph-transformer operator designed specifically for learning large-scale aerodynamics in engineering applications. AeroGTO combines local feature extraction through message passing and global correlation capturing via projection-inspired attention, employing a frequency-enhanced graph neural network augmented with k-nearest neighbors to handle three-dimensional (3D) irregular geometries. Moreover, the transformer architecture adeptly manages multi-level dependencies with only linear complexity concerning the number of mesh points, enabling fast inference of the model. Given a car's 3D mesh, AeroGTO accurately predicts surface pressure and estimates drag. In comparisons with five advanced models, AeroGTO is extensively tested on two industry-standard benchmarks, Ahmed-Body and DrivAerNet, achieving a 7.36% improvement in surface pressure prediction and a 10.71% boost in drag coefficient estimation, with fewer FLOPs and only 1% of the parameters used by the prior leading method.

Terrestrial-aerial bimodal vehicles (TABVs) can fly to avoid obstacles and move safely on the ground to save energy, offering enhanced adaptability and flexibility in various challenging environments. However, a robust localization approach becomes a bottleneck to stably applying the TABVs in real-world tasks. Besides the general limitations of visual SLAM methods, large FoV differences between the two modes, abrupt motion strikes in mode transitions, and unstable attitude in ground mode pose great challenges. In this paper, we present an environment-aware robust localization system specifically designed for passive-wheel-based TABVs, which feature two passive wheels alongside a standard quadrotor. The localization system tightly integrates data from multiple sensors, including a stereo camera, Inertial Measurement Units (IMUs), encoders, and single-point laser distance sensors. First, we introduce a terrain-aware odometer model that accurately estimates terrain slope and vehicle's velocity. Then, we propose an anomaly-aware method that senses anomalous sensors and dynamically adjusts the optimization weights accordingly. By explicitly estimating the environmental conditions, such as ground terrain slopes and visual information qualities, the robot can achieve accurate and robust localization results on the ground. To validate our localization approach, we conducted extensive experiments across various challenging scenarios, demonstrating the effectiveness and reliability of our system for real-world applications.

Terrestrial and aerial bimodal vehicles have gained significant interest due to their energy efficiency and versatile maneuverability across different domains. However, most existing passive-wheeled bimodal vehicles rely on attitude regulation to generate forward thrust, which inevitably results in energy waste on producing lifting force. In this work, we propose a novel passive-wheeled bimodal vehicle called TrofyBot that can rapidly change the thrust direction with a single servo motor and a transformable parallelogram linkage mechanism (TPLM). Cooperating with a bidirectional force generation module (BFGM) for motors to produce bidirectional thrust, the robot achieves flexible mobility as a differential driven rover on the ground. This design achieves 95.37% energy saving efficiency in terrestrial locomotion, allowing the robot continuously move on the ground for more than two hours in current setup. Furthermore, the design obviates the need for attitude regulation and therefore provides a stable sensor field of view (FoV). We model the bimodal dynamics for the system, analyze its differential flatness property, and design a controller based on hybrid model predictive control for trajectory tracking. A prototype is built and extensive experiments are conducted to verify the design and the proposed controller, which achieves high energy efficiency and seamless transition between modes.

With the increasing integration of robots into human life, their role in architectural spaces where people spend most of their time has become more prominent. While motion capabilities and accurate localization for automated robots have rapidly developed, the challenge remains to generate efficient, smooth, comprehensive, and high-quality trajectories in these areas. In this paper, we propose a novel efficient planner for ground robots to autonomously navigate in large complex multi-layered architectural spaces. Considering that traversable regions typically include ground, slopes, and stairs, which are planar or nearly planar structures, we simplify the problem to navigation within and between complex intersecting planes. We first extract traversable planes from 3D point clouds through segmenting, merging, classifying, and connecting to build a plane-graph, which is lightweight but fully represents the traversable regions. We then build a trajectory optimization based on motion state trajectory and fully consider special constraints when crossing multi-layer planes to maximize the robot's maneuverability. We conduct experiments in simulated environments and test on a CubeTrack robot in real-world scenarios, validating the method's effectiveness and practicality.

Autonomous navigation of car-like robots on uneven terrain poses unique challenges compared to flat terrain, particularly in traversability assessment and terrain-associated kinematic modelling for motion planning. This paper introduces SEB-Naver, a novel SE(2)-based local navigation framework designed to overcome these challenges. First, we propose an efficient traversability assessment method for SE(2) grids, leveraging GPU parallel computing to enable real-time updates and maintenance of local maps. Second, inspired by differential flatness, we present an optimization-based trajectory planning method that integrates terrain-associated kinematic models, significantly improving both planning efficiency and trajectory quality. Finally, we unify these components into SEB-Naver, achieving real-time terrain assessment and trajectory optimization. Extensive simulations and real-world experiments demonstrate the effectiveness and efficiency of our approach. The code is at https://github.com/ZJU-FAST-Lab/seb_naver.

Terrain analysis is critical for the practical application of ground mobile robots in real-world tasks, especially in outdoor unstructured environments. In this paper, we propose a novel spatial-temporal traversability assessment method, which aims to enable autonomous robots to effectively navigate through complex terrains. Our approach utilizes sparse Gaussian processes (SGP) to extract geometric features (curvature, gradient, elevation, etc.) directly from point cloud scans. These features are then used to construct a high-resolution local traversability map. Then, we design a spatial-temporal Bayesian Gaussian kernel (BGK) inference method to dynamically evaluate traversability scores, integrating historical and real-time data while considering factors such as slope, flatness, gradient, and uncertainty metrics. GPU acceleration is applied in the feature extraction step, and the system achieves real-time performance. Extensive simulation experiments across diverse terrain scenarios demonstrate that our method outperforms SOTA approaches in both accuracy and computational efficiency. Additionally, we develop an autonomous navigation framework integrated with the traversability map and validate it with a differential driven vehicle in complex outdoor environments. Our code will be open-source for further research and development by the community, https://github.com/ZJU-FAST-Lab/FSGP_BGK.

This study developed a multisensor asynchronous fusion method for vision cameras and laser sensors for real-time tracking of a millimeter-scale piezoelectric microrobot with high sampling rates. The design, manufacture, and realization of the proposed microrobot using a monolithic integrated manufacturing process is realized with a size of 36×36×18 mm and a weight of 4.9 g. Two local estimators for the vision camera and laser sensors are established independently for 2-D tracking of the millimeter-scale microrobot. An asynchronous fusion method for the camera and laser sensors is developed by optimally weighting the two local estimators in a linear minimum variance framework using the Lagrange multiplier method at the synchronized update time. High-precision 2-D trajectory tracking of the proposed microrobot was achieved with an average velocity of 301.6 mm/s and a sampling rate of up to 4000 Hz. Experiments tracking different trajectories of the microrobot at different velocities and sampling rates demonstrated the effectiveness of the proposed fusion method.

The tractor-trailer robot consists of a drivable tractor and one or more non-drivable trailers connected via hitches. Compared to typical car-like robots, the addition of trailers provides greater transportation capability. However, this also complicates motion planning due to the robot’s complex kinematics, high-dimensional state space, and deformable structure. To efficiently plan safe, time-optimal trajectories that adhere to the kinematic constraints of the robot and address the challenges posed by its unique features, this paper introduces a lightweight, compact, and high-order smooth trajectory representation for tractor-trailer robots. Based on it, we design an efficiently solvable spatial-temporal trajectory optimization problem. To deal with deformable structures, which leads to difficulties in collision avoidance, we fully leverage the collision-free regions of the environment, directly applying deformations to trajectories in continuous space. This approach not requires constructing safe regions from the environment using convex approximations through collision-free seed points before each optimization, avoiding the loss of the solution space, thus reducing the dependency of the optimization on initial values. Moreover, a multi-terminal fast path search algorithm is proposed to generate the initial values for optimization. Extensive simulation experiments demonstrate that our approach achieves severalfold improvements in efficiency compared to existing algorithms, while also ensuring lower curvature and trajectory duration. Real-world experiments involving the transportation, loading and unloading of goods in both indoor and outdoor scenarios further validate the effectiveness of our method. The source code is accessible at https://github.com/Tracailer/Tracailer.

Differential drive robots are widely used in various scenarios thanks to their straightforward principle, from household service robots to disaster response field robots. The nonholonomic dynamics and possible lateral slip of these robots lead to difficulty in getting feasible and high-quality trajectories. Although there are several types of driving mechanisms for real-world applications, they all share a similar driving principle, which involves controlling the relative motion of independently actuated tracks or wheels to achieve both linear and angular movement. Therefore, a comprehensive trajectory optimization to compute trajectories efficiently for various kinds of differential drive robots is highly desirable. In this paper, we propose a universal trajectory optimization framework, enabling the generation of high-quality trajectories within a restricted computational timeframe for these robots. We introduce a novel trajectory representation based on polynomial parameterization of motion states or their integrals, such as angular and linear velocities, which inherently matches the robots’ motion to the control principle. The trajectory optimization problem is formulated to minimize computation complexity while prioritizing safety and operational efficiency. We then build a full-stack autonomous planning and control system to demonstrate its feasibility and robustness. We conduct extensive simulations and real-world testing in crowded environments with three kinds of differential drive robots to validate the effectiveness of our approach.Note to Practitioners—The Differential drive robot, known for its simple mechanics and high maneuverability, is widely used in many applications. However, current methods have limitations in practice when high-performance motion is needed. Due to the state representation in Cartesian space, path planning makes it difficult to consider nonholonomic constraints directly. The existing trajectory optimization cannot effectively constrain the angular velocity and it is difficult to model forward and backward motion into a continuous trajectory. This paper provides a novel trajectory representation that inherently utilizes the motion performance of differential drive robots, which ensures its universality for different platforms, and reduces the time required to generate trajectories to ensure real-time performance. Based on this, we propose a robust planning and control framework to achieve efficient navigation. We release the source code athttps://zju-fast-lab.github.io/DDR-opt/facilitating expansion and deployment for practitioners. We validate this framework through extensive experiments, demonstrating its capability to navigate challenging environments.

Due to the limited online computational resources and the inherent probability of hardware and software failures of real-world robots, large-scale formation planning faces two common challenges: computational intractability and agent failures. Based on the theory of sparse graphs and the maximum clique, we achieve a resilient and efficient formation planning (RE-Formation) to address these issues. To improve the computational efficiency of trajectory planning while ensuring flexible formation maneuvers, we introduce sparse graphs to describe connection relationships and present a sparse graph construction method with closed-form solutions. The sparse graphs ensure theGlobalRigidity for uniquely corresponding to a geometric shape andPreserve the mainFeatures of complete graphs, denoted as the GRPF sparse graph. To prevent the impact of abnormal agents, the problem of eliminating abnormal agents is transformed into an outlier rejection problem that can be solved by computing the maximum clique. We approximate the maximum clique by periodically triggering the calculation of the maximumk-coreto meet the real-time computational demands of large-scale swarms. We validate the performance through real-world experiments and implement formation planning with 100 drones in simulation. Benchmark comparisons and ablation experiments demonstrate the effectiveness of our method.

Achieving large-scale aerial swarms is challenging due to the inherent contradictions in balancing computational efficiency and scalability. This paper introducesPrimitive-Swarm, an ultra-lightweight and scalable planner designed specifically for large-scale autonomous aerial swarms. The proposed approach adopts a decentralized and asynchronous replanning strategy. Within it is a novel motion primitive library consisting of time-optimal and dynamically feasible trajectories. They are generated utlizing a novel time-optimial path parameterization algorithm based on reachability analysis (TOPP-RA). Then, a rapid collision checking mechanism is developed by associating the motion primitives with the discrete surrounding space according to conflicts. By considering both spatial and temporal conflicts, the mechanism handles robot-obstacle and robot-robot collisions simultaneously. Then, during a replanning process, each robot selects the safe and minimum cost trajectory from the library based on user-defined requirements. Both the time-optimal motion primitive library and the occupancy information are computed offline, turning a time-consuming optimization problem into a linear-complexity selection problem. This enables the planner to comprehensively explore the non-convex, discontinuous 3-D safe space filled with numerous obstacles and robots, effectively identifying the best hidden path. Benchmark comparisons demonstrate that our method achieves the shortest flight time and traveled distance with a computation time of less than 1 ms in dense environments. Super large-scale swarm simulations, involving up to 1000 robots, running in real-time, verify the scalability of our method. Real-world experiments validate the feasibility and robustness of our approach. The code will be released to foster community collaboration.

Convex polytopes have compact representations and exhibit convexity, which makes them suitable for abstracting obstacle-free spaces from various environments. Existing generation methods struggle with balancing high-quality output and efficiency. Moreover, another crucial requirement for convex polytopes to accurately contain certain seed point sets, such as a robot or a front-end path, is proposed in various tasks, which we refer to as manageability. In this paper, we propose Fast Iterative Regional Inflation (FIRI) to generate high-quality convex polytope while ensuring efficiency and manageability simultaneously. FIRI consists of two iteratively executed submodules: Restrictive Inflation (RsI) and Maximum Volume Inscribed Ellipsoid (MVIE) computation. By explicitly incorporating constraints that include the seed point set, RsI guarantees manageability. Meanwhile, iterative MVIE optimization ensures high-quality result through monotonic volume bound improvement. In terms of efficiency, we design methods tailored to the low-dimensional and multi-constrained nature of both modules, resulting in orders of magnitude improvement compared to generic solvers. Notably, in 2-D MVIE, we present the first linear-complexity analytical algorithm for maximum area inscribed ellipse, further enhancing the performance in 2-D cases. Extensive benchmarks conducted against state-of-the-art methods validate the superior performance of FIRI in terms of quality, manageability, and efficiency. Furthermore, various real-world applications showcase the generality and practicality of FIRI. The high-performance code of FIRI will be open-sourced.

Perception is necessary for autonomous navigation in an unknown area crowded with obstacles. It’s challenging for a robot to navigate safely without any sensors that can sense the environment, resulting in a blind robot, and becomes more difficult when comes to a group of robots. However, it could be costly to equip all robots with expensive perception or SLAM systems. In this paper, we propose a novel system named ColAG, to solve the problem of autonomous navigation for a group of blind UGVs by introducing cooperation with one UAV, which is the only robot that has full perception capabilities in the group. The UAV uses SLAM for its odometry and mapping while sharing this information with UGVs via limited relative pose estimation. The UGVs plan their trajectories in the received map and predict possible failures caused by the uncertainty of its wheel odometry and unknown risky areas. The UAV dynamically schedules waypoints to prevent UGVs from collisions, formulated as a Vehicle Routing Problem with Time Windows to optimize the UAV’s trajectories and minimize time when UGVs have to wait to guarantee safety. We validate our system through extensive simulation with up to 7 UGVs and real-world experiments with 3 UGVs.

Mutual localization stands as a foundational component within various domains of multi-robot systems. Nevertheless, in relative pose estimation, time synchronization is usually underappreciated and rarely addressed, although it significantly influences estimation accuracy. In this paper, we introduce time synchronization into mutual localization to recover the time offset and relative poses between robots simultaneously. Under a constant velocity assumption in a short time, we fuse time offset estimation with our previous bearing-based mutual localization by a novel error representation. Based on the error model, we formulate a joint optimization problem and utilize semi-definite relaxation (SDR) to furnish a lossless relaxation. By solving the relaxed problem, time synchronization and relative pose estimation can be achieved when time drift between robots is limited. To enhance the application range of time offset estimation, we further propose an iterative method to recover the time offset from coarse to fine. Comparisons between the proposed method and the existing ones through extensive simulation tests present prominent benefits of time synchronization on mutual localization. Moreover, real-world experiments are conducted to show the practicality and robustness.

Drone racing has become a popular international competition and has attained wide attention in recent years. However, the requirements of high-level operation keep the novice pilots away from participating in it. This paper presents a trajectory-based flight assistive system that enables various operators to fly the drone in a racing scene at a high speed. The whole system is structured hierarchically, consisting of both offline and online components. In the offline part, a global time-optimal trajectory is generated as the expert reference, and a dense flight corridor is constructed to provide sufficiently large safe region. In the online part, a remote control-mapped primitive is designed to fast encapsulate pilots’ inputs, and the time mapping based trajectory progress is customized to further capture intention. Then, a trajectory planner is proposed to generate intention-aligned, smooth, feasible, and safe trajectories periodically. Additionally, a yaw planning that provides the pilot with the best suitable view angle is employed to further alleviate the operation difficulty. Simulations and real world experiments are implemented to verify the performance of our system. The maximum flight speed can reach 6.0 m/s for a novice drone pilot in a real racing scene. Our code is released as an open-source package1.

Just as humans can become disoriented in featureless deserts or thick fogs, not all environments are conducive to the Localization Accuracy and Stability (LAS) of autonomous robots. This paper introduces an efficient framework designed to enhance LiDAR-based LAS through strategic trajectory generation, known as Perception-aware Planning. Unlike vision-based frameworks, the LiDAR-based requires different considerations due to unique sensor attributes. Our approach focuses on two main aspects: firstly, assessing the impact of LiDAR observations on LAS. We introduce a perturbation-induced metric to provide a comprehensive and reliable evaluation of LiDAR observations. Secondly, we aim to improve motion planning efficiency. By creating a Static Observation Loss Map (SOLM) as an intermediary, we logically separate the time-intensive evaluation and motion planning phases, significantly boosting the planning process. In the experimental section, we demonstrate the effectiveness of the proposed metrics across various scenes and the feature of trajectories guided by different metrics. Ultimately, our framework is tested in a real-world scenario, enabling the robot to actively choose topologies and orientations preferable for localization. The source code is accessible at https://github.com/ZJU-FAST-Lab/LF-3PM.

Active reconstruction technique enables robots to autonomously collect scene data for full coverage, relieving users from tedious and time-consuming data capturing process. However, designed based on unsuitable scene representations, existing methods show unrealistic reconstruction results or the inability of online quality evaluation. Due to the recent advancements in explicit radiance field technology, online active high-fidelity reconstruction has become achievable. In this paper, we propose GS-Planner, a planning framework for active high-fidelity reconstruction using 3D Gaussian Splatting. With improvement on 3DGS to recognize unobserved regions, we evaluate the reconstruction quality and completeness of 3DGS map online to guide the robot. Then we design a sampling-based active reconstruction strategy to explore the unobserved areas and improve the reconstruction geometric and textural quality. To establish a complete robot active reconstruction system, we choose quadrotor as the robotic platform for its high agility. Then we devise a safety constraint with 3DGS to generate executable trajectories for quadrotor navigation in the 3DGS map. To validate the effectiveness of our method, we conduct extensive experiments and ablation studies in highly realistic simulation scenes.

Autonomous target tracking with quadrotors has wide applications in many scenarios, such as cinematographic follow-up shooting or suspect chasing. Target motion prediction is necessary when designing the tracking planner. However, the widely used constant velocity or constant rotation assumption can not fully capture the dynamics of the target. The tracker may fail when the target happens to move aggressively, such as sudden turn or deceleration. In this paper, we propose an intention-aware planner by additionally considering the intention of the target to enhance safety and robustness in aerial tracking applications. Firstly, a designated intention prediction method is proposed, which combines a user-defined potential assessment function and a state observation function. A reachable region is generated to speci cally evaluate the turning intentions. Then we design an intention-driven hybrid A* method to predict the future possible positions for the target. Finally, an intention-aware optimization approach is designed to generate a spatial-temporal optimal trajectory, allowing the tracker to perceive unexpected situations from the target. Benchmark comparisons and real-world experiments are conducted to validate the performance of our method.

In recent years, multirotors have become more and more widely used, such as in aerial photography and delivery. Ensuring a safe landing in emergencies is the most basic requirement, and it is important to make full use of all the sensors of the multirotor. To improve the safety of UAV landing in unknown unstructured scenes, this paper proposes a multi-FOV-constrained trajectory planning algorithm. Due to the discontinuity of multi-FOV constraints and the nonlinearity of UAV dynamics, the entire trajectory planning problem is a nonlinear optimization problem with non-convex constraints. To address this problem, our algorithm contains two stages, a multi-fov-constrained path search algorithm and a safe landing trajectory optimization algorithm. The multi-fov-constrained path search algorithm is used to generate a safe initial path that satisfies the FOV constraint. Then, the safe landing trajectory optimization algorithm generates a safe trajectory, which considers FOV constraints, dynamics, smoothness, and obstacle avoidance. We conducted simulation experiments and real-world experiments to verify the robustness and effectiveness of our algorithm.

In many situations such as city delivery and wild inspection, quadrotors are often required to follow a predefined reference trajectory. However, these reference trajectories cannot be perfectly safe, resulting in conflicts between tracking the reference precisely, flying safely, and finishing the mission timely. This paper proposes to solve the above problem, by introducing a replanning framework that first generates a topological consistent collision-free initial path and then flexibly optimizes the rejoin point and trajectory duration to generate a smooth and safe local rejoining trajectory. To avoid local trajectory switching in different directions during high-frequency replanning, we propose a topology-preserving path search algorithm based on kinodynamic RRT*. To satisfy dynamic constraints, avoid delays, and achieve a smooth rejoin of the reference trajectory, we propose an optimization-based approach to refine the initial trajectory. The simulation results confirm that our proposed topological consistency and flexible optimization methods can reduce the risk of local trajectory and decrease obstacle avoidance delay for tracking reference trajectory. We also conduct real-world experiments in challenging environments and verify the effectiveness of our method.

Tracked robots with reconfigurable mechanisms exhibit great maneuverability due to their adaptability to complex ground conditions. Reconfigurable tracked robots with geometry-changing tracks show further obstacle-crossing capabilities with compact dimensions. However, existing systems face deployment limitations due to either complex transmission mechanisms or unsustainable designs when maintaining the tension in the tracks. To address these challenges, we introduce a novel design of a reconfigurable tracked robot with geometry-changing tracks, which achieves strong terrain traversability with good mechanical properties. We achieve the elliptical trajectory of key planetary wheels through a novel Quad-slider Elliptical Trammel Mechanism (Qs-ETM), allowing the tracks to maintain fixed tension while changing their geometry. Furthermore, the combination of direct drive motors significantly enhances its mechanical properties and agility. A detailed analysis of the kinematic and dynamic characteristics has been conducted and proved with a series of simulations. We built a fully functional prototype of the design and tested it in real-world experiments to validate its advantages. The result shows that our design can reduce the torque required by up to 68.3% and the shear stress of the flipper by up to 67.1%.

As a fundamental component of autonomous driving systems, motion planning has garnered significant attention from both academia and industry. This paper focuses on efficient and spatial-temporal optimal trajectory optimization in unstructured environments using compact convex approximations of vehicle shapes. Conventional approaches typically model the task as an optimal control problem by discretizing the motion process in state configuration space. However, this often results in a tradeoff between optimality and efficiency since generating high-quality motion trajectories often requires high-precision discretization of the dynamic process, which imposes a substantial computational burden. To address this issue, we leverage the differential flatness property of car-like robots to simplify the trajectory representation and analytically formulate the spatial-temporal joint optimization problem with flat outputs in a compact manner, while ensuring the feasibility of nonholonomic dynamics. Moreover, we achieve efficient obstacle avoidance with a collision-free driving corridor for unmodelled obstacles and signed distance approximations for dynamic moving objects. We present comprehensive benchmarks with State-of-the-Art methods, demonstrating the significance of the proposed method in terms of efficiency and trajectory quality. Real-world experiments verify the practicality of our algorithm. We will release our codes for the research community.

In this article, we propose a novel nonlinear observer based on neural networks (NNs), called neural observers, for observation tasks of linear time-invariant (LTI) systems and uncertain nonlinear systems. In particular, the neural observer designed for uncertain systems is inspired by the active disturbance rejection control, which can measure the uncertainty in real time. The stability analysis (e.g., exponential convergence rate) of LTI and uncertain nonlinear systems (involving neural observers) are presented and guaranteed, where it is shown that the observation problems can be solved only using the linear matrix inequalities (LMIs). Also, it is revealed that the observability and controllability of the system matrices are required to demonstrate the existence of solutions for LMIs. Finally, the effectiveness of neural observers is verified in three simulation cases, including the X-29A aircraft model, the nonlinear pendulum, and the four-wheel steering vehicle.

In the field of trajectory generation for objects, ensuring continuous collision-free motion remains a huge challenge, especially for non-convex geometries and complex environments. Previous methods either oversimplify object shapes, which results in a sacrifice of feasible space or rely on discrete sampling, which suffers from the "tunnel effect". To address these limitations, we propose a novel hierarchical trajectory generation pipeline, which utilizes the Swept Volume Signed Distance Field (SVSDF) to guide trajectory optimization for Continuous Collision Avoidance (CCA). Our interdisciplinary approach, blending techniques from graphics and robotics, exhibits outstanding effectiveness in solving this problem. We formulate the computation of the SVSDF as a Generalized Semi-Infinite Programming model, and we solve for the numerical solutions at query points implicitly, thereby eliminating the need for explicit reconstruction of the surface. Our algorithm has been validated in a variety of complex scenarios and applies to robots of various dynamics, including both rigid and deformable shapes. It demonstrates exceptional universality and superior CCA performance compared to typical algorithms. The code will be released at https://github.com/ZJU-FAST-Lab/Implicit-SVSDF-Planner for the benefit of the community.

Perching on the moving platforms is a promising solution to enhance the endurance and operational range of quadrotors, which could benefit the efficiency of a variety of air ground cooperative tasks. To ensure robust perching, tracking with a steady relative state and reliable perception is a prerequisite. This paper presents an adaptive dynamic tracking and perching scheme for autonomous quadrotors to achieve tight integration with moving platforms. For reliable perception of dynamic targets, we introduce elastic visibility aware planning to actively avoid occlusion and target loss. Additionally, we propose a flexible terminal adjustment method that adapts the changes in flight duration and the couple d terminal states, ensuring full state synchronization with the time varying perching surface at various angles. A relaxation strategy is developed by optimizing the tangential relative speed to address the dynamics and safety violations brought by hard bo undary conditions. Moreover, we take SE(3) motion planning into account to ensure no collision until the contact moment. Furthermore, we propose an efficient spatiotemporal trajectory optimization framework considerin g full state dynamics The proposed method is extensively tested through benchmark comparisons and ablation studies. To facilitate the application of academic research to industry and to validate the efficiency under strictly limited computational resources, we deploy our system on a commercial drone (DJI MAVIC3) with a full size sport utility vehicle (SUV). We conduct extensive real world experiments, where the drone successfully tracks and perches at 30 km/h (8.3 m/ s) on the top of the SUV, and at 3.5∼m/s with 60° inclined into the trunk of the SUV.

Collision evaluation is of essential importance in various applications. However, existing methods are either cumbersome to calculate or not exact. Therefore, considering the cost of implementation, most whole-body planning works, which require evaluating collision between robots and environments, struggle to tradeoff between accuracy and computationally efficiency. In this paper, we propose a zero-gap whole-body collision evaluation that can be formulated as a low-dimensional linear programming. This evaluation can be solved analytically in linear complexity. Moreover, the method provides gradient efficiently, making it accessible to optimization-based applications. Additionally, this method provides support for obstacles represented by either points or hyperplanes. Experiments on the widely used aerial and car-like robots validate the versatility and practicality of our method.

With the development of robotics, ground robots are no longer limited to planar motion. Passive height variation due to complex terrain and active height control provided by special structures on robots require a more general navigation planning framework beyond 2D. Existing methods rarely considers both simultaneously, limiting the capabilities and applications of ground robots. In this paper, we proposed an optimization-based planning framework for ground robots considering both active and passive height changes on the z-axis. The proposed planner first constructs a penalty field for chassis motion constraints defined in$\mathbb{R}^{3}$such that the optimal solution space of the trajectory is continuous, resulting in a high-quality smooth chassis trajectory. Also, by constructing custom constraints in the z-axis direction, it is possible to plan trajectories for different types of ground robots which have z-axis degree of freedom. We performed simulations and real-world experiments to verify the efficiency and trajectory quality of our algorithm.

In constrained solution spaces with a huge number of homotopy classes, standalone sampling-based kinodynamic planners suffer low efficiency in convergence. Local optimization is integrated to alleviate this problem. In this paper, we propose to thrive the trajectory tree growing by optimizing the tree in the forms of deformation units, and each unit contains one tree node and all the edges connecting it. The deforming proceeds both spatially and temporally by optimizing the node state and edge time durations efficiently. Deforming the unit only changes the tree locally yet improves the overall quality of a corresponding subtree. Further, to consider the computation burden and optimizing level, patterns to deform different tree parts in combination of different deformation units are studied and compared, all showing much faster convergence. The proposed deformation can be easily integrated into different RRT-based kinodynamic planning methods, and numerical experiments show that integrating the spatio-temporal deformation greatly accelerates the convergence and outperforms the spatial-only deformation.

Service robots have attracted extensive attention due to specially designed functions, such as mobile manipulators or robots with extra structures. For robots that have changing shapes, autonomous navigation in the real world presents new challenges. In this paper, we propose a trajectory optimization method for differential-drive mobile robots with controllable changing shapes in dense 3D environments. We model the whole-body trajectory as a polynomial trajectory that satisfies the nonholonomic dynamics of the base and dynamics of the extra joints. These constraints are converted into soft constraints, and an activation function for dense sampling is applied to avoid nonlinear mutations. In addition, we guarantee the safety of full shape by limiting the system's distance from obstacles. To comprehensively simulate a large extent of height and width changes, we designed a novel Shape-Changing Robot with a Differential Base (SCR-DB). Our global trajectory optimization gives a smooth and collision-free trajectory for SCR-DB at a low computational cost. We present vast simulations and real-world experiments to validate our performance, including coupled whole-body and independent differential-driven vehicle motion planning.

For letting mobile robots travel flexibly through complicated environments, increasing attention has been paid to the whole-body collision evaluation. Most existing works either opt for the conservative corridor-based methods that impose strict requirements on the corridor generation, or ESDF-based methods that suffer from high computational overhead. It is still a great challenge to achieve fast and accurate whole-body collision evaluation. In this paper, we propose a Robo-centric ESDF (RC-ESDF) that is pre-built in the robot body frame and is capable of seamlessly applied to any-shape mobile robots, even for those with non-convex shapes. RC-ESDF enjoys lazy collision evaluation, which retains only the minimum information sufficient for whole-body safety constraint and significantly speeds up trajectory optimization. Based on the analytical gradients provided by RC-ESDF, we optimize the position and rotation of robot jointly, with whole-body safety, smoothness, and dynamical feasibility taken into account. Extensive simulation and real-world experiments verified the reliability and generalizability of our method.

Robot swarm is a hot spot in robotic research community. In this paper, we propose a decentralized framework for car-like robotic swarm which is capable of real-time planning in cluttered environments. In this system, path finding is guided by environmental topology information to avoid frequent topological change, and search-based speed planning is leveraged to escape from infeasible initial value's local minima. Then spatial-temporal optimization is employed to generate a safe, smooth and dynamically feasible trajectory. During optimization, the trajectory is discretized by fixed time steps. Penalty is imposed on the signed distance between agents to realize collision avoidance, and differential flatness cooperated with limitation on front steer angle satisfies the non-holonomic constraints. With trajectories broadcast to the wireless network, agents are able to check and prevent potential collisions. We validate the robustness of our system in simulation and real-world experiments. Code will be released as open-source packages.

The development of autonomous rotary-wing UAVs has shown an evident tendency in miniaturization. However, the side effects brought by miniaturization, such as decreased load capability, shorter flight duration and reduced autonomous ability, seriously hinder its process. In this paper, we first investigate the configurations of different rotary-wing aircraft and optimize the configuration selection. Afterward, with several elaborate mechanisms contributing to the miniaturization, we present the hardware design and control strategy of a mini coaxial helicopter, which is 62% smaller than the state-of-the-art autonomous mini quadrotor so far in collision area [1]. Meanwhile, abundant experiments reveal that it achieves impressive traversability and is capable of conducting autonomous tasks in unknown dense scenarios, while maintaining satisfactory performance regarding loadability and flight duration.

In recent years, there is a noteworthy advance-ment in autonomous drone racing. However, the primary focus is on attaining execution times, while scant attention is given to the challenges of dynamic environments. The high-speed nature of racing scenarios, coupled with the potential for unforeseeable environmental alterations, present stringent requirements for online replanning and its timeliness. For racing in dynamic environments, we propose an online replanning framework with an efficient polynomial trajectory representation. We trade off between aggressive speed and flexible obstacle avoidance based on an optimization approach. Additionally, to ensure safety and precision when crossing intermediate racing waypoints, we formulate the demand as hard constraints during planning. For dynamic obstacles, parallel multi-topology trajectory planning is designed based on engineering considerations to prevent racing time loss due to local optimums. The framework is integrated into a quadrotor system and successfully demonstrated at the DJI Robomaster Intelligent UAV Championship, where it successfully complete the racing track and placed first, finishing in less than half the time of the second-place11https://pro-robomasters-hz-n5i3.oss-cn-hangzhou.aliyuncs.com/sass/event-list.html.

Autonomous navigation of ground robots on uneven terrain is being considered in more and more tasks. However, uneven terrain will bring two problems to motion planning: how to assess the traversability of the terrain and how to cope with the dynamics model of the robot associated with the terrain. The trajectories generated by existing methods are often too conservative or cannot be tracked well by the controller since the second problem is not well solved. In this paper, we propose terrain pose mapping to describe the impact of terrain on the robot. With this mapping, we can obtain the SE(3) state of the robot on uneven terrain for a given state in SE(2). Then, based on it, we present a trajectory optimization framework for car-like robots on uneven terrain that can consider both of the above problems. The trajectories generated by our method conform to the dynamics model of the system without being overly conservative and yet able to be tracked well by the controller. We perform simulations and real-world experiments to validate the efficiency and trajectory quality of our algorithm.

Mutual localization plays a crucial role in multi-robot cooperation. CREPES, a novel system that focuses on six degrees of freedom (DOF) relative pose estimation for multi-robot systems, is proposed in this paper. CREPES has a compact hardware design using active infrared (IR) LEDs, an IR fish-eye camera, an ultra-wideband (UWB) module and an inertial measurement unit (IMU). By leveraging IR light communication, the system solves data association between visual detection and UWB ranging. Ranging measurements from the UWB and directional information from the camera offer relative 3-DOF position estimation. Combining the mutual relative position with neighbors and the gravity constraints provided by IMUs, we can estimate the 6-DOF relative pose from a single frame of sensor measurements. In addition, we design an estimator based on the error-state Kalman filter (ESKF) to enhance system accuracy and robustness. When multiple neighbors are available, a Pose Graph Optimization (PGO) algorithm is applied to further improve system accuracy. We conduct enormous experiments to demonstrate CREPES’ accuracy between robot pairs and a team of robots, as well as performance under challenging conditions.

Terrestrial and aerial bimodal vehicles have gained widespread attention due to their cross-domain maneuverability. Nevertheless, their bimodal dynamics significantly increase the complexity of motion planning and control, thus hindering robust and efficient autonomous navigation in unknown environments. To resolve this issue, we develop a model-based planning and control framework for terrestrial aerial bi-modal vehicles. This work begins by deriving a unified dynamic model and the corresponding differential flatness. Leveraging differential flatness, an optimization-based trajectory planner is proposed, which takes into account both solution quality and computational efficiency. Moreover, we design a tracking controller using nonlinear model predictive control based on the proposed unified dynamic model to achieve accurate trajectory tracking and smooth mode transition. We validate our framework through extensive benchmark comparisons and experiments, demonstrating its effectiveness in terms of planning quality and control performance.

Optimization-based trajectory generation methods are widely used in whole-body planning for robots. However, existing work either oversimplifies the robot's geometry and environment representation, resulting in a conservative trajectory or suffers from a huge overhead in maintaining additional information such as the Signed Distance Field (SDF). To bridge the gap, we consider the robot as an implicit function, with its surface boundary represented by the zero-level set of its SDF. We further employ another implicit function to lazily compute the signed distance to the swept volume generated by the robot and its trajectory. The computation is efficient by exploiting continuity in space-time, and the implicit function guarantees continuous collision evaluation even for nonconvex robots with complex surfaces. We also propose a trajectory optimization pipeline applicable to the implicit SDF. Simulation and real-world experiments validate the high performance of our approach for arbitrarily shaped robot trajectory optimization.

Formation flight has a vast potential for aerial robot swarms in various applications. However, the existing methods lack the capability to achieve fully autonomous large-scale formation flight in dense environments. To bridge the gap, we present a complete formation flight system that effectively integrates real-world constraints into aerial formation navigation. This article proposes a differentiable graph-based metric to quantify the overall similarity error between formations. This metric is invariant to rotation, translation, and scaling, providing more freedom for formation coordination. We design a distributed trajectory optimization framework that considers formation similarity, obstacle avoidance, and dynamic feasibility. The optimization is decoupled to make large-scale formation flights computationally feasible. To improve the elasticity of formation navigation in highly constrained scenes, we present a swarm reorganization method that adaptively adjusts the formation parameters and task assignments by generating local navigation goals. A novel swarm agreement strategy called global-remap-local-replan and a formation-level path planner is proposed in this article to coordinate the global planning and local trajectory optimizations.To validate the proposed method, we design comprehensive benchmarks and simulations with other cutting-edge works in terms of adaptability, predictability, elasticity, resilience, and efficiency. Finally, integrated with palm-sized swarm platforms with onboard computers and sensors, the proposed method demonstrates its efficiency and robustness by achieving the largest scale formation flight in dense outdoor environments.

This paper proposes Elastic Tracker, a flexible trajectory planning framework that can deal with challenging tracking tasks with guaranteed safety and visibility. Firstly, an object detection and intension-free motion prediction method is designed. Then an occlusion-aware path finding method is proposed to provide a proper topology. A smart safe flight corridor generation strategy is designed with the guiding path. An analytical occlusion cost is evaluated. Finally, an effective trajectory optimization approach enables to generate a spatio-temporal optimal trajectory within the resultant flight corridor. Particular formulations are designed to guarantee both safety and visibility, with all the above requirements optimized jointly. The experimental results show that our method works more robustly but with less computation than the existing methods, even in some challenging tracking tasks.

The visible capability is critical in many robot applications, such as inspection and surveillance, etc. Without the assurance of the visibility to targets, some tasks end up not being complete or even failing. In this paper, we propose a visibility guaranteed planner by star-convex constrained optimization. The visible space is modeled as star convex polytope (SCP) by nature and is generated by finding the visible points directly on point cloud. By exploiting the properties of the SCP, the visibility constraint is formulated for trajectory optimization. The trajectory is confined in the safe and visible flight corridor which consists of convex polytopes and SCPs. We further make a relaxation to the visibility constraints and transform the constrained trajectory optimization problem into an unconstrained one that can be reliably and efficiently solved. To validate the capability of the proposed planner, we present the practical application in site inspection. The experimental results show that the method is efficient, scalable, and visibility guaranteed, presenting the prospect of application to various other applications in the future.

For aerial swarms, navigation in a prescribed formation is widely practiced in various scenarios. However, the associated planning strategies typically lack the capability of avoiding obstacles in cluttered environments. To address this deficiency, we present an optimization-based method that ensures collision-free trajectory generation for formation flight. In this paper, a novel differentiable metric is proposed to quantify the overall similarity distance between formations. We then formulate this metric into an optimization framework, which achieves spatial-temporal planning using polynomial trajectories. Minimization over collision penalty is also incorporated into the framework, so that formation preservation and obstacle avoidance can be handled simultaneously. To validate the efficiency of our method, we conduct benchmark comparisons with other cutting-edge works. Integrated with an autonomous distributed aerial swarm system, the proposed method demonstrates its efficiency and robustness in real-world experiments with obstacle-rich surroundings11https://www.youtube.com/watch?v=lFumtOrJci4. We will release the source code for the reference of the community22https://github.com/ZJU-FAST-Lab/Swarm-Formation.

Recently, the community has witnessed numerous datasets built for developing and testing state estimators. However, for some applications such as aerial transportation or search-and-rescue, the contact force or other disturbance must be perceived for robust planning and control, which is beyond the capacity of these datasets. This paper introduces a Visual-Inertial-Dynamical (VID) dataset, not only focusing on traditional six degrees of freedom (6-DOF) pose estimation but also providing dynamical characteristics of the flight platform for external force perception or dynamics-aided estimation. The VID dataset contains hardware synchronized imagery and inertial measurements, with accurate ground truth trajectories for evaluating common visual-inertial estimators. Moreover, the proposed dataset highlights rotor speed and motor current measurements, control inputs, and ground truth 6-axis force data to evaluate external force estimation. To the best of our knowledge, the proposed VID dataset is the first public dataset containing visual-inertial and complete dynamical information in the real world for pose and external force evaluation. The dataset1and related files2are open-sourced.

This letter presents a complete framework Meeting-Merging-Mission for multi-robot exploration under communication restriction. Considering communication is limited in both bandwidth and range in the real world, we propose a lightweight environment presentation method and an efficient cooperative exploration strategy. For lower bandwidth, each robot uses specific polytopes to maintain free space and to generate Super Frontier Information (SFI), which serves as the source for exploration decision-making. To reduce repeated exploration, we develop a mission-based protocol that drives robots to share collected information in stable rendezvous. We also design a complete path planning scheme for both centralized and decentralized cases. To validate that our framework is practical and generic, we present an extensive benchmark and deploy our system into multi-UGV and multi-UAV platforms.

Free-space-oriented roadmaps typically generate a series of convex geometric primitives, which constitute the safe region for motion planning. However, a static environment is assumed for this kind of roadmap. This assumption makes it unable to deal with dynamic obstacles and limits its applications. In this paper, we present a dynamic free-space roadmap, which provides feasible spaces and a navigation graph for safe quadrotor motion planning. Our roadmap is constructed by continuously seeding and extracting free regions in the environment. In order to adapt our map to environments with dynamic obstacles, we incrementally decompose the polyhedra intersecting with obstacles into obstacle-free regions, while the graph is also updated by our well-designed mechanism. Extensive simulations and real-world experiments demonstrate that our method is practically applicable and efficient.

This paper presents a novel trajectory planning method for aerial perching. Compared with the existing work, the terminal states and the trajectory durations can be adjusted adaptively, instead of being determined in advance. Further-more, our planner is able to minimize the tangential relative speed on the premise of safety and dynamic feasibility. This feature is especially notable on micro aerial robots with low maneuverability or scenarios where the space is not enough. Moreover, we design a flexible transformation strategy to eliminate terminal constraints along with reducing optimization variables. Besides, we take precise SE(3) motion planning into account to ensure that the drone would not touch the landing platform until the last moment. The proposed method is validated onboard by a palm-sized micro aerial robot with quite limited thrust and moment (thrust-to-weight ratio 1.7) perching on a mobile inclined surface. Sufficient experimental results show that our planner generates an optimal trajectory within 20ms, and replans with warm start in 2ms.

In this paper, we describe a robust multi-drone planning framework for high-speed trajectories in large scenes. It uses a free-space-oriented map to free the optimization from cumbersome environment data. A capsule-like safety constraint is designed to avoid reciprocal collisions when vehicles deviate from their nominal flight progress under disturbance. We further show the minimum-singularity differential flatness of our drone dynamics with nonlinear drag effects involved. Leveraging the flatness map, trajectory optimization is efficiently conducted on the flat outputs while still subject to physical limits considering drag forces at high speeds. The robustness and effectiveness of our framework are both validated in large-scale simulations. It can compute collision-free trajectories satisfying high-fidelity vehicle constraints for hundreds of drones within 10 minutes.

For real-time multirotor kinodynamic planning, the efficiency of sampling-based methods is usually hindered by difficult-to-sample homotopy classes like narrow passages. In this paper, we address this issue by a hybrid scheme. We firstly propose a fast regional optimizer exploiting the information of local environments and then integrate it into a bidirectional global sampling process. The incorporation of the local optimization shows significantly improved success rates and less planning time in various types of challenging environments. We further present a refinement module utilizing the same framework as the regional optimizer. It comprehensively investigates the resulting trajectory of the global sampling and improves its smoothness with nearly negligible computation effort. Benchmark results illustrate that our proposed method can better exploit a previous trajectory compared to the state-of-the-art ones. The planning methods are applied to generate trajectories for a quadrotor system in simulation and real-world, and their capability is validated in real-time applications.

Online trajectory planners enable quadrotors to safely and smoothly navigate in unknown cluttered environments. However, tuning parameters is challenging since modern planners have become too complex to mathematically model and predict their interaction with unstructured environments. This work takes humans out of the loop by proposing a planner parameter adaptation framework that formulates objectives into two complementary categories and optimizes them asynchronously. Objectives evaluated with and without trajectory execution are optimized using Bayesian Optimization (BayesOpt) and Particle Swarm Optimization (PSO), respectively. By combining two kinds of objectives, the total convergence rate of the black-box optimization is accelerated while the dimension of optimized parameters can be increased. Benchmark comparisons demonstrate its superior performance over other strategies. Tests with changing obstacle densities validate its real-time environment adaption, which is difficult for prior manual tuning. Real-world flights with different drone platforms, environments, and planners show the proposed framework's scalability and effectiveness.

The precise operation control of high-speed trains is pivotal to maintain the safety and efficiency of trains, while the inevitable state delays will seriously attenuate the performance of control system. In this paper, an adaptive iterative learning control (ILC) approach for high-speed trains is presented in the presence of the nonlinearly parameterized uncertainties and multiple unknown state delays, aiming to drive that the displacements and velocities of trains can track the desired reference trajectories. To describe the operational dynamics of trains more realistically, the multi-particle model of trains involving multiple time-varying delays is established by analyzing the aerodynamic resistance, mechanical resistance, and coupler force acting on different cars. The proposed adaptive ILC scheme fully leverages various techniques, e.g., the hyperbolic tangent function, the parameter separation, to cope with the inherent nonlinearities, uncertainties and couplings of system. Specially, to eliminate the negative influence of unknown delays, an appropriate Krasovskii function is integrated into the Lyapunov criterion to devise the learning controller and check the stability of control systems. The novelties of our work lie in that the refinement model and periodical characteristic are simultaneously utilized to improve the practicability and performance of control scheme for the high-speed trains with multiple state delays.

In this article, we present an optimization-based framework for multicopter trajectory planning subject to geometrical configuration constraints and user-defined dynamic constraints. The basis of the framework is a novel trajectory representation built upon our novel optimality conditions for unconstrained control effort minimization. We design linear-complexity operations on this representation to conduct spatial–temporal deformation under various planning requirements. Smooth maps are utilized to exactly eliminate geometrical constraints in a lightweight fashion. A variety of state-input constraints are supported by the decoupling of dense constraint evaluation from sparse parameterization and the backward differentiation of flatness map. As a result, this framework transforms a generally constrained multicopter planning problem into an unconstrained optimization that can be solved reliably and efficiently. Our framework bridges the gaps among solution quality, planning efficiency, and constraint fidelity for a multicopter with limited resources and maneuvering capability. Its generality and robustness are both demonstrated by applications to different flight tasks. Extensive simulations and benchmarks are also conducted to show its capability of generating high-quality solutions while retaining the computation speed against other specialized methods by orders of magnitude.

Handling uncertainty is one of the most important challenges in real physical systems. In this article, we study an approximate optimal magnetic control strategy for a one-dimensional (1-D) magnetohydrodynamic (MHD) system with uncertainty quantification within the learning framework of the underlying MHD model. First, the MHD flow system is modeled by coupled partial differential equations (PDEs) wherein the Reynolds number is not deterministic but random. Then, the optimal magnetic control problem is formulated and reduced to a parameter selection problem with stochastic PDE constraints by means of the control parameterization method. Significantly different from the conventional sensitivity analysis and adjoint methods, a polynomial chaos expansion (PCE) based on a multifidelity model is developed to construct the underlying PDEs model by polynomial functions. Thus, the relationship between the objective function and the control sequence is derived explicitly, which, in turn, transforms the optimal parameter selection problem with stochastic PDE constraints into a typical algebraic optimization problem that can be easily solved by the existing nonlinear programming algorithm. Numerical simulations are illustrated to demonstrate the high performance of our proposed method.

Recently, quadrotors are gaining significant attention in aerial transportation and delivery. In these scenarios, an accurate estimation of the external force is as essential as the six degree-of-freedom (DoF) pose since it is of vital importance for planning and control of the vehicle. To this end, we propose a tightly-coupled Visual-Inertial-Dynamics (VID) system that simultaneously estimates the external force applied to the quadrotor along with the six DoF pose. Our method builds on the state-of-the-art optimization-based Visual-Inertial system [1], with a novel deduction of the dynamics and external force factor extended from VIMO [2]. Utilizing the proposed dynamics and external force factor, our estimator robustly and accurately estimates the external force even when it varies widely. Moreover, since we explicitly consider the influence of the external force, when compared with VIMO [2] and VINS-Mono [1], our method shows comparable and superior pose accuracy, even when the external force ranges from neglectable to significant. The robustness and effectiveness of the proposed method are validated by extensive real-world experiments and application scenario simulation. We will release an open-source package of this method along with datasets with ground truth force measurements for the reference of the community.

This paper proposes a systematic solution that uses an unmanned aerial vehicle (UAV) to aggressively and safely track an agile target. It properly handles the challenging situations where the intent of the target and the dense environments are unknown. Our work is divided into two parts: target motion prediction and tracking trajectory planning. The target motion prediction method utilizes target observations to reliably predict its future motion. The tracking trajectory planner follows the hierarchical workflow. A target informed kinody-namic searching method is adopted as the front-end, which heuristically searches for a safe tracking trajectory. The back- end optimizer then refines it into a spatial-temporal optimal trajectory. The proposed solution is integrated into an onboard quadrotor system. We fully test the system in challenging real-world tracking missions. Moreover, benchmark comparisons validate that the proposed method surpasses the cutting-edge methods on time efficiency and tracking effectiveness.

Maintaining a map online is resource-consuming while a robust navigation system usually needs environment abstraction via a well-fused map. In this paper, we propose a mapless local planner which directly conducts such abstraction on the unfused sensor data. A limited-memory data structure with a reliable proximity query algorithm is proposed for maintaining raw historical information. A sampling-based scheme is designed to extract the free-space skeleton. A smart waypoint selection strategy enables to generate high-quality trajectories within the resultant flight corridors. Our planner differs from other mapless ones in that it can abstract and exploit the environment information more efficiently. The online replan consistency and success rate are both significantly improved against conventional mapless methods.

The quadrotor is popularly used in challenging environments due to its superior agility and flexibility. In these scenarios, trajectory planning plays a vital role in generating safe motions to avoid obstacles while ensuring flight smoothness. Although many works on quadrotor planning have been proposed, a research gap exists in incorporating self-adaptation into a planning framework to enable a drone to automatically fly slower in denser environments and increase its speed in a safer area. In this paper, we propose an environmental adaptive planner to adjust the flight aggressiveness effectively based on the obstacle distribution and quadrotor state. Firstly, we design an environmental adaptive safety aware method to assign the priority of the surrounding obstacles according to the environmental risk level and instantaneous motion tendency. Then, we apply it into a multi-layered model predictive contouring control (Multi-MPCC) framework to generate adaptive, safe, and dynamical feasible local trajectories. Extensive simulations and real-world experiments verify the efficiency and robustness of our planning framework. Benchmark comparison also shows superior performances of our method with another advanced environmental adaptive planning algorithm. Moreover, we release our planning framework as open-source ros-packages1.

For quadrotor trajectory planning, describing a polynomial trajectory through coefficients and end-derivatives both enjoy their own convenience in energy minimization. We name them double descriptions of polynomial trajectories. The transformation between them, causing most of the inefficiency and instability, is formally analyzed in this paper. Leveraging its analytic structure, we design a linear-complexity scheme for both jerk/snap minimization and parameter gradient evaluation, which possesses efficiency, stability, flexibility, and scalability. With the help of our scheme, generating an energy optimal (minimum snap) trajectory only costs 1 µs per piece at the scale up to 1,000,000 pieces. Moreover, generating large-scale energy-time optimal trajectories is also accelerated by an order of magnitude against conventional methods.

Multicopters are able to perform high maneuverability yet their potential have not been fully achieved. In this work, we propose a full-body, optimization-based motion planning framework that takes shape and attitude of aerial robot into consideration such that the aggressiveness of drone maneuvering improves significantly in cluttered environment. Our method takes in a series of intersecting polyhedrons that describe a range of 3D free spaces and outputs a time-indexed trajectory in real-time with full-body collision-free guarantee. The drone is modeled as a tilted cuboid, yet we argue that our framework can be freely adjusted to fit multicopters of different shapes. Guaranteeing dynamic feasibility and safety conditions, our framework transforms the original constrained nonlinear programming problem to an unconstrained one in higher dimensions which is further solved by quasi-Newton methods. Benchmark has shown that our method improves the state-of-art with orders of magnitude in terms of computation time and memory usage. Simulations and onboard experiments are carried out as validation.

This paper presents a decentralized and asynchronous systematic solution for multi-robot autonomous navigation in unknown obstacle-rich scenes using merely onboard resources. The planning system is formulated under gradient-based local planning framework, where collision avoidance is achieved by formulating the collision risk as a penalty of a nonlinear optimization problem. In order to improve robustness and escape local minima, we incorporate a lightweight topological trajectory generation method. Then agents generate safe, smooth, and dynamically feasible trajectories in only several milliseconds using an unreliable trajectory sharing network. Relative localization drift among agents is corrected by using agent detection in depth images. Our method is demonstrated in both simulation and real-world experiments. The source code is released for the reference of the community.

The development of aerial autonomy has enabled aerial robots to fly agilely in complex environments. However, dodging fast-moving objects in flight remains a challenge, limiting the further application of unmanned aerial vehicles (UAVs). The bottleneck of solving this problem is the accurate perception of rapid dynamic objects. Recently, event cameras have shown great potential in solving this problem. This paper presents a complete perception system including ego-motion compensation, object detection, and trajectory prediction for fast-moving dynamic objects with low latency and high precision. Firstly, we propose an accurate ego-motion compensation algorithm by considering both rotational and translational motion for more robust object detection. Then, for dynamic object detection, an event camera-based efficient regression algorithm is designed. Finally, we propose an optimization-based approach that asynchronously fuses event and depth cameras for trajectory prediction. Extensive real-world experiments and benchmarks are performed to validate our framework. Moreover, our code will be released to benefit related researches.

The visibility of targets determines performance and even success rate of various applications, such as active slam, exploration, and target tracking. Therefore, it is crucial to take the visibility of targets into explicit account in trajectory planning. In this paper, we propose a general metric for target visibility, considering observation distance and angle as well as occlusion effect. We formulate this metric into a differentiable visibility cost function, with which spatial trajectory and yaw can be jointly optimized. Furthermore, this visibility-aware trajectory optimization handles dynamic feasibility of position and yaw simultaneously. To validate that our method is practical and generic, we integrate it into a customized quadrotor tracking system. The experimental results show that our visibility-aware planner performs more robustly and observes targets better. In order to benefit related researches, we release our code to the public.

In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones into an occupancy grid map. To achieve dynamic avoidance, we design a planning method composed of modified kinodynamic path searching and gradient-based optimization. The method leverages manually constructed gradients without maintaining a signed distance field (SDF), making the planning procedure finished in milliseconds. We integrate the above methods into a customized quadrotor system and thoroughly test it in real-world experiments, verifying its effective collision avoidance in dynamic environments.

In autonomous navigation, sensors suffer from massive occlusion in cluttered environments, leaving a significant amount of space unknown. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance. Therefore, aggressiveness and safety cannot be satisfied at the same time. Mimicking human behavior, in this paper, we propose a method based on deep neural network to predict occupancy distribution of unknown space. Specifically, the proposed method utilizes contextual information of environments and prior knowledge to predict obstacle distributions in the occluded space. Our self-supervised learning method use unlabeled and no-ground-truth data and augments the data by simulating navigation trajectories. Our Occupancy Prediction Network is faster than current SOTA scene completion models and is successfully applied to unseen test environments without any refinement. Results show that our predictor leverages the performance of a kinodynamic planner by improving security with no reduction of speed in cluttered environments.

Model order reduction is a modeling method that can facilitate the analysis and the control synthesis of distributed parameter systems, which are governed by partial differential equations. Many techniques have been proposed to further improve the property of the reduced order model based on full state information, which is usually not available in practical applications. In this paper, a reduced-order modeling method based on sparse point measurements is proposed for the benchmark system of flow past a cylinder. This method involves solving the dynamic optimization problems online, whose objective functions measure the differences between the predicted and observed variables on each time horizon. In this method, the sensor placement is also discussed, and we determine the sensor locations depending on a model-free optimization technique. Several numerical examples of different scenarios are simulated not only to illustrate the effectiveness of the proposed method but also to verify the discussion on the sensor placement strategy.

This paper proposes a hierarchical decision-making and control algorithm for the shepherd game, the seventh mission in the International Aerial Robotics Competition (IARC). In this game, the agent (a multirotor aerial robot) is required to contact targets (ground vehicles) sequentially and drive them to a certain boundary to earn score. During the game of 10 min, the agent should be fully autonomous without any human interference. Regarding the lower-level controller and dynamics of the agent, each action takes a duration of time to accomplish. Denoted as an action delay, in this paper, this action duration is nonconstant and is related to the final reward. Therefore, the challenging point is making the agent "aware of time" when applying a certain action. We solve this problem by two approaches: deep Q-networks and lookup table. The action delay predictor in the decision-level is fitted by a lower-level controller. Through simulations by the example of the shepherd game, the effectiveness and efficiency of this approach are validated. This paper helps our team winning the first prize in IARC 2017, and keeps the best record of this mission since it was released in 2013.

The approach of dynamical system (DS) is promising for modeling robot motion, and provides a flexible means of realizing robot learning and control. Accuracy, stability, and learning speed are the three main factors to be considered when learning robot movements from human demonstrations with DS. Some approaches yield stable dynamical systems, but these may result in a poor reproduction performance, while some approaches yield good reproduction performance but are quite complex and time-consuming. In this paper, we address the accuracy-stability-speed issues simultaneously. We present a learning method named the fast and stable modeling for dynamical systems, which is based on the extreme learning machine to efficiently and accurately learn the parameters of the DS as well as to ensure the asymptotic stability at the target. We confirm the proposed approach by performing both 2-D tasks of learning handwriting motions and a set of robot experiments.

For a multirotor micro aerial vehicle (MAV) flying in the outdoor environment, its downside hemisphere has richest visual information. All that information can be obtained by a single fisheye camera with larger than 180 degrees field of view (FOV). Traditionally, the unrestored fisheye image is restored to a flat image before subsequent processing, which is both resource and time consuming. In this paper, to save resource and time, a method of fisheye object detection and localization on the unrestored fisheye image is proposed. A single-stage neural network is built for object detection. To improve the performance of detector, its submodules are designed specifically by combining the central rotational property and severe distortion of the fisheye image. To meet the real-time requirements of onboard computation, the detector is also tuned to be light-weight. After that, the detected objects are localized with assistance of by a data fusion on the fisheye model and MAV sensory data (altitude, attitude, etc.), The experimental results have validated the effectiveness of the proposed methods in this paper.

Reconstructing the shape and position of plasma is an important issue in Tokamaks. Equilibrium and fitting (EFIT) code is generally used for plasma boundary reconstruction in some Tokamaks. However, this magnetic method still has some inevitable disadvantages. In this paper, we present an optical plasma boundary reconstruction algorithm. This method uses EFIT reconstruction results as the standard to create the optimally optical reconstruction. Traditional edge detection methods cannot extract a clear plasma boundary for reconstruction. Based on global contrast, we propose an edge detection algorithm to extract the plasma boundary in the image plane. Illumination in this method is robust. The extracted boundary and the boundary reconstructed by EFIT are fitted by same-order polynomials and the transformation matrix exists. To acquire this matrix without camera calibration, the extracted plasma boundary is transformed from the image plane to the Tokamak poloidal plane by a mathematical model, which is optimally resolved by using least squares to minimize the error between the optically reconstructed result and the EFIT result. Once the transform matrix is acquired, we can optically reconstruct the plasma boundary with only an arbitrary image captured. The error between the method and EFIT is presented and the experimental results of different polynomial orders are discussed.

In this paper, we consider a motion sense problem motivated by the International Aerial Robotics Competition Mission-7, where an aerial robot is required to provide detection and estimation about mobile vehicles. Dense optical flow computing is employed first to provide a velocity field from image sequences. Then, region growing based on the optical flow field is used to extract moving objects on the background, and motion estimation is eventually achieved while both camera and objects are moving. In addition, classical optical flow techniques do not work in the competition since there may be illumination changes, such as flashlights and reflections in the arena. To deal with this problem, the procedures of the brightness constancy relaxation and intensity normalization are combined in the optical flow algorithm. Experimental results have demonstrated the robustness against varying illumination. The proposed approach can provide motion estimation results of acceptable accuracy for several benchmark data sets and image sequences generated with micro aerial vehicles.