For Unmanned Aerial Vehicle (UAV) swarms operating in large-scale unknown environments, lightweight long-range mapping is crucial for enhancing safe navigation. Traditional stereo cameras constrained by a short fixed baseline suffer from limited perception ranges. To overcome this limitation, we present Flying Co-Stereo, a cross-agent collaborative stereo vision system that leverages the wide-baseline spatial configuration of two UAVs for long-range dense mapping. However, realizing this capability presents several challenges. First, the independent motion of each UAV leads to a dynamic and continuously changing stereo baseline, making accurate and robust estimation difficult. Second, efficiently establishing feature correspondences across independently moving viewpoints is constrained by the limited computational capacity of onboard edge devices. To tackle these challenges, we introduce the Flying Co-Stereo system within a novel Collaborative Dynamic-Baseline Stereo Mapping (CDBSM) framework. We first develop a dual-spectrum visual-inertial ranging estimator to achieve robust and precise online estimation of the baseline between the two UAVs. In addition, we propose a hybrid feature association strategy that integrates cross-agent feature matching—based on a computationally intensive yet accurate deep neural network—with intra-agent, optical-flow based lightweight feature tracking. Furthermore, benefiting from the wide baselines between the two UAVs, our system accurately recovers long-range co-visible 3D sparse points. We then employ a monocular depth network to predict up-to-scale dense depth maps, which are refined using accurate metric scales derived from the triangulated sparse points via exponential fitting. Extensive real-world experiments demonstrate that the proposed Flying Co-Stereo system achieves robust and accurate dynamic baseline estimation in complex environments while maintaining efficient feature matching with resource-constrained computers under varying viewpoints. Ultimately, our system achieves dense 3D mapping at distances of up to 70 meters with a relative error between 2.3% and 9.7%. This corresponds to up to a 350% improvement in maximum perception range and up to a 450% increase in coverage area compared to conventional stereo vision systems with fixed compact baselines.

Representing the 3D environment with instance-aware semantic and geometric information is crucial for interaction-aware robots in dynamic environments. Nevertheless, creating such a representation poses challenges due to sensor noise, instance segmentation and tracking errors, and the objects' dynamic motion. This paper introduces a novel particle-based instance-aware semantic occupancy map to tackle these challenges. Particles with an augmented instance state are used to estimate the Probability Hypothesis Density (PHD) of the objects and implicitly model the environment. Utilizing a State-augmented Sequential Monte Carlo PHD (S$^{2}$MC-PHD) filter, these particles are updated to jointly estimate occupancy status, semantic, and instance IDs, mitigating noise. Additionally, a memory module is adopted to enhance the map's responsiveness to previously observed objects. Experimental results on the Virtual KITTI 2 dataset demonstrate that the proposed approach surpasses state-of-the-art methods across multiple metrics under different noise conditions. Subsequent tests using real-world data further validate the effectiveness of the proposed approach.

For a swarm of Unmanned Aerial Vehicle (UAV), long-distance visual mapping is advantageous for pre-planning navigation paths in unknown urban building environments. This work leverages two cameras on two UAVs to build a wide-baseline collaborative stereo camera, which can construct a navigable mesh map for remote building obstacles. We present a complete framework of the collaborative stereo camera for long-distance mapping, including online extrinsic parameter estimation of the stereo camera, real-time cross-camera feature association, and semantic mesh map generation of remote buildings. Extensive simulations and real-world experiments verify the effectiveness of the collaborative stereo camera. With a 3m baseline, the collaborative stereo camera achieves long-distance mapping of buildings (20m ~ 50m) away with a relative error of approximately 10%. The constructed remote map enables UAVs to pre-detect large obstacles and pre-plan navigation paths in large-scale building environments. Hopefully, this work can provide a novel and practical approach for collaborative visual tasks of UAV swarm.Video - https://youtu.be/a0kj-1zb6KI

Perching on the surface of moving objects, like vehicles, could extend the flight time and range of quadrotors. Suction cups are usually adopted for surface attachment due to their durability and large adhesive force. To seal on a surfaces, suction cups must be aligned with the surface and possess proper relative tangential velocity. However, quadrotors’ attitude and relative velocity errors would become significant when the object surface is moving and inclined. To address this problem, we proposed a real-time trajectory planning algorithm. The time-optimal aggressive trajectory is efficiently generated through multimodal search in a dynamic time-domain. The velocity errors relative to the moving surface are alleviated. To further adapt to the residual errors, we design a compliant gripper using self-sealing cups. Multiple cups in different directions are integrated into a wheel-like mechanism to increase the tolerance to attitude errors. The wheel mechanism also eliminates the requirement of matching the attitude and tangential velocity. Extensive tests are conducted to perch on static and moving surfaces at various inclinations. Results demonstrate that our proposed system enables a quadrotor to reliably perch on moving inclined surfaces (up to$1.07m/s$and$90^\circ$) with a success rate of$70\%$or higher. The efficacy of the trajectory planner is also validated. Our gripper has larger adaptability to attitude errors and tangential velocities than conventional suction cup grippers. The success rate increases by 45% in dynamic perches.Note to Practitioners—This paper was motivated by the problem of perching on moving inclined surfaces using quadrotors. It can be used for perching on the various angled surfaces of an automobile to save energy and enlarge flight distance. It can also be exploited in air-ground cooperative tasks. In recent years, various grippers and trajectory planning methods have been devised to enable quadrotors to perch on static inclined surfaces. These strategies can not be used for perching on moving inclined surfaces. The task poses high requirements for the efficiency of trajectory planning and the grippers’ adaptability to pose errors. This paper proposes a real-time planning method to generate trajectories. The trajectories respect kinematic and motor lift constraints. They allow large attitude maneuvering of quadrotors. A multimodal search in a dynamic time-domain is developed to seek the minimum feasible time for the trajectory planner. Considering large pose errors in dynamic perching, a compliant gripper based on multiple independent suction cups is designed. The wheel and multidirectional layout of cups increase the adaptability to attitude and velocity errors. Results suggest that the proposed system is effective and reliable. In practice, our multiple cups would add weight penalty and increase the drag. The suction cups should be placed close to a quadrotor’s central axis to reduce the drag and disturbance in flow. The proposed system relies on an external motion capture system. In future research, we will focus on onboard sensing and control methods to achieve perching on moving inclined surfaces outdoors.

Particle-based dynamic occupancy maps were proposed in recent years to model the obstacles in dynamic environments. Current particle-based maps describe the occupancy status in discrete grid form and suffer from the grid size problem, wherein a large grid size is unfavorable for motion planning while a small grid size lowers efficiency and causes gaps and inconsistencies. To tackle this problem, this article generalizes the particle-based map into continuous space and builds an efficient 3-D egocentric local map. A dual-structure subspace division paradigm, composed of a voxel subspace division and a novel pyramid-like subspace division, is proposed to propagate particles and update the map efficiently with the consideration of occlusions. The occupancy status at an arbitrary point in the map space can then be estimated with the weights of the particles. To reduce the noise in modeling static and dynamic obstacles simultaneously, an initial velocity estimation approach and a mixture model are utilized. Experimental results show that our map can effectively and efficiently model both dynamic obstacles and static obstacles. Compared to the state-of-the-art grid-form particle-based map, our map enables continuous occupancy estimation and substantially improves the mapping performance at different resolutions.

This paper proposes a perception-shared and swarm trajectory global optimal (STGO) algorithm fused UAVs formation motion planning framework aided by an active sensing system. First, the point cloud received by each UAV is fit by the gaussian mixture model (GMM) and transmitted in the swarm. Resampling from the received GMM contributes to a global map, which is used as the foundation for consensus. Second, to improve flight safety, an active sensing system is designed to plan the observation angle of each UAV considering the unknown field, overlap of the field of view (FOV), velocity direction and smoothness of yaw rotation, and this planning problem is solved by the distributed particle swarm optimization (DPSO) algorithm. Last, for the formation motion planning, to ensure obstacle avoidance, the formation structure is allowed for affine transformation and is treated as the soft constraint on the control points of the B-spline. Besides, the STGO is introduced to avoid local minima. The combination of GMM communication and STGO guarantees a safe and strict consensus between UAVs. Tests on different formations in the simulation show that our algorithm can contribute to a strict consensus and has a success rate of at least 80% for obstacle avoidance in a dense environment. Besides, the active sensing system can increase the success rate of obstacle avoidance from 50% to 100% in some scenarios.

In this article, an artificially weighted spanning tree coverage (AWSTC) algorithm is proposed for the distributed path planning of multiple flying robots. To balance well the efficiency, redundancy, and robustness in the cooperative coverage problem, each robot simultaneously constructs its spanning tree, which grows toward the center of inertia of the uncovered area and keeps away from the trees of its partners. Based on this, each robot covers a concerned area with almost equal trajectory length and computational burden. To guarantee dynamical consistency, a trajectory-smoothing method is developed utilizing Bézier curve transition. As transition in the spanning tree path planning is always conducted around the corner with right angle, geometric and velocity profiles of this kind of transition can be preevaluated as a basic component and then connected to establish more complex real-time trajectories according to the constructed spanning trees. Numerical evaluations and real-time flight experiments are carried out at last. Results demonstrate that the proposed strategy can generate a smooth trajectory for an area coverage problem while ensuring efficiency and robustness. In particular, the efficiency of the proposed algorithm is quite close to the typical centralized spanning tree coverage (STC) algorithm, while ensuring the fulfillment of the coverage task regardless of any breakdown in the individual robot.Note to Practitioners—In applications such as infrastructure inspection and plant protection, the concerned area or terrain is required to be fully covered with the smallest possible time consumption. In view of these real-world requirements, multiple unmanned systems with proper path planning provide a promising solution that can significantly enhance the efficiency as well as the robustness, compared with a single-robot system. Especially, centralized STC algorithms, concerning the computation efficiency and trajectory redundancy of multiple unmanned systems, demonstrate satisfactory effectiveness, and thereby receive a lot of attention. Unfortunately, most of the STC algorithms are proposed in a centralized control framework. Therefore, it is worth to study its decentralized version that could exploit the advantages of the distributed multiple robots. With this consideration, a decentralized AWSTC algorithm is proposed in this article. This spanning tree-based strategy artificially assigns priority weight for each cell, with respect to each individual robot, in the concerned area. With specially designed weight assignment, each robot tends to construct a spanning tree with cells that are uncovered but keep away from the trees of its partners. Based on this, the task burden as well as computational cost of each robot are almost equal. To improve the real-time performance, a Bézier transition trajectory-generation method is also used. With proper parameter selection, robots can generate smooth trajectory with constant velocity in real-time missions. Numerical evaluations and experiments with real-time flight demonstrate the effectiveness of this methodology.

This paper presents a trajectory tracking control strategy based on the subspace stabilization approach to accurately manipulate an under-actuated flying robot from a known initial state to the desired terminal state. To facilitate the development of this tracking strategy, the dynamical model of the quadrotor is firstly proposed. Subsequently, an optimal trajectory generation algorithm is adopted to generate dynamically consistent trajectories regarding the initial and terminal state constraints in specific missions. Then, a trajectory tracking control strategy based on the subspace stabilization approach is developed considering the lumped disturbances and time delays. The developed control strategy is applied for ball juggling of a highly under-actuated quadrotor, which is a popular flying robot in recent years. Real-time experimental results show that the quadrotor can be accurately manipulated from a known initial state to the desired terminal state within a given time horizon. In the consecutive juggling tasks, the quadrotor with a racket of radius 0.065 m can consecutively juggle the ball for averagely 4 hits in each rally, and a longest rally achieved by the developed control strategy is 14 hits. The feasibility of the developed control strategy is also preliminarily verified through the cooperative juggling between two quadrotors. All of these results demonstrate the effectiveness of the developed control strategy.