This paper presents a simulator framework tailored Integrated Aerial Platforms (IAPs) using multiple quadrotors. Our framework prioritizes photo and contact fidelity, achieved through a modular design that balances rendering and dynamics computation. Key features include: i) support for diverse IAP configurations; ii) a customizable physics engine for realistic motion and contact simulation for aerial manipulation; and iii) Unreal Engine 5 for lifelike rendering, with sensor designs for visual-inertial SLAM positioning simulation. We showcase our framework’s versatility through a range of scenarios, including trajectory tracking for both fully and under-actuated IAPs, peg-in-hole and direct wrench control tasks under external wrench influence, tightly-coupled SLAM positioning with physical constraints, and air docking task training and testing using offline-to-online reinforcement learning. Furthermore, we validate our simulator framework’s fidelity by comparing results with real flight data for trajectory tracking and direct wrench control tasks. Our simulator framework promises to be valuable for developing and testing integrated aerial platform systems for aerial manipulation. Note to Practitioners—Motivated by the demand for effective simulation tools for Integrated Aerial Platforms (IAPs), this research addresses a significant gap in the availability of comprehensive simulation platforms designed to meet their unique challenges. This paper presents a high-fidelity simulation platform tailored specifically for IAPs, supporting a variety of configurations and capabilities. The platform not only generates high-fidelity image data and facilitates contact simulation but also serves as a vital resource for advancing perception, control, and learning for IAPs. By offering a robust simulation environment, this work aims to bridge the divide between theoretical research and practical applications, ultimately driving advancements in the field of aerial robotics.

Integrated Aerial Platforms (IAPs), consisting of multiple interconnected aircraft, offer a promising framework for aerial manipulation tasks by enhancing localization accuracy and reliability. Unlike aerial swarms, the interconnected nature of IAPs allows for exploiting physical constraints among aircraft to improve positioning and navigation systems. In this paper, we present an advanced decentralized multi-aircraft visual-inertial-range-physical odometry system that considers the position, velocity, and attitude constraints inherent to IAPs. By tightly fusing visual-inertial-range odometry and Ultra-Wideband (UWB) with kinematic constraints, we optimize odometry accuracy through the use of novel constraint-based methods. Our algorithm’s performance is validated on simulated datasets and self-collected datasets, with flight experiments conducted on the IAP, demonstrating a significant improvement with a 28.7% reduction in drift over the baseline on real-world datasets and a 24.5% reduction on simulation datasets.

RGB-D Salient Object Detection (SOD) aims to segment the most prominent areas and objects in a given pair of RGB and depth images. Most current models adopt a dual-stream structure to extract information from both RGB and depth images. However, this leads to an exponential increase in the number of parameters and computations in the model. Moreover, the discrepancy between RGB pretrained and the 3D geometric relationships in depth maps present a challenge for the encoder in capturing spatial structural details. These issues impact the model's accuracy in locating salient objects and distinguishing edge details. To address these, we propose a novel early feature fusion network, named FasterSal, which enables more efficient RGB-D SOD. FasterSal uses a single stream structure to receive RGB images and depth maps, extracting features based on the 3D geometric relationships in the depth map while fully leveraging the pretrained RGB encoder. This approach effectively avoids the inconsistencies between depth modality and the RGB pretrained encoder. It also significantly reduces the number of network parameters while maintaining efficient feature encoding capabilities. To achieve finer edge learning, the detail-aware loss and texture enhancement module are introduced. These modules are designed to extract latent details in high-frequency component features and to enhance the edge learning capability of the model using distance information. Experimental results on several benchmark datasets confirm the effectiveness and superiority of our method over the state-of-the-art approaches, achieving a good balance between performance and speed with only 3.4 million parameters and a CPU operating speed of 63 FPS.

Connecting multiple aerial vehicles to a rigid central platform through passive spherical joints holds the potential to construct a fully-actuated aerial platform. The integration of multiple vehicles enhances efficiency in tasks like mapping and object reconnaissance. This paper proposes a control and state estimation framework for the Integrated Aerial Platform (IAP), enabling it to perform versatile tasks like object reconnaissance and physical interactive tasks with only onboard sensors. In the framework, the 6D motion control serves as the low-level controller, while the high-level controller comprises a 6D admittance filter and a perception-aware attitude correction module. The 6D admittance filter, serving as the interaction controller, is adaptable for aerial interaction tasks. The perception-aware attitude correction algorithm is carefully designed by adopting a geometric Model Predictive Controller (MPC). This algorithm, incorporating both offline and online calculations, proves to be well-suited for the intricate dynamics of an IAP. A 6D direct wrench controller is also developed for the IAP. Notably, both the interaction controller and the direct wrench controller operate without reliance on force/torque sensors. Instead, a wrench observer algorithm is devised, considering external disturbances. Additionally, based on the kinematics constraints of the multiple aerials in the platform, a fusion algorithm for multiple Visual-Inertial Odometry (VIO) and kinematics constraints is developed, providing more accurate localization. A prototype of the IAP is constructed, and its capabilities are demonstrated through experiments including perception-aware object reconnaissance, aerial mapping, aerial peg-in-hole task, and 6D contact wrench generation. All experiments are conducted exclusively with onboard sensors. These tasks exemplify the merits of the proposed IAP and validate the effectiveness of the proposed control framework and fusion algorithm.

Integrated aerial Platforms (IAPs), comprising multiple aircrafts, are typically fully actuated and hold significant potential for aerial manipulation tasks. Differing from a multiple aerial swarm, the aircrafts within the IAP are interconnected, presenting promising opportunities for enhancing localization. Incorporating the physical constraints of these multiple aircrafts to improve the accuracy and reliability of integrated aircraft positioning and navigation systems is a challenging yet highly significant problem. In this paper, we introduce a distributed multi-aircraft visual-inertial-range odometry system that analyzes the position, velocity, and attitude constraints within the IAP. Leveraging constraint relationships in the IAP, we propose corresponding methods that tightly fuse visual-inertial-range odometry and kinematic constraints to optimize odometry accuracy. Our system’s performance is validated using a collected dataset, resulting in a notable 28.7% reduction in drift compared to the baseline.

In this paper, we introduce PADUAV, a novel 5-DOF aerial platform designed to overcome the limitations of traditional tiltrotor vehicles. PADUAV features a unique mechanical design that incorporates two off-the-shelf quadrotors passively articulated to a rigid frame. This innovation enables free pitch rotation without mechanical constraints like cable winding, significantly enhancing its capabilities for various tasks. To control PADUAV’s 5 degrees of freedom, we propose a versatile and straightforward 5-DOF geometric tracking control strategy that generates 2D force and 3D torque. A decomposition approach is designed to distribute the output to the torque and thrust commands for each subplane, with no need for complex optimization. We validate our approach through three simulation experiments conducted in the Gazebo environment, leveraging the utilities provided by the RotorS simulator. These experiments not only demonstrate the feasibility of our platform but also provide new perspectives for future aerial platform development, particularly in terms of simulation-based approaches.

A compliant control model based on reinforcement learning (RL) is proposed to allow robots to interact with the environment more effectively and autonomously execute force control tasks. The admittance model learns an optimal adjustment policy for interactions with the external environment using RL algorithms. The model combines energy consumption and trajectory tracking of the agent state using a cost function. Therein, an Unmanned Aerial Vehicle (UAV) can operate stably in unknown environments where interaction forces exist. Furthermore, the model ensures that the interaction process is safe, comfortable, and flexible while protecting the external structures of the UAV from damage. To evaluate the model performance, we verified the approach in a simulation environment using a UAV in three external force scenes. We also tested the model across different UAV platforms and various low-level control parameters, and the proposed approach provided the best results.

In this paper, a safe trajectory planning and tracking algorithm for autonomous vehicles is proposed. Specially, the safety problem considers the geometric constraints including the obstacle avoiding and the road side constraints, and the non-convex input constraints defined from the sideslip angle of the wheels and input boundedness. Control barrier function (CBF) is adopted to deal with the state and input constraints and generate nominal trajectory. For this purpose, the nominal dynamics of the autonomous vehicle is defined as the virtual dynamics, from which the CBF safety certificates are derived. By constructing appropriate feedback control, the tracking error of the actual trajectory can be bounded into a tube, which guarantees the geometric safety of the actual vehicle. Two safety certificates, the bearing and the distance safety certificates are derived for multiple-obstacle avoidance. In order to deal with the non-convex input constraints, a safe braking maneuver is carefully considered. The feasible initial velocity set for safe braking is proposed as a part of state constraints. The feasibility and the safety of the overall system is proved. The algorithm to synthesis the CBF certificates for multiple-obstacle avoidance, and the input constraints is proposed. Simulation results from an autonomous vehicle including disturbances demonstrate the feasibility of the algorithm. The software implementation of the proposed algorithm was developed in C++ intended for real-world testing.

In this paper, a multi-propeller multifunction aerial robot (MMAR) capable of flying, wall-climbing and arm-operating is presented. Four propellers are devoted to the attitude control of the robot, and two manipulators are designed for the wall-climbing and arm-operating modes. When the aerial robot works in wall-climbing and arm-operating, there are dynamics coupling between the manipulators and the main body. The dynamics of manipulators depends on the motion of the main body, and the motion of manipulators will have reaction force and torque applied on the main body. The dynamics modeling of the robot is investigated by using recursive method. Based on the model, the trajectory linearization control of the robot is proposed. The controller of the robot when it transitions between its flight mode and wall-climbing mode is then designed. The simulation verification of the controller is presented, which can verified the feasibility of the controller.