This paper presents a simulator framework tailored Integrated Aerial Platforms (IAPs) using multiple quadrotors. Our framework prioritizes photo and contact fidelity, achieved through a modular design that balances rendering and dynamics computation. Key features include: i) support for diverse IAP configurations; ii) a customizable physics engine for realistic motion and contact simulation for aerial manipulation; and iii) Unreal Engine 5 for lifelike rendering, with sensor designs for visual-inertial SLAM positioning simulation. We showcase our framework’s versatility through a range of scenarios, including trajectory tracking for both fully and under-actuated IAPs, peg-in-hole and direct wrench control tasks under external wrench influence, tightly-coupled SLAM positioning with physical constraints, and air docking task training and testing using offline-to-online reinforcement learning. Furthermore, we validate our simulator framework’s fidelity by comparing results with real flight data for trajectory tracking and direct wrench control tasks. Our simulator framework promises to be valuable for developing and testing integrated aerial platform systems for aerial manipulation. Note to Practitioners—Motivated by the demand for effective simulation tools for Integrated Aerial Platforms (IAPs), this research addresses a significant gap in the availability of comprehensive simulation platforms designed to meet their unique challenges. This paper presents a high-fidelity simulation platform tailored specifically for IAPs, supporting a variety of configurations and capabilities. The platform not only generates high-fidelity image data and facilitates contact simulation but also serves as a vital resource for advancing perception, control, and learning for IAPs. By offering a robust simulation environment, this work aims to bridge the divide between theoretical research and practical applications, ultimately driving advancements in the field of aerial robotics.

Integrated Aerial Platforms (IAPs), consisting of multiple interconnected aircraft, offer a promising framework for aerial manipulation tasks by enhancing localization accuracy and reliability. Unlike aerial swarms, the interconnected nature of IAPs allows for exploiting physical constraints among aircraft to improve positioning and navigation systems. In this paper, we present an advanced decentralized multi-aircraft visual-inertial-range-physical odometry system that considers the position, velocity, and attitude constraints inherent to IAPs. By tightly fusing visual-inertial-range odometry and Ultra-Wideband (UWB) with kinematic constraints, we optimize odometry accuracy through the use of novel constraint-based methods. Our algorithm’s performance is validated on simulated datasets and self-collected datasets, with flight experiments conducted on the IAP, demonstrating a significant improvement with a 28.7% reduction in drift over the baseline on real-world datasets and a 24.5% reduction on simulation datasets.

Connecting multiple aerial vehicles to a rigid central platform through passive spherical joints holds the potential to construct a fully-actuated aerial platform. The integration of multiple vehicles enhances efficiency in tasks like mapping and object reconnaissance. This paper proposes a control and state estimation framework for the Integrated Aerial Platform (IAP), enabling it to perform versatile tasks like object reconnaissance and physical interactive tasks with only onboard sensors. In the framework, the 6D motion control serves as the low-level controller, while the high-level controller comprises a 6D admittance filter and a perception-aware attitude correction module. The 6D admittance filter, serving as the interaction controller, is adaptable for aerial interaction tasks. The perception-aware attitude correction algorithm is carefully designed by adopting a geometric Model Predictive Controller (MPC). This algorithm, incorporating both offline and online calculations, proves to be well-suited for the intricate dynamics of an IAP. A 6D direct wrench controller is also developed for the IAP. Notably, both the interaction controller and the direct wrench controller operate without reliance on force/torque sensors. Instead, a wrench observer algorithm is devised, considering external disturbances. Additionally, based on the kinematics constraints of the multiple aerials in the platform, a fusion algorithm for multiple Visual-Inertial Odometry (VIO) and kinematics constraints is developed, providing more accurate localization. A prototype of the IAP is constructed, and its capabilities are demonstrated through experiments including perception-aware object reconnaissance, aerial mapping, aerial peg-in-hole task, and 6D contact wrench generation. All experiments are conducted exclusively with onboard sensors. These tasks exemplify the merits of the proposed IAP and validate the effectiveness of the proposed control framework and fusion algorithm.

Integrated aerial Platforms (IAPs), comprising multiple aircrafts, are typically fully actuated and hold significant potential for aerial manipulation tasks. Differing from a multiple aerial swarm, the aircrafts within the IAP are interconnected, presenting promising opportunities for enhancing localization. Incorporating the physical constraints of these multiple aircrafts to improve the accuracy and reliability of integrated aircraft positioning and navigation systems is a challenging yet highly significant problem. In this paper, we introduce a distributed multi-aircraft visual-inertial-range odometry system that analyzes the position, velocity, and attitude constraints within the IAP. Leveraging constraint relationships in the IAP, we propose corresponding methods that tightly fuse visual-inertial-range odometry and kinematic constraints to optimize odometry accuracy. Our system’s performance is validated using a collected dataset, resulting in a notable 28.7% reduction in drift compared to the baseline.