Robotic simulators play a crucial role in the development and testing of autonomous systems, particularly in the realm of Uncrewed Aerial Vehicles (UAV). However, existing simulators often lack high-level autonomy, hindering their immediate applicability to complex tasks such as autonomous navigation in unknown environments. This limitation stems from the challenge of integrating realistic physics, photorealistic rendering, and diverse sensor modalities into a single simulation environment. At the same time, the existing photorealistic UAV simulators use mostly hand-crafted environments with limited environment sizes, which prevents the testing of long-range missions. This restricts the usage of existing simulators to only low-level tasks such as control and collision avoidance. To this end, we propose the novel FlightForge UAV opensource simulator. FlightForge offers advanced rendering capabilities, diverse control modalities, and, foremost, procedural generation of environments. Moreover, the simulator is already integrated with a fully autonomous UAV system capable of long-range flights in cluttered unknown environments. The key innovation lies in novel procedural environment generation and seamless integration of high-level autonomy into the simulation environment. Experimental results demonstrate superior sensor rendering capability compared to existing simulators, and also the ability of autonomous navigation in almost infinite environments.

In this paper, we address the problem of tracking high-speed agile trajectories for Unmanned Aerial Vehicles (UAVs), where model inaccuracies can lead to large tracking errors. Existing Nonlinear Model Predictive Controller (NMPC) methods typically neglect the dynamics of the low-level flight controllers such as underlying PID controller present in many flight stacks, and this results in suboptimal tracking performance at high speeds and accelerations. To this end, we propose a novel NMPC formulation, LoL-NMPC, which explicitly incorporates low-level controller dynamics and motor dynamics in order to minimize trajectory tracking errors while maintaining computational efficiency. By leveraging linear constraints inside low-level dynamics, our approach inherently accounts for actuator constraints without requiring additional reallocation strategies. The proposed method is validated in both simulation and real-world experiments, demonstrating improved tracking accuracy and robustness at speeds up to 98.57 km h−1and accelerations of 3.5 g. Our results show an average 21.97 % reduction in trajectory tracking error over standard NMPC formulation, with LoL-NMPC maintaining real-time feasibility at 100 Hz on an embedded ARM-based flight computer.

In this article, we introduce an algorithm designed to address the problem of time-optimal formation reshaping in three-dimensional environments while preventing collisions between agents. The utility of the proposed approach is particularly evident in mobile robotics, where agents benefit from being organized and navigated in formation for a variety of real-world applications requiring frequent alterations in formation shape for efficient navigation or task completion. Given the constrained operational time inherent to battery-powered mobile robots, the time needed to complete the formation reshaping process is crucial for their efficient operation, especially in case of multi-rotor uncrewed aerial vehicles (UAVs). The proposed collision-aware time-optimal formation reshaping algorithm (CAT-ORA) builds upon the Hungarian algorithm for the solution of the robot-to-goal assignment implementing the interagent collision avoidance through direct constraints on mutually exclusive robot-goal pairs combined with a trajectory generation approach minimizing the duration of the reshaping process. Theoretical validations confirm the optimality of CAT-ORA, with its efficacy further showcased through simulations, and a real-world outdoor experiment involving 19 UAVs. Thorough numerical analysis shows the potential of CAT-ORA to decrease the time required to perform complex formation reshaping tasks by up to 49%, and 12% on average compared to commonly used methods in randomly generated scenarios.

This paper introduces a control architecture for real-time and onboard control of Unmanned Aerial Vehicles (UAVs) in environments with obstacles using the Model Predictive Path Integral (MPPI) methodology. MPPI allows the use of the full nonlinear model of UAV dynamics and a more general cost function at the cost of a high computational demand. To run the controller in real-time, the sampling-based optimization is performed in parallel on a graphics processing unit onboard the UAV. We propose an approach to the simulation of the nonlinear system which respects low-level constraints, while also able to dynamically handle obstacle avoidance, and prove that our methods are able to run in real-time without the need for external computers. The MPPI controller is compared to MPC and SE(3) controllers on the reference tracking task, showing a comparable performance. We demonstrate the viability of the proposed method in multiple simulation and real-world experiments, tracking a reference at up to 44 km h−1and acceleration close to 20 m s−2, while still being able to avoid obstacles. To the best of our knowledge, this is the first method to demonstrate an MPPI-based approach in real flight.

Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications comes the need for highly robust, safety-critical algorithms that can operate drones in complex and uncertain environments. Additionally, flying fast enables drones to cover more ground, increasing productivity and further strengthening their use case. One proxy for developing algorithms used in high-speed navigation is the task of autonomous drone racing, where researchers program drones to fly through a sequence of gates and avoid obstacles as quickly as possible using onboard sensors and limited computational power. Speeds and accelerations exceed over 80 kph and 4 g, respectively, raising significant challenges across perception, planning, control, and state estimation. To achieve maximum performance, systems require real-time algorithms that are robust to motion blur, high dynamic range, model uncertainties, aerodynamic disturbances, and often unpredictable opponents. This survey covers the progression of autonomous drone racing across model-based and learning-based approaches. We provide an overview of the field, its evolution over the years, and conclude with the biggest challenges and open questions to be faced in the future.

Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time. However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera's limited field of view and the underactuated nature of a quadrotor. We propose a learning-based system that achieves perception-aware, agile flight in cluttered environments. Our method combines imitation learning with reinforcement learning (RL) by leveraging a privileged learning-by-cheating framework. Using RL, we first train a perception-aware teacher policy with full-state information to fly in minimum time through cluttered environments. Then, we use imitation learning to distill its knowledge into a vision-based student policy that only perceives the environment via a camera. Our approach tightly couples perception and control, showing a significant advantage in computation speed (10×faster) and success rate. We demonstrate the closed-loop control performance using hardware-in-the-loop simulation. Video: https://youtu.be/9q059CFGcVA

In multi-goal path planning, the task is to find a sequence to visit a set of target locations in an environment using the shortest path. Finding the order can be achieved by solving an instance of the Traveling Salesman Problem (TSP). This requires to determine paths and their lengths between the individual targets, which is solved using robotic path planning. In this paper, we propose a randomized planner for multi-goal path planning for non-holonomic mobile robots. Multiple trees are constructed simultaneously from the targets and expanded by collision-free configurations until they touch each other or obstacles. The trees are expanded using Dubins maneuvers. Therefore, the resulting trajectories satisfy the kinematic constraints of the mobile robots and they can also be used by Unmanned Aerial Vehicles flying at a constant altitude. The efficiency of the proposed planning approach is demonstrated in the multi-goal path planning for environments with tens of targets and compared to state-of-the-art approaches.

Path planning of 3D solid objects has many practical applications. Sampling-based planners like Rapidly-exploring Random Tree (RRT) tackle the problem by the randomized sampling of the configuration space. The well-known issue of the sampling-based planners is the narrow passage problem. Narrow passages are small collision-free regions in the configuration space that are, due to their low volume, difficult to cover by the random samples, which prevents the sampling-based planners from finding a path leading through the passages. To increase the success rate of the planners, the search in the configuration space can be guided using an approximate solution. An approximate solution is found, for example, considering a smaller object (robot). The performance of this planning concept depends on the way of reducing the size of the object and the ability to find an initial solution. In this paper, we propose a novel technique to reduce the geometry of the object by a combination of triangulation and iterative removal of surface triangles. This technique is suitable for both convex and non-convex objects. In many real-world applications, narrow passages can also contain the initial or goal configurations. We thus propose the extension of the goal region to increase the probability of finding the initial approximate solution. Experiments have shown that the proposed modification outperforms the simple guiding using approximate solutions, as well as other related state-of-the-art planners.

In multi-goal path planning, the task is to find a sequence to visit a set of target locations in an environment. The combinatorial part of the problem (finding the sequence) can be solved as an instance of Traveling Salesman Problem, which requires knowledge about collision-free paths (and distances) between the individual targets. Finding the collision-free paths between the targets is essential in this task. Sampling-based planners like Probabilistic Roadmaps (PRM) and Rapidly-exploring Random Tree (RRT) can be used to find these paths. However, PRM can be computationally demanding, as it attempts to connect each node in the roadmap to its neighbors, regardless of their later usage in the solution. Contrary, RRT is a tree-based planner, and one run can only provide paths starting in the root of the tree (a single target). In this paper, we propose a novel planner for multi-goal path planning. Multiple trees (forest) are constructed simultaneously from the targets and expanded by collision-free configurations until they touch each other or obstacles. Each tree, therefore, does not explore the whole configuration space (as in the case of RRT), and its construction is faster than PRM, as it uses lower number of edges. The efficiency of this new planning approach is demonstrated in the multi-goal path planning in 2D environment with tens of targets and with narrow passages.

This paper introduces the Close Enough Orienteering Problem (CEOP) for planning missions with multi-rotor aerial vehicles considering their maximal velocity and acceleration limits. The addressed problem stands to select the most rewarding target locations and sequence to visit them in the given limited travel budget. The reward is collected within a non-zero range from a particular target location that allows saving the travel cost, and thus collect more rewards. Hence, we are searching for the fastest trajectories to collect the most valuable rewards such that the motion constraints are not violated, and the travel budget is satisfied. We leverage on existing trajectory parametrization based on Bézier curves recently deployed in surveillance planning using unsupervised learning, and we propose to employ the learning in a solution of the introduced multi-vehicle CEOP. Feasibility of the proposed approach is supported by empirical evaluation and experimental deployment using multi-rotor vehicles.

Sampling-based motion planners are often used to solve motion planning problems for robots with many degrees of freedom. These planners explore the related configuration space by random sampling. The well-known issue of the sampling-based planners is the narrow passage problem. Narrow passages are small collision-free regions in the configuration space that are, due to their volume, difficult to cover by the random samples. The volume of the narrow passages can be artificially increased by reducing the size of the robot, e.g., by scaling-down its geometry, which increases the probability of placing the random samples into the narrow passages. This allows us to find an approximate solution (trajectory) and use it as a guide to find the solution for a larger robot. Guiding along an approximate solution may, however, fail if this solution leads through such parts of the configuration space that are not reachable or traversable by a larger robot. To improve this guiding process, we propose to compute several approximate solutions leading through different parts of the configuration space, and use all of them to guide the search for a larger robot. We introduce the concept of disabled regions that are prohibited from the exploration using the sampling process. The disabled regions are defined using trajectories already found in the space being searched. The proposed method can solve planning problems with narrow passages with higher success rate than other state-of-the-art planners.

In this paper, we address a generalization of the Orienteering Problem (OP) for curvature-constrained vehicles and to problems where it is allowed to collect a reward associated to each target location within a specified distance from the target. The addressed problem combines challenges of the combinatorial optimization of the OP (to select the most rewarding targets and find the optimal sequence to visit them) with the continuous optimization related to the determination of the waypoint locations and suitable headings at the waypoints for the considered Dubins vehicle such that the curvature-constrained path does not exceed the given travel budget and the sum of the collected rewards is maximized. The proposed generalization is called the Close Enough Dubins Orienteering Problem (CEDOP) and novel unsupervised learning approach is proposed to address computational requirements of this challenging planning problem. Based on the presented results, the proposed approach is feasible and provides a bit worse solution of CEDOP than the existing combinatorial approach but with significantly lower computational requirements.

In this paper, we address the Orienteering problem (OP) by the unsupervised learning of the self-organizing map (SOM). We propose to solve the OP with a new algorithm based on SOM for the Traveling salesman problem (TSP). Both problems are similar in finding a tour visiting the given locations; however, the OP stands to determine the most valuable tour that maximizes the rewards collected by visiting a subset of the locations while keeping the tour length under the specified travel budget. The proposed stochastic search algorithm is based on unsupervised learning of SOM and it constructs a feasible solution during each learning epoch. The reported results support feasibility of the proposed idea and show the performance is competitive with existing heuristics. Moreover, the key advantage of the proposed SOM-based approach is the ability to address the generalized OP with Neighborhoods, where rewards can be collected by traveling anywhere within the neighborhood of the locations. This problem generalization better fits data collection missions with wireless data transmission and it allows to save unnecessary travel costs to visit the given locations.