Maximizing the endurance of unmanned aerial vehicles (UAVs) in large-scale monitoring missions spanning over large areas requires addressing their limited battery capacity. Deploying unmanned ground vehicles (UGVs) as mobile recharging stations offers a practical solution, extending UAVs' operational range. This introduces the challenge of optimizing UAV-UGV routes for efficient mission point coverage and seamless recharging coordination. In this paper, we present a risk-aware deep reinforcement learning (Ra-DRL) framework with a multi-head attention mechanism within an encoder-decoder transformer architecture to solve this cooperative routing problem for a UAV-UGV team. Our model minimizes mission time while accounting for the stochastic fuel consumption of the UAV, influenced by environmental factors like wind velocity, ensuring adherence to a risk threshold to avoid mid-mission energy depletion. Extensive evaluations on various problem sizes show that our method significantly outperforms nearest-neighbor heuristics in both solution quality and risk management. We validate the Ra-DRL policy in a Gazebo-ROS SITL environment with a PX4-based custom UAV and Clearpath Husky UGV. The results demonstrate the robustness and adaptability of our policy, making it highly effective for mission planning in dynamic, uncertain scenarios.

This paper presents an iterative planning framework for multi-agent systems with hybrid state spaces. The framework uses transition systems to mathematically represent planning tasks and employs multiple solvers to iteratively improve the plan until computational resources are exhausted. When integrating different solvers for iterative planning, we establish theoretical guarantees for recursive feasibility. The proposed framework enables continual improvement of solutions to reduce sub-optimality, efficiently using allocated computational resources. The proposed method is validated by applying it to an energy-aware UAV-UGV cooperative task site assignment problem. The results demonstrate continual solution improvement while preserving real-time implementation ability compared to algorithms proposed in the literature.Note to Practitioners—This paper presents an iterative planning solution for cooperative planning problems in multi-agent systems, which integrates multiple solvers to create an optimization framework. The proposed planning framework has been theoretically validated and applied in an energy-aware cooperative planning scenario for multi-vehicle task site assignments. The proposed framework can be applied to plan for any generalized task site assignment using multiple solvers iteratively.

Unmanned aerial vehicles (UAVs) possess the capability to survey vast areas, yet their operational range is limited by their battery capacity. Deploying mobile recharging stations via unmanned ground vehicles (UGVs) can significantly enhance the endurance and effectiveness of UAVs. However, optimizing the routes for both UAVs and UGVs, referred to as the UAV-UGV cooperative routing problem, requires a sophisticated planning framework to determine the vehicles’ routes and their recharging points. To address this, in this paper, we utilize a deep reinforcement learning (DRL) based framework equipped with multi-head attention layers. The framework is designed to sequentially select actions to construct routes for the UAV and UGV and to establish their rendezvous points for recharging. We evaluate our framework across various problem instance sizes and distributions, comparing it against recent heuristic-based methods and an existing learning-based method as baselines. Our proposed algorithm surpasses these baselines in terms of solution quality and runtime efficiency in the test scenarios, thus proving its effectiveness. Additionally, we investigate the application of our DRL policy in online mission planning to accommodate dynamic changes within the mission scenario.

We introduce and investigate the recharging rendezvous problem for a collaborative team of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs), in which UAVs with limited battery capacity and UGVS persistently monitor an area. The UGVs also act as mobile recharging stations for the UAVs. In contrast to prior work on such problems, we consider the challenge of dealing with stochastic energy consumption in a risk-aware fashion. Specifically, we consider a bi-criteria optimization problem of minimizing the time taken by the UAVs on recharging detours while ensuring that the probability that no UAV runs out of charge is greater than a user-defined risk tolerance. This problem (termed Risk-aware Recharging Rendezvous Problem (RRRP)) is a combinatorial problem with a matching constraint — to ensure UAVs are assigned to the limited UGV recharging slots, and a knapsack constraint — to capture the risk tolerance. We propose a novel bicriteria approximation algorithm to solve RRRP and demonstrate its effectiveness in the context of a persistent monitoring mission compared to baseline methods.

Scalable systems can increase or decrease their size with costs that are proportionate to the resulting change in performance. These costs can be monetary or related to other factors such as integration effort, operator training, or infrastructure upgrades. The options to increase in size to meet growing demand and decrease in size to minimize costs while servicing low demand make scalable systems attractive for completing tasks under uncertainty. Multirobot systems are used in many tasks characterized by uncertainty, such as search and rescue, mapping, and perimeter defense. When human operators interact with multirobot systems, the scalability can be limited by the human's cognitive abilities, decision making speed, and performance under stress. Human problem solving and creativity can also be beneficial to the system to overcome potential scaling challenges. This paper summarizes the literature on humans' effects on scalability of multirobot systems and presents an illustrative simulation of the challenges of scaling a multioperator, multirobot surveillance system.