The Multi-Agent Path Finding (MAPF) problem aims to determine the shortest and collision-free paths for multiple agents in a known, potentially obstacle-ridden environment. It is the core challenge for robotic deployments in large-scale logistics and transportation. Decentralized learningbased approaches have shown great potential for addressing the MAPF problems, offering more reactive and scalable solutions. However, existing learning-based MAPF methods usually rely on agents making decisions based on a limited field of view (FOV), resulting in short-sighted policies and inefficient cooperation in complex scenarios. There, a critical challenge is to achieve consensus on potential movements between agents based on limited observations and communications. To tackle this challenge, we introduce a new framework that applies sheaf theory to decentralized deep reinforcement learning, enabling agents to learn geometric cross-dependencies between each other through local consensus and utilize them for tightly cooperative decision-making. In particular, sheaf theory provides a mathematical proof of conditions for achieving global consensus through local observation. Inspired by this, we incorporate a neural network to approximately model the consensus in latent space based on sheaf theory and train it through self-supervised learning. During the task, in addition to normal features for MAPF as in previous works, each agent distributedly reasons about a learned consensus feature, leading to efficient cooperation on pathfinding and collision avoidance. As a result, our proposed method demonstrates significant improvements over state-of-the-art learning-based MAPF planners, especially in relatively large and complex scenarios, demonstrating its superiority over baselines in various simulations and real-world robot experiments.

Many multi-robot deployments, such as automated construction of buildings, distributed search, or cooperative mapping, often require agents to intelligently coordinate their trajectories and form coalition over a large domain, to complete spatially distributed tasks as quickly as possible. We focus on scenarios involving homogeneous robots, but where tasks vary in the number of agents required to start them. For example, construction robots may need to collaboratively air-lift heavy objects at different locations (e.g., prefabricated rooms, crates of material/equipment), where the weight of each payload defines the required coalition size. To balance the total travel time of the agents and their waiting time (before task initiation), agents need to carefully sequence tasks but also dynamically form/disband coalitions. While simpler problems can be approached using heuristics or optimization, these methods struggle with more complex instances involving large task-to-agent ratios, where frequent coalition changes are needed. In this work, we propose to let agents learn to iteratively build cooperative schedules to solve such problems, by casting the problem in the reinforcement learning framework. Our approach relies on an attention-based neural network, allowing agents to reason about the current state of the system to sequence movement decisions that optimize short-term coalition formation and long-term task scheduling. We further propose a novel leader-follower technique to boost cooperation learning and compare our performance to conventional baselines in a wide variety of scenarios. There, our method closely matches or outperforms the baselines; in particular, it yields higher-quality solutions and is at least 2 orders of magnitude faster than exact solver in cases where frequent coalition updates are required.

Mobile edge computing (MEC) offers an opportunity for devices relying on wireless power transfer (WPT), to accomplish computationally demanding tasks. Such WPT-powered MEC systems have yet to be optimized for long-term efficiency, due to random and changing task demands and wireless channel states of the devices. This paper presents an augmented two-staged deep Q-network (DQN), referred to as “TS-DQN,” for online optimization of WPT-powered MEC systems, where the WPT, offloading schedule, channel allocation, and the CPU configurations of the edge server and devices are jointly optimized to minimize the long-term average energy requirement of the systems. The key idea is to design a DQN for learning the channel allocation and task admission, while the WPT, offloading time and CPU configurations are efficiently optimized to precisely evaluate the reward of the DQN and substantially reduce its action space. Another important aspect is that a new action generation method is developed to expand and diversify the actions of the DQN, further accelerating its convergence. As validated by simulations, the proposed TS-DQN is much more energy efficient and converges much faster, than its potential alternative directly using the state-of-the-art Deep Deterministic Policy Gradient algorithm to learn all decision variables.

This paper presents a new two-stage deep Q-network (DQN), referred to as "TS-DQN", for online optimization of wireless power transfer (WPT)-powered mobile edge computing (MEC) systems, where the WPT, offloading schedule, channel allocation, and the CPU configurations of the edge server and devices are jointly optimized to minimize the long-term average energy requirement of the systems. The key idea is to design a DQN to learn the channel allocation and task admission, while the WPT, offloading time and CPU configurations are efficiently optimized to precisely evaluate the reward of the DQN and substantially reduce its action space. A new action generation method is developed to expand and diversify the actions of the DQN, hence further accelerating its convergence. Simulation shows that the gain of the TS-DQN in energy saving is nearly 60% compared to its potential alternatives.

Unmanned aerial vehicle (UAV) swarms require accurate relative localization to safeguard flight missions in the global navigation satellite system-denied environment due to a lack of absolute position information. The existing work of relative localization faces challenges, such as the ranging information loss and the low localization frequency caused by long-distance ranging and large-scale characteristic of the UAV swarm. This article proposes a clustering-based cooperative relative localization scheme for UAV swarm, which contains a two-level framework: inter/intra-cluster localization. In order to investigate the tradeoff between intracluster cooperation and intercluster packet loss, the clustering-based problem is constructed as a coalition formation game (CFG) model. Given the designed coalition value, preference relation, and the coalition formation principles, it is proved that the proposed CFG model has a Nash stable partition. The designed coalition formation algorithm includes coalition heads and beacon drones selection mechanism. Simulation results show that the proposed CFG algorithms shorten the ranging time compared with global localization and achieve better localization performance (localization error and success rate) than contrast algorithms.