Combinatorial optimization problems involving multiple agents are notoriously challenging due to their NP-hard nature and the necessity for effective agent coordination. Despite advancements in learning-based methods, existing approaches often face critical limitations, including suboptimal agent coordination, poor generalization, and high computational latency. To address these issues, we propose PARCO (Parallel AutoRegressive Combinatorial Optimization), a general reinforcement learning framework designed to construct high-quality solutions for multi-agent combinatorial tasks efficiently. To this end, PARCO integrates three key novel components: (1) transformer-based communication layers to enable effective agent collaboration during parallel solution construction, (2) a multiple pointer mechanism for low-latency, parallel agent decision-making, and (3) priority-based conflict handlers to resolve decision conflicts via learned priorities. We evaluate PARCO in multi-agent vehicle routing and scheduling problems, where our approach outperforms state-of-the-art learning methods, demonstrating strong generalization ability and remarkable computational efficiency. We make our source code publicly available to foster future research: https://github.com/ai4co/parco.

Reinforcement learning (RL) has emerged as an effective approach for signal control, and several studies attempt to apply RL to network-wide signal control problems. Most existing methods for network-wide signal controls simply apply the pretrained model independently to the network or simply expand the observation range for the neighbors. While several recent studies have proposed cooperative signal control models using multi-agent reinforcement learning (MARL) approaches, they have not considered the transferability of the policy. This issue becomes more critical in treating a multi-agent signal control problem, since there exist innumerable possibilities of data distributions for the demand patterns. Hence, this research aims to develop a transferable and cooperative multi-agent traffic signal control model. As the key idea, we propose a decentralized and communication-based MARL approach that considers the spatial correlation of traffic dynamics in the model design and employs a graph representation for the state. Three experiments in simulated environments of the Gangnam district in Seoul, South Korea are conducted to 1) validate the model, 2) assess the policy’s transferability, and 3) evaluate the efficiency of the multi-agent cooperation. The experimental results show that the proposed model obtains a transferable policy so that it adapts to the unexperienced demand scenario. In addition, it is concluded that the joint action of the proposed RL model cooperatively rebalances the traffic demands so that it improves the efficiency of the network-wide signal controls.

In this work, we propose a novel method, NLNS-MASPF, to solve the Multi-Agent Scheduling and Pathfinding (MASPF) problem. The problem exhibits a bi-level structure, consisting of High-level Scheduling and Low-level Pathfinding. Our method applies a graph neural network in the high-level scheduling process and utilizes a MAPF solver with a schedule segmenting technique in the low-level pathfinding process. Through these approaches, NLNS-MASPF has experimentally demonstrated superior performance compared to the previous state-of-the-art MASPF algorithm, LNS-PBS, in solving the MASPF problem.

In this work, we consider the problem of allocating tasks and planning paths for multiple robots to operate cooper-atively. We formulate the problem as a bi-level optimization that involves optimizing the scheduling of robots and the collision-free path for each robot. To address the complexity of the environment with obstacles, we introduce a congestion-aware state representation technique with the aid of graph neural networks. We also derive a joint scheduling policy using multi-agent reinforcement learning, and we propose an additional auxiliary loss that promotes coordination among the robots. Through empirical evaluation, we demonstrate the effectiveness of our approach in solving the combined task allocation and path planning problem in a cooperative multi-robot system.