In recent years, deep multi-agent reinforcement learning (MARL) has demonstrated remarkable potential in solving complex cooperative tasks by enabling decentralized yet efficient coordination among agents. However, during decentralized training, agent policy updates induced by different joint action samples may conflict, leading to gradient interference that hinders convergence and the emergence of coordinated behavior. In this paper, we analyze and empirically validate the phenomenon of gradient interference. To address this, we then propose Gradient-Protected Value Decomposition (GPVD), a novel MARL framework that explicitly protects the gradient signals of optimal collaborative actions by suppressing the impact of interfering actions. GPVD employs a dynamic gradient protection mechanism that identifies optimal collaborative joint actions and reweights the loss to attenuate gradients from non-collaborative interfering actions. To effectively identify high-value collaborative actions, we apply SimHash-based state grouping to discover consistent collaboration patterns across similar states. Furthermore, a count-based intrinsic reward is incorporated to encourage exploration and improve the coverage of potentially optimal joint actions. Experiments on challenging multi-agent benchmarks demonstrate that GPVD achieves faster convergence, stronger coordination, and greater training stability compared to state-of-the-art value decomposition methods.

Despite the success of Multi-Agent Reinforcement Learning (MARL) algorithms in cooperative tasks, previous works, unfortunately, face challenges in heterogeneous scenarios since they simply disable parameter sharing for agent specialization. Sequential updating scheme was thus proposed, naturally diversifying agents by encouraging agents to learn from preceding ones. However, the exploration strategy in sequential scheme has not been investigated. Benefiting from updating one-by-one, agents have the access to the information from preceding agents. Thus, in this work, we propose to exploit the preceding information to enhance exploration and heterogeneity sequentially. We present Multi-Agent Divergence Policy Optimization (MADPO), equipped with mutual policy divergence maximization framework. We quantify the policy discrepancies between episodes to enhance exploration and between agents to heterogenize agents, termed intra-agent and inter-agent policy divergence. To address the issue that traditional divergence measurements lack stability and directionality, we propose to employ the conditional Cauchy-Schwarz divergence to provide entropy-guided exploration incentives. Extensive experiments show that the proposed method outperforms state-of-the-art sequential updating approaches in two challenging multi-agent tasks with various heterogeneous scenarios. Source code is available at \url{https://github.com/hwdou6677/MADPO}.

Abstract Corona Virus disease 2019 (COVID‐19) has shattered people's daily lives and is spreading rapidly across the globe. Existing non‐pharmaceutical intervention solutions often require timely and precise selection of small areas of people for containment or even isolation. Although such containment has been successful in stopping or mitigating the spread of COVID‐19 in some countries, it has been criticized as inefficient or ineffective, because of the time‐delayed and sophisticated nature of the statistics on determining cases. To address these concerns, we propose a GSA‐ELM model based on a gravitational search algorithm to forecast the global number of active cases of COVID‐19. The model employs the gravitational search algorithm, which utilises the gravitational law between two particles to guide the motion of each particle to optimise the search for the global optimal solution, and utilises an extreme learning machine to address the effects of nonlinearity in the number of active cases. Extensive experiments are conducted on the statistical COVID‐19 dataset from Johns Hopkins University, the MAPE of the authors’ model is 7.79%, which corroborates the superiority of the model to state‐of‐the‐art methods.

Extensive traffic flow data are received from the loop detector networks every second, which requires us to develop an effective and efficient algorithm to predict future traffic flow. However, dynamic traffic conditions on a road are not just influenced by sequential patterns in the temporal dimension, but also by other roadways in the spatial dimension. Although many successful models have been developed in previous studies to forecast future traffic flows, most of them have shortcomings in modeling spatial and temporal dependencies. In this article, we focus on spatial-temporal factors and propose a new adaptive spatial-temporal transformer graph network (ASTTGN) to improve the accuracy of traffic forecasting by jointly modeling the spatial-temporal information of road networks. Specifically, we propose an adaptive spatial-temporal transformer module, which contains two developed adaptive transformer modules for capturing dynamic spatial dependence and temporal dependence across multiple time steps, respectively. Finally, feature fusion is performed through a gated feature aggregation layer to simulate the effect of complex spatial-temporal factors on traffic conditions. In particular, the multihead attention mechanism employed by the transformer can effectively explore the potential spatial-temporal dependence patterns in different subspaces. Experimental results on two real-world traffic data sets demonstrate the superiority of the proposed model compared to existing techniques.