Abstract Multi-agent credit assignment is a research hotspot in the field of cooperative multi-agent reinforcement learning, and its key is how to accurately measure the individual contribution of each agent in the system to promote multi-agent cooperation. Existing solutions mainly use value function factorization or intrinsic reward mechanism, each of which has its own limitations, and both of them utilize global state information, which is not consistent with the information conditions in the actual confrontation. Therefore, this paper proposes a novel value factorization method for multi-agent deep reinforcement learning, which can solve the problem of credit assignment without using global state information. Our method establishes an explicit individual contribution evaluation mechanism for each agent, which portrays the role of each agent in the system by comparing the differences of joint value functions under different information conditions, so that more important agents get more attention, so as to improve the cooperative ability of agents. Experimental results show that our method outperforms all baselines in terms of learning efficiency and stability in multiple scenarios of StarCraft II, and its performance is comparable to that of the method based on global state information in easy scenarios.

Network slicing is the most promising paradigm that enables the provision of services for aerial communication tasks, such as data traffic offloading, disaster relief, and data collection, with one shared physical network. However, current researchers primarily focus on the modeling of utilizing service function chaining (SFC) or task offloading, which have inherent limitations for areial communication tasks. While the former restricts the source-sink nodes to be fixed, the latter models network slicing as a whole. To further unleash the flexibility of network slicing, we introduce the hybrid network slicing (HNS) modeling in this paper. In our HNS, each aerial service is abstracted as multiple SFCs with dispersed source-sink nodes. These SFCs can be flexibly deployed in a unified or split manner based on the service workload and Quality of Service (QoS) requirements. Then, we formulate the HNS problem in a multi-tier network system as a mixed integer programming (MILP) problem and propose exact and community detection-based heuristic approaches to address the proposed problem. The simulation results demonstrate that the proposed heuristic approaches can effectively mitigate computational complexity than the exact method while providing near-optimal solutions for aerial communication tasks.

Abstract Value factorization is a popular method for cooperative multi-agent deep reinforcement learning. In this method, agents generally have the same ability and rely only on individual value function to select actions, which is calculated from total environment reward. It ignores the impact of individual characteristics of heterogeneous agents on actions selection, which leads to the lack of pertinence during training and the increase of difficulty in learning effective policies. In order to stimulate individual awareness of heterogeneous agents and improve their learning efficiency and stability, we propose a novel value factorization method based on Personality Characteristics, PCQMIX, which assigns personality characteristics to each agent and takes them as internal rewards to train agents. As a result, PCQMIX can generate heterogeneous agents with specific personality characteristics suitable for specific scenarios. Experiments show that PCQMIX generates agents with stable personality characteristics and outperforms all baselines in multiple scenarios of the StarCraft II micromanagement task.