Finding the Nash equilibrium in large-scale zero-sum games has long been a challenging problem due to the vast and unknown policy space and the utility matrix. The Policy-Space Response Oracles (PSRO) framework, as a combination of conventional game analysis with deep reinforcement learning, iteratively constructs a restricted game by including a pure policy that is the best response to the equilibrium of the restricted game in the previous iteration. However, adding a pure policy at a time is inefficient, as there may be multiple policies that dominate the equilibrium of the current restricted game. In this regard, we propose Mixed Policy-Space Response Oracles (M-PSRO), which add a mixed policy rather than a pure policy to the current policy set. To obtain more effective mixed policy candidates, we adopt a parallelized training framework to promote policy diversity. Theoretical analysis shows that M-PSRO can converge to an approximate Nash equilibrium. We conduct extensive experiments across a wide range of complex games, and results show that the M-PSRO algorithm achieves state-of-the-art performance across all the games and can approach the Nash equilibrium more efficiently. Numerous ablation studies show that the improvement benefits from the usage of mixed policies in M-PSRO, which offer an even more flexible optimization space.

Multi-agent scheduling problems are common in applications such as intelligent transportation and smart manufacturing. When the agents are non-cooperative and inertially constrained, finding a safe and efficient policy under the trajectory uncertainty of other agents is a non-trivial problem. In this article, we establish a minimax framework to optimize the worst-case scheduling performance under the two-agent model. Specifically, a unified representation is proposed to characterize the trajectory uncertainty of the other agent, and a function is derived to evaluate different target states. Based on this evaluation, we further develop a control policy by adopting the minimax method, where a trajectory leading to the most robust target state is generated at each step. Algorithms are also provided to ensure the computational tractability of the policy. Furthermore, the safety of the policy is proved, and the global robustness is verified by numerical simulations, which show that the proposed policy reduces the worst-case scheduling cost by 13.1% compared with heuristic policies.

Scheduling of physical space in multi-agent systems is crucial in widespread applications including transportation and industrial manufacturing. However, few existing works focused on improving the scheduling efficiency when agents are inertially constrained and some non-cooperative agents with unknown and uncontrollable trajectories exist. In this article, we establish a minimax framework aiming to ensure the robustness of scheduling against the uncertainty of non-cooperative agents. Specifically, we propose a function characterizing the preference of different states based on a given situation information, and formulate a trajectory planning policy by establishing a minimax optimization problem. Furthermore, the tractability of the proposed policy is ensured by developing an approximate algorithm and a truncation method, and the safety guarantee of the policy is also proved. Finally, numerical simulations suggest a 90% reduction on the empirical probability of high-cost scenarios compared with heuristic policies, validating the robustness of the proposed policy.

Intelligent multi-agent systems are prospective in various applications such as smart cities and smart factories, while the efficiency of scheduling problems of the agents often suffers from the kinematic constraints. In this paper, we establish the performance limit in the distributed scheduling of two agents with kinematic constraints. Specifically, the minimum achievable loss under full information is first analyzed, and then the robust loss is derived according to the available information. The robust analysis provides key insights on how to evaluate a situation, how to quantify the value of information for the efficient control, and how to characterize the benefit of cooperation between agents. Furthermore, a robust control policy is proposed based on the analysis, and simulation results are presented validating the efficiency of the policy. The robust analysis proposed in this paper provides a new perspective for handling the scheduling problems in intelligent multi-agent systems both theoretically and algorithmically.

Locations of mobile agents are often requisite information for wireless applications such as sensor networks and Internet of Things (IoT). As the network size increases, verifying the localizability of all nodes in a network quickly becomes intractable. In this article, we turn to analyzing the unbounded localizability of infinite stochastic networks under sequential localization methods. Specifically, we prove the existence of the phase transition on the probability of localizing an unbounded subnetwork from a bounded initial anchor set in Poisson point process networks. The phase transition occurs when the node intensity of the network reaches a critical intensity, which is determined by the adopted sequential localization method. Furthermore, we develop a simulation method to obtain tight upper and lower bounds of the critical intensity for two-dimensional (2-D) networks with high confidence, and provide the numerical bounds under several typical sequential localization methods. We also show by simulation that the percentage of localizable nodes increases rapidly near the critical intensity, which provides guidelines for network design and deployment.

Intersection vehicle scheduling is a core issue influencing urban traffic fluency, and the development of the intelligent transportation system (ITS) provides novel potential for developing more appealing scheduling frameworks. Under most current approaches, the speed of vehicles passing the intersection is lower than the normal speed, which reduces the time-efficiency of the intersection. In this paper, we propose a distributed framework of long-horizon vehicle scheduling to eliminate potential conflicts among vehicles before they approach the intersection. Specifically, an algorithm for vehicles to select communication links and adjust their own positions based on the obtained delayed information is presented. The convergence guarantee of the algorithm is also provided for any finite vehicle flow. Furthermore, the influence of parameter selections on the algorithm performance is illustrated by numerical simulations, which also verify that the proposed algorithm better maintains traffic fluency than the heuristic approach. The proposed framework and algorithm provide a novel solution to improve traffic efficiency for intersections with enough preparation distances.

Due to the abundant marine resources, smart ocean has attracted much attention of the government, industry, and academy. The Internet-of-Things (IoT) architectures for smart ocean have been proposed to collect various of data from the ocean, thereby assisting environmental protection, military reconnaissance, and so on. However, few researchers have paid attention to the security and privacy issues of data collection and transmission. In this article, for the unreliable underwater environment, we present a secure, efficient, and complete data collection, and transmission and storage scheme for IoT in smart ocean. Especially, to prolong the lifetime of the underwater node, two novel data compression algorithms [lossy data compression algorithm (LCA) and lossless data compression algorithm (NLCA)] are also proposed. Moreover, due to the vulnerability of underwater nodes, we also propose a corresponding IoT framework and data collection pattern to resist the single point failure attack. Besides, to guarantee the confidentiality, reliability, and integrity of transmitting data, Elliptic Curve-ElGamal (EC-ElGamal) and elliptic curve digital signature algorithm (ECDSA) are employed. The consensus algorithm and blacklisting mechanism are also employed to detect and address failure or malicious nodes. Finally, the security analysis demonstrates that our scheme is able to resist many typical attacks for underwater nodes, such as manipulation attacks, Distributed Denial-of-Service (DDoS) attacks, malicious node injection attacks, and so on. Additionally, relevant experimental results show that the scheme is feasibility and efficiency.

The ability to localize a majority of nodes is important in many scenarios concerning large-scale stochastic networks. Classical percolation, i.e., the existence of infinite connected components in a stochastic network, has been analyzed in previous studies. However, a single connection is not enough for localization, which requires at least two connections to anchors. In this paper, we generalize classical percolation to localization problems. We find that for distributed localization, phase transition appears around a critical network density, i.e., an infinite localizable subnetwork exists with probability zero in the sub- critical case and with probability one in the super-critical case. Further, simulation methods are developed to estimate upper and lower bounds of critical densities, and precise estimates of densities in angle-of-arrival (AOA) and time-of- arrival (TOA) localization are performed using proposed methods.