In swarm robotics, confrontation scenarios, including strategic confrontations, require efficient decision– making that integrates discrete commands and continuous actions. Traditional task and motion planning methods separate decision–making into two layers, but their unidirectional structure fails to capture the interdependence between these layers, limiting adaptability in dynamic environments. Here, we propose a novel bidirectional approach based on hierarchical reinforcement learning, enabling dynamic interaction between the layers. This method effectively maps commands to task allocation and actions to path planning, while leveraging cross– training techniques to enhance learning across the hierarchical framework. Furthermore, we introduce a trajectory prediction model that bridges abstract task representations with actionable planning goals. In our experiments, it achieves over 80% in confrontation win rate and under 0.01 seconds in decision time, outperforming existing approaches. Demonstrations through large–scale tests and real–world robot experiments further emphasize the generalization capabilities and practical applicability of our method.

In swarm robotics, confrontation including the pursuit-evasion game is a key scenario. High uncertainty caused by unknown opponents’ strategies, dynamic obstacles, and insufficient training complicates the action space into a hybrid decision process. Although the deep reinforcement learning method is significant for swarm confrontation since it can handle various sizes, as an end-to–end implementation, it cannot deal with the hybrid process. Here, we propose a novel hierarchical reinforcement learning approach consisting of a target allocation layer, a path planning layer, and the underlying dynamic interaction mechanism between the two layers, which indicates the quantified uncertainty. It decouples the hybrid process into discrete allocation and continuous planning layers, with a probabilistic ensemble model to quantify the uncertainty and regulate the interaction frequency adaptively. Furthermore, to overcome the unstable training process introduced by the two layers, we design an integration training method including pre-training and cross-training, which enhances the training efficiency and stability. Experiment results in both comparison, ablation, and real-robot studies validate the effectiveness and generalization performance of our proposed approach. In our defined experiments with twenty to forty agents, the win rate of the proposed method reaches around ninety percent, outperforming other traditional methods. Note to Practitioners—With artificial intelligence rapidly developing, robots will play a significant role in the future. Especially, the swarm formed by many robots holds promising potential in civil and military applications. Promoting the swarm into games or battles is rather riveting. The reinforcement learning method provides a plausible solution to realize the battle of robotic swarms. There are still some issues that need to be addressed. On one hand, we focus on the uncertainty caused by the battlefield nature and the environment which limits our ability for the implementation of swarms. On the other hand, we solve the problem that the decision process combined with commands and actions is a hybrid system, which cannot be directly reflected in the confrontation of swarms. Overall, our approaches throw light on artificial general intelligence and also reveal a solution to interpretable intelligence.

In mobile robotics, effectively covering known terrains is essential. While grid-based methods surpass exact cell decomposition in path length and multi-robot scalability, they face challenges in irregular areas. Here we develop a model for shortening coverage paths in arbitrary environments using grid-based methods, which redefines the path optimization problem as finding the largest Hamiltonian sub-graph of a given grid graph. Additionally, we present a Hamiltonian cycle expansion strategy to simplify the resolution process and propose a low-repetitive coverage path planner based on the strategy. Our path planner enables the quick finding of an efficient full coverage path in any region. Simulation results show that our algorithm consistently produces efficient coverage paths across diverse settings and demonstrates its adaptability in multi-robot systems.

Using reinforcement learning (RL) algorithm to optimize guidance law can address non-idealities in complex environment. However, the optimization is difficult due to huge state-action space, unstable training, and high requirements on expertise. In this paper, the constrained guidance policy of a neural guidance system is optimized using improved RL algorithm, which is motivated by the idea of traditional model-based guidance method. A novel optimization objective with minimum overload regularization is developed to restrain the guidance policy directly from generating redundant missile maneuver. Moreover, a bi-level curriculum learning is designed to facilitate the policy optimization. Experiment results show that the proposed minimum overload regularization can reduce the vertical overloads of missile significantly, and the bi-level curriculum learning can further accelerate the optimization of guidance policy.

The missile-target assignment (MTA) is a typical weapon-target assignment problem in Command and Control of modern warfare. Despite the significance of the problem, traditional algorithms still lack efficiency, solution quality, and practicability in the adversarial environment. In this article, we propose a data-driven policy optimization with deep reinforcement learning (PODRL) for the adversarial MTA. We design a comprehensive reward function to motivate the optimization of assignment policy. As such, the learned policy can implicitly model the penetration of missiles under an adversarial environment in a data-driven way. We also present a fair sample strategy to improve the sample efficiency and accelerate the policy optimization. Experimental results show that PODRL can adaptively generate satisfactory solutions in both small-scale and large-scale instances. Furthermore, we evaluate the effectiveness of PODRL in a multiobjective scenario. The result demonstrates that a well-optimized policy can achieve high-quality allocation and demand forecast of the missile resources simultaneously.

This article presents a full mathematical proof of the characteristic modeling approach for high-order linear dynamical systems. It explores the nature of the characteristic model in rigorous mathematical forms, also showing why and how the high-order dynamics can be compressed into the lower-order characteristic model. The relationships between high-order linear continuous dynamical systems, discrete-time characteristic model coefficients, and sampling-time intervals are investigated.

In this paper, characteristic model-based modeling and control approaches for complex dynamical networks based on sampled data are studied. It shows that the characteristic model, in which underline network topological structures are simplified, can provide a straightforward and implicit description for network dynamics. The induced parameter estimation method can further make the model adaptive and purely data-driven. Moreover, a control law based on this model is also proposed to govern the network dynamics. Finally, the theoretical results are verified through numerical simulations of modeling and stabilizing a dynamical network.

This paper proposes a sampled data-based modeling approach for complex network systems. A compression method, known as characteristic modeling, is used to construct microscopic models from macroscopic observations. Based on this model, a novel control method is also developed to achieve network synchronization. The proposed approach can reduce both the complexity of microscopic dynamics and overall networks. Its application to pinning control design validates the effectiveness of this approach.