In multi-robot coverage control, ground robots aim to cover and monitor a domain optimally. However, when covering an extensive domain like a densely forested potential fire site, the sensing capabilities of ground robots are limited, resulting in poor coverage. Leveraging the aerial robots’ ability to expand sensing ranges through high-altitude flight, this paper proposes a fully distributed, air-ground coverage control scheme to address this challenge. First, aerial robots provide a low-resolution coverage of the domain. Then, they use coarse but broad sensing information to guide ground robots, with short-range but high-resolution sensing, to achieve a high-resolution coverage. Simultaneously, each aerial robot dynamically adjusts its cell size to match its load, enhancing the coverage performance. The convergence of the control scheme is proved and its performance is evaluated through simulations and experiments.

As an effective feature extractor, Vision Transformer (ViT) has been widely applied to both image classification and object tracking tasks. In this paper, we revisit and enhance the classic Data-efficient image Transformer (DeiT) for these two tasks. The DeiT is optimized step-by-step across different modules, including its patch stem, position embedding, and the development of efficient linear attention mechanisms. To address the performance degradation of linear attention, we propose Shift Expansion Linear Attention (SELA) which generates new heads with rich feature diversity through a simple but efficient cyclic shift operation. Additionally, SELA similarity minimization is added to cross-entropy loss to further enhance feature diversity. Based on these improvements, we develop SELA-ViT for image classification and further build SELA-Track for object tracking. With comparable model size and speed, SELA-ViT-T achieves a +4.8% improvement in Top-1 accuracy over DeiT-T on ImageNet-1K and establishes a new state-of-the-art performance among linear attention methods. Furthermore, we validate SELA-ViT on five small datasets. On four benchmark object tracking datasets, SELA-Track exhibits improved tracking performance. The code and models are available at: https://github.com/saizhou777/SELA-ViT.

Traffic signal control (TSC) is still one of the most challenging and promising research issues in the field of transportation. Since traditional methods have difficulty in handling dynamically changing traffic flows, reinforcement learning (RL) methods have been introduced into TSC. However, the cost of practical application is critically high due to multiple sampling trials and long learning process. The Transformer architecture has recently attained remarkable results in natural language processing (NLP), but when applied to the field of RL, the standard Transformer architecture is difficult to optimize and faces the problem of hyperparameter sensitivity. In the paper, we transform TSC into a sequence modeling issue and propose a new evolution Transformer architecture to adjust the autoregressive model through reward, past states and actions in the traffic environment to directly generate the best predicted action. In addition, we use the feature evolution module (FEM) instead of residual connections to make the learning process more stable and efficient. Through experiments on public datasets, we demonstrate that our ETLight model achieves a state-of-the-art (SOTA): 1) It achieves the overall best performance on average travel time (ATT) metric, with improvements of up to 6.85%, 3.73% and 3.10% over the best conventional, RL and Transformer methods, respectively; 2) It has a more stable learning process, faster learning speed and better convergence compared to published TSC methods so far; and; 3) it has good robustness and is less sensitive to hyperparameter selection.

Multi-agent adversarial tasks such as swarm robotics and autonomous vehicle coordination, demand efficient decentralized collaboration under partial observability. While model-free multi-agent RL (MF-MARL) methods suffer from necessitating extensive environment interactions, most existing multi-agent model-based RL (MA-MBRL) methods fail to align with the Centralized Training with Decentralized Execution (CTDE) paradigm, which limits system flexibility. This paper proposes a novel modular architecture with refined observations (MARO) to achieve the CTDE paradigm by decoupling agents from the world model. Key innovations include: 1) an enhanced world model with weighted loss and history-augmented rollout for high-quality data generation; 2) a dual-stream semantic decomposition network (DSDN) that performs fine-grained decomposition of observations to refine action mapping and mitigate performance degradation from information loss. Extensive experiments on the StarCraft Multi-Agent Challenge (SMAC) demonstrate superior performance over opponents, validating the effectiveness and advancement of MARO.

This letter is concerned with a three-dimensional target motion analysis issue using azimuth and elevation measurements. The nonlinear relationship between these measurements and target dynamics often poses challenges for conventional methods, especially in high-noise environments. To address this challenge, a novel multi-agent deep reinforcement learning (MADRL)-based estimator is proposed for target motion parameter estimation. Specifically, by modeling each component of the target motion parameter as an individual agent, the target motion parameter estimation process is framed as a cooperative Markov game. An MADRL framework is then introduced to solve this problem. Simulation results demonstrate that the proposed algorithm achieves higher estimation accuracy than existing estimators.

Information-based autonomous robot exploration methods, aiming to maximize the exploration rewards, e.g., mutual information (MI), get more prevalent in field robotics applications. However, most MI-based exploration methods assume known poses or use inaccurate pose uncertainty approximation, which may lead to deviation or even failure when exploring prior unknown environments. In this paper, we explicitly consider full-state (pose & map) uncertainty for balancing exploration and localizability, i.e., avoiding the robot guiding itself to complex scenes with high exploration rewards but hard to localize. We first propose a Rao-Blackwellized particle filter-based localization and mapping framework (RBPF-CLAM) for a dense environmental map with continuous occupancy distribution. Then we develop a new closed-form particle weighting method to improve the localization accuracy and robustness. We further use these weighted particles to approximate the unknown pose uncertainty and combine it with our previous confidence-rich mutual information (CRMI) metric to evaluate the expected information utility of the robot’s new control actions. This new information metric is calleduncertainCRMI (UCRMI). Dataset experiments show our RBPF-CLAM improves about 44.7% average root mean square error than the state-of-the-art RBPF localization method, and real-world experimental results show that our UCRMI reduces the pose uncertainty about 32.85% more than CRMI and 25.36% time cost than UGPVR in the exploration of unknown and unstructured scenes given sparse measurements, which shows better performance than other state-of-the-art information metrics.Note to Practitioners—This work was motivated by the problem of ‘planning for state estimation’ for a range-sensing robot, i.e., the robot can choose a better future place to facilitate its localization more accurately and explore new areas rationally to gather more information. Existing methods mainly assume the robot’s poses during the exploration can be estimated by an independent localization approach or simply propagated via a predefined probabilistic distribution. However, localization failure would lead to higher planning deviation for the planner that does not consider the pose uncertainty, and manually set parametric distribution is more prone to overestimate the pose uncertainty. This paper proposes an RBPF-based localization and mapping scheme and an improved particle weight update method in a confidence-rich map, then uses the weighted particles to approximate trajectory entropy and combines it with CRMI to evaluate the expected information gain of a candidate action/node. Our newly defined information function ‘UCRMI’ can prevent the robot from exploring too aggressively without considering its localizability in prior unknown and unstructured environments. These scenes may lack robust features to conduct feature-based SLAM or lack accurate external localization information such as GPS. This method can be applied in underwater, planetary, and subterranean robot exploration tasks, even using low-resolution sensors. Future work mainly involves adapting UCRMI to applications in large-scale scenes using small autonomous platforms.

To handle the non-stationarity of the environment and the curse of dimensionality issues in multi-agent reinforcement learning, gathering information through communication is a critical part. Existing frameworks have proposed centralized or distributed structures to deal with the problem. However, they either have problems of robustness or problems of high communication costs. This paper adopts a hierarchical zeroth-order policy optimization (HZOPO) algorithm for cooperative multi-agent reinforcement learning (MARL) problems. The agents are divided into different groups with high-and low-level entities. A hierarchical communication structure is implemented to reach global consensus. It is shown that the HZOPO algorithm can balance both convergence and communication efficiency in cooperative MARL environments. The convergence of the algorithm is also proved.

This paper investigates the problem of distributed target tracking via underwater acoustic sensor networks (UASNs) with fading channels. The degradation of signal quality due to wireless channel fading can significantly impact network reliability and subsequently reduce the tracking accuracy. To address this issue, we propose a modified distributed unscented Kalman filter (DUKF) named DUKF-Fc, which takes into account the effects of measurement fluctuation and transmission failure induced by channel fading. The channel estimation error is also considered when designing the estimator and a sufficient condition is established to ensure the stochastic boundedness of the estimation error. The proposed filtering scheme is versatile and possesses wide applicability to numerous scenarios, e.g., tracking a maneuvering underwater target with underwater sensor nodes (USNs) equipped with acoustic sensors. Considering the constraints of network energy resources, the issue of investigating the energy cost of DUKF-Fc is discussed in the simulation and accordingly, the results demonstrate the robustness and energy-efficiency of the proposed filtering procedure.

The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the nonconvex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov decision process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with the convex optimization-based method, the convex relaxation is not required; 2) compared with the least squares method, the proposed estimators are capable of converging to a global optimal state; and 3) compared with the discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Currently, decarbonization has become an emerging trend in the power system arena. However, the increasing number of photovoltaic units distributed into a distribution network may result in voltage issues, providing challenges for voltage regulation across a large-scale power grid network. Reinforcement learning based intelligent control of smart inverters and other smart building energy management (EM) systems can be leveraged to alleviate these issues. To achieve the best EM strategy for building microgrids in a power system, this paper presents two large-scale multi-agent strategy evaluation methods to preserve building occupants’ comfort while pursuing system-level objectives. The EM problem is formulated as a general-sum game to optimize the benefits at both the system and building levels. The α -rank algorithm can solve the general-sum game and guarantee the ranking theoretically, but it is limited by the interaction complexity and hardly applies to the practical power system. A new evaluation algorithm (TcEval) is proposed by practically scaling the α -rank algorithm through a tensor complement to reduce the interaction complexity. Then, considering the noise prevalent in practice, a noise processing model with domain knowledge is built to calculate the strategy payoffs, and thus the TcEval-AS algorithm is proposed when noise exists. Both evaluation algorithms developed in this paper greatly reduce the interaction complexity compared with existing approaches, including ResponseGraphUCB (RG-UCB) and α InformationGain ( α -IG). Finally, the effectiveness of the proposed algorithms is verified in the EM case with realistic data.

Marine target searching is a complex task due to large search areas, unique signal propagation characteristics, and limited visibility, posing significant challenges for single-agent or homogeneous multi-agent systems. In response, we propose a novel hierarchical heterogeneous multi-agent (HHMA) framework designed for underwater search scenarios. This framework integrates three types of vehicles moving in different domains—unmanned aerial, surface, and underwater vehicles, effectively overcoming the limitations of single or double-agent configurations. We begin by elucidating the advantages of the HHMA system in target searching, providing the kinematic modeling, while also transforming sonar detecting data and defining the search problem. The mission is decomposed to three human-comprehensible subtasks that are adaptive to both environmental conditions and equipment capabilities: moving, target estimating and trajectory planning. The target estimating subtask is effectively modeled as a Markov Decision Process, retaining its memory capability. Additionally, we extend multi-agent reinforcement learning to multi-policy reinforcement learning, facilitating the training of interdependent policies. The efficacy of our approach is demonstrated through simulations, comparing it with rule-based methods. Simulation results underscore the significance of the HHMA system and validate the proposed training methodology.

Under the purpose of achieving the optimal voltage control strategy in power grid system, multi-agent evaluation algorithms like a-rank are widely used. However, in large-scale systems with massive agents and strategies, these methods are not time feasible. Therefore, a two-stage population-based multi-agent evaluation algorithm is proposed to solve voltage control problem in large-scale power grid systems. For stage one, a population is first established for each agent. And then, individuals in the populations randomly combined to form joint strategies. Base on the max and mean reward from the interaction between joint strategies and the environment, populations evolve to a near-optimal joint strategy. Stage two takes the above near-optimal joint strategy as the starting point, and uses a strategy search algorithm with maximum transfer possibility to find the Markov-Conley chain in the system. Finally, the above two-stage method is simulated in 10 and 32-agent power grid systems to verify the effectiveness.

Passive detection can work for a long time with low energy consumption in underwater surveillance. However, tracking unknown noncooperative targets with only direction angles is challenging, and the tracking performance of multiple targets is poor. Based on several passive sensors in the underwater sensor network (UWSN), a feature-aided state estimation method is used to start tracking unknown targets. The feature-aided joint probabilistic data association combined with the particle filter method is also proposed to improve the passive tracking performance of multiple targets. The track management and the fusion strategy are given to remove fake tracks and obtain correct trajectories of unknown targets. The simulation results show that the feature-aided method can quickly start and effectively track multiple noncooperative targets with passive sensors. Compared with other methods, the proposed method can track targets more accurately with the advantages of low energy consumption and less exposure in various environments.

This paper mainly studies the localization and mapping of range sensing robots in the confidence-rich map (CRM) and then extends it to provide a full state estimate for information-theoretic exploration. Most previous works about active simultaneous localization and mapping and exploration always assumed the known robot poses or utilized inaccurate information metrics to approximate pose uncertainty, resulting in imbalanced exploration performance and efficiency in the unknown environment. This inspires us to extend the confidence-rich mutual information (CRMI) with measurable pose uncertainty. Specifically, we propose a Rao- Blackwellized particle filter-based localization and mapping scheme (RBPF -CLAM) for CRM, then we develop a new closed-form weighting method to improve the localization accuracy without scan matching. We further derive the uncertain CRMI (UCRMI) with the weighted particles by a more accurate approximation. Simulations and experimental evaluations show the localization accuracy and exploration performance of the proposed methods.

This article investigates the underwater node dynamic localization and prediction problems in a dynamic sensor network. Node localization in the Internet of underwater things is the basis of target tracking and ocean monitoring. At present, most of the node location algorithms assume calm sea and fixed node location. However, the current velocity is uncertain in space and time. The nodes are drifted with the current motion. Therefore, most of the localization algorithms lose efficacy in the actual marine environment. In order to solve the above problems, a node dynamic prediction algorithm is proposed. First, the node mobility model is improved, which is more suitable for the actual marine environment. Second, a frequency-based anchor node prediction algorithm is designed to improve anchor node location accuracy. Third, when the ordinary node receives the signals sent by anchor nodes of different depths, a deep information-based weighted fusion method is designed for the ordinary node localization in order to mine more information in each direction. Finally, location and prediction simulation in sensor networks is carried out. The results show that the proposed node localization and prediction algorithm is more accurate than SMLP and high-precision localization with mobility prediction algorithms and prove the enhanced effect of our method in dynamic marine.

This paper describes a route planner that enables an autonomous underwater vehicle to selectively complete part of the predetermined tasks in the operating ocean area when the local path cost is stochastic. The problem is formulated as a variant of the orienteering problem. Based on the genetic algorithm (GA), we propose the greedy strategy based GA (GGA) which includes a novel rebirth operator that maps infeasible individuals into the feasible solution space during evolution to improve the efficiency of the optimization, and use a differential evolution planner for providing the deterministic local path cost. The uncertainty of the local path cost comes from unpredictable obstacles, measurement error, and trajectory tracking error. To improve the robustness of the planner in an uncertain environment, a sampling strategy for path evaluation is designed, and the cost of a certain route is obtained by multiple sampling from the probability density functions of local paths. Monte Carlo simulations are used to verify the superiority and effectiveness of the planner. The promising simulation results show that the proposed GGA outperforms its counterparts by 4.7%–24.6% in terms of total profit, and the sampling-based GGA route planner (S-GGARP) improves the average profit by 5.5% compared to the GGA route planner (GGARP).

This article investigates the cooperative output regulation problem for discrete-time heterogeneous multiagent Markov jump systems. Two cases are studied: 1) output regulation quadratic control in the case where the exosystem is accessible to all agents and 2) cooperative output regulation quadratic control in the case where only a part of agents can directly communicate with the exosystem. The hidden Markov models are employed to describe the asynchronous modes of the agents and their corresponding controllers. Via the jumping regulator equation, asynchronous control laws are constructed and the algorithms to obtain control parameters are presented in terms of linear matrix inequalities. For the first case, the optimal synchronous/mode-dependent control law, which is a special case of the asynchronous control protocol, is also given via the stochastic dynamic programming approach. Finally, an example is given to illustrate the effectiveness of the proposed approaches.

This paper investigates the task coordination of multi-robot where each robot has a private individual temporal logic task specification; and also has to jointly satisfy a globally given collaborative temporal logic task specification. To efficiently generate feasible and optimized task execution plans for the robots, we propose a hierarchical multi-robot temporal task planning framework, in which a central server allocates the collaborative tasks to the robots, and then individual robots can independently synthesize their task execution plans in a decentralized manner. Furthermore, we propose an execution plan adjusting mechanism that allows the robots to iteratively modify their execution plans via privacy-preserved inter-agent communication, to improve the expected actual execution performance by reducing waiting time in collaborations for the robots. The correctness and efficiency of the proposed method are analyzed and also verified by extensive simulation experiments.

This paper presents a hierarchical structure to solve the consensus problem of multiagent systems. The new scheme divides the agents into several groups, with each group containing a value concerning all of the intragroup agents' states, which we call group information. For each single agent, it receives not only the agent information from its intragroup neighbors, but also the group information from its neighboring groups. It is then shown that global consensus can be achieved under the proposed scheme in both discrete time and continuous time. Moreover, a sufficient condition to achieve average consensus is provided. This hierarchical model can be well used in the PageRank algorithm to reduce the communication loads, and to reveal the attractors for Boolean networks by reducing the computational complexity.

We study the distributed Kalman filtering problem in relative sensing networks with rigorous analysis. The relative sensing network is modeled by an undirected graph while nodes in this network are running homogeneous dynamical models. The sufficient and necessary condition for the observability of the whole system is given with detailed proof. By local information and measurement communication, we design a novel distributed suboptimal estimator based on the Kalman filtering technique for comparison with a centralized optimal estimator. We present sufficient conditions for its convergence with respect to the topology of the network and the numerical solutions of n linear matrix inequality (LMI) equations combining system parameters. Finally, we perform several numerical simulations to verify the effectiveness of the given algorithms.

This paper investigates two formation control problems for a leader-follower network in 3-D. One is called the formation marching control problem, the objective of which is to steer the agents to maintain a target formation shape while moving with the synchronized velocity. The other one is called the formation rotating control problem, whose goal is to drive the agents to rotate around a common axis with a target formation. For the above two problems, we consider directed and switching sensing topologies while the communication is assumed to be bidirectional and switching. We develop approaches utilizing barycentric coordinates toward these two problems. Local control laws and graphical conditions are acquired to ensure global convergence in both scenarios.

We consider the control of formations of a leader-follower network, where the objective is to steer a team of multiple mobile agents into a formation of variable size. We assume that the shape description of the formation is known to all the agents, which is captured by a complex-valued Laplacian associated with the sensing graph, but the size scaling of the formation is not known or only known to two agents, called the leaders in the network. A distributed linear control strategy is developed in this paper such that the agents converge to the desired formation shape, for which the size of the formation is determined by the two leaders. Moreover, in order to make all agents in a formation move with a common velocity, the distributed control law also incorporates a velocity consensus component, which is implemented with the help of a communication network that may, in general, be of different topology from the sensing graph. Both the setup of single-integrator kinematics and the one of double-integrator dynamics are addressed in the same framework except that the acceleration control in the double-integrator setup has an extra damping term.

This article investigates the controllability problem of multi-agent systems. Each agent is assumed to be governed by a second-order consensus control law corresponding to a directed and weighted graph. Two types of topology are considered. The first is concerned with directed trees, which represent the class of topology with minimum information exchange among all controllable topologies. A very simple necessary and sufficient condition regarding the weighting scheme is obtained for the controllability of double integrator multi-agent systems in this scenario. The second is concerned with a more general graph that can be reduced to a directed tree by contracting a cluster of nodes to a component. A similar necessary and sufficient condition is derived. Finally, several illustrative examples are provided to demonstrate the theoretical analysis results.

This paper investigates the problem of steering a nonholonomic mobile robot to achieve a circular motion around a target. We propose control schemes that require only bearing measurements and deal with two types of targets: point target and disk target. Circumnavigation schemes are developed to achieve efficient encirclement of the target. We show that using the proposed control schemes, the robot can circle the target from a prescribed radius without distance measurement and avoid collision with disk target as well. The validity of the proposed control schemes is supported by experiments on an e-puck robot.