Embodied intelligence and related disciplines have identified several mechanisms that help embodied agents learn how to solve complex problems. Reinforcement learning (RL) is one of the most promising computational approaches toward enhancement of the learning-based problem-solving abilities of such agents. Given the recent rapid evolution of artificial intelligence, RL has become a keystone technology, accelerating scientific discoveries and also finding applications in many other domains. In RL, an agent collects data when interacting with the environment, which optimizes a policy ensuring a higher return. Further improvement requires more exploration of the action space. However, not all actions in that space are safe and acceptable. The exploration of an agent must be constrained. In this work, a novel mirror descent safe policy optimization (MDSPO) algorithm is proposed to ensure the safety of an RL agent. The algorithm leverages mirror descent optimization to maximize the return while satisfying the safety constraint. A novel optimization objective is formulated, and an innovative three-stage optimization strategy is employed-comprising gradient descent without the cost constraint, projection onto the nonparametric policy space with the cost constraint, and projection onto the parametric policy space. Compared to previous methods, MDSPO is a simple and easy to implement first-order approach, which does not impose a hard constraint on the trust region. Theoretical analysis of the MDSPO reveals a lower bound on return improvement and an upper bound on constraint violation at the time of each policy update. The numerical results obtained from two sets of different constrained locomotive experiments demonstrate that MDSPO improves the average return by about 12% and better satisfies the cost constraints than other state-of-the-art methods do. In a real-world obstacle avoidance experiment using an unmanned surface vessel, MDSPO both finds the optimal path and guarantees agent safety.

In target tracking with sensor network under decentralized architecture, the sensor network resources including sensing and communication resources are always constrained, meanwhile, the quality of measurements acquired by different sensor nodes toward the same target usually differs from each other. Both the selected sensor nodes and communication topology among them significantly affect not only the tracking accuracy but also network resource consumption. Thus, to improve resource utilization efficiency, this paper proposes a sensor network resource management method for target tracking under decentralized architecture, focusing on the selection of sensor nodes acquiring high-quality measurements and the optimization of their communication topology. First, this paper derives an iteration-adaptive decentralized PCRLB(IA-DPCRLB) as a tracking accuracy metric, and a tracking accuracy constraint function is established based on the IA-DPCRLB. Additionally, an objective function is constructed by modeling both the sensor scheduling cost and the communication cost. Finally, to solve the non-convex optimization problem, an enumeration-based genetic algorithm is employed. Simulation results demonstrate that the proposed algorithm can adaptively select sensor nodes and dynamically optimize the communication topology according to the target’s motion state, achieving the minimized system resource consumption while ensuring the predefined tracking accuracy, thereby significantly improving resource utilization efficiency.

In complex building environments, centralized multi-sensor information fusion for target automatic tracking typically faces two critical challenges: measurement origin ambiguity due to the presence of multi-target and clutter interference, as well as multiple multi-step-lag out-of-sequence measurements (MM-OOSMs) caused by random communication delay and non-full-rate transmission. To address these issues for improved target automatic tracking, this paper proposes the Augmented State Multi-Sensor Integrated Probabilistic Data Association (AS-MSIPDA) method. To resolve measurement origin ambiguity, the joint association events that pairing measurements from the same origins are enumerated, meanwhile, the hybrid states across multiple consecutive scans are augmented to make the measurements in MM-OOSMs originating from the same target are statistically independent. Numerical results demonstrate the proposed AS-MSIPDA is capable of dynamically estimating both the number and states of targets as accurate as the optimal benchmark, but with dramatically reduced computational complexity. In addition, the proposed method enables to output smoothed estimate as significant side-benefit for off-line analysis.

This paper considers the distributed consensus fusion for industrial multi-target tracking in clutter using sensor network with complementary field-of-view (FOV). Due to complementary FOV, the local state estimation accuracies of different nodes towards the same target significantly differs from each other, and designing appropriate consensus weight to fuse those accuracy-discrepancy local estimates communicated from neighbor nodes becomes critically important. Additionally, the track origin ambiguity caused by presence of multi-target and clutter disturbance also poses significant challenges to the distributed consensus fusion system. Motivated by addressing aforementioned problems, we propose a Probability-Weighted Multi-Step Consensus Distributed Fusion (PWMC-DFA) Algorithm. The proposed PWMC-DFA adaptively calculates the consensus weight based on node's confidence which represents its state estimation accuracy, and leverages the track-to-track association and merging to pair multisensory tracks following the same target. The simulation results show that proposed algorithm delivers more accurate state estimation and less disagreement among different nodes, compared with state-of-the-art approach.

This paper investigates the multi-sensor distributed fusion to estimate the state of target involved in industrial agriculture under heavily-constrained communication. To fully address problems of both random communication delay and limitted transmission rate, we derive a multi-sensor distributed tracklet fusion based on augment state (AS-DTF) method. The derived AS-DTF leverages the augmented equivalent measurement to remove correlations of state estimation errors across different sensor at multiple times and is able to fuse multiple multi-step out-of-sequence tracks (OOSTs) in an optimal manner. The simulation results validate the proposed AS-DTF achieves the same state estimation accuracy as the optimal fusion benchmark.

Bearing-only multi-target localization (BOMTL) using multiple sensors is generally required to solve the sophisticated data association problem which determines a designated sensor measurement originated from a particular target. In this paper, a novel spatial sparse representation based BOMTL method is proposed by fully utilizing a wireless array network structure. With array spatial features, the BOMTL problem can be formulated as a binary sparse vector recovery problem using the converted “pseudo-measurements” in frequency domain. The proposed method transforms the source location estimation problem into a spatial sparse representation (SSR) framework, which avoids dealing with the conventional data association. With orthogonal matching pursuit (OMP) exploiting the binary property of the sparse vector to be estimated, we develop a BOMTL-OMP algorithm to reconstruct the sparse vector. The numerical simulations demonstrate the performance of the proposed method.