With the growth of the magnitude of multiagent networks, distributed optimization holds considerable significance within complex systems. Convergence, a pivotal goal in this domain, is contingent upon the analysis of infinite products of stochastic matrices (IPSMs). In this work, the convergence properties of inhomogeneous IPSMs are investigated. The convergence rate of inhomogeneous IPSMs toward an absolute probability sequence $\pi $ is derived. We also show that the convergence rate is nearly exponential, which coincides with existing results on ergodic chains. The methodology employed relies on delineating the interrelations among Sarymsakov matrices, scrambling matrices, and positive-column matrices. Based on the theoretical results on inhomogeneous IPSMs, we propose a decentralized projected subgradient method for time-varying multiagent systems with graph-related stretches in (sub)gradient descent directions. The convergence of the proposed method is established for convex objective functions and extended to nonconvex objectives that satisfy Polyak-Lojasiewicz (PL) conditions. To corroborate the theoretical findings, we conduct numerical simulations, aligning the outcomes with the established theoretical framework.

Federated learning (FL) has emerged recently as an appealing and promising technique to deal with distributed learning issues in the sixth generation (6G) communication systems. Recent studies focus on developing FL schemes for terrestrial radio networks, where the variation in transmission data rates caused by transmission distance changes is negligible over one communication round. However, this variation has considerable influences for underwater acoustic channels. In this paper, we propose an underwater dynamic federated learning (UDFL) scheme by jointly considering characteristics of underwater acoustic channels and moving behavior of autonomous underwater vehicles. Moreover, an energy consumption minimization problem is formulated based on the scheme. To meet the challenges of transmission latency and FL performances, we consider them separately and provide closed-form solutions to the two individual problems. Specifically, we theoretically characterize the connections between transmission power and FL performances, and derive the optimal transmission policy given transmission latency constraints. Based on the two solutions, a dynamic programming based online power control algorithm is proposed to determine the transmission power across all time slots. Numerical simulations are conducted to demonstrate that the designed scheme is effective and the proposed online algorithm can achieve latency constrained energy-efficient UDFL.

The distributed subgradient (DSG) method is a widely used algorithm for coping with large-scale distributed optimization problems in machine-learning applications. Most existing works on DSG focus on ideal communication between cooperative agents, where the shared information between agents is exact and perfect. This assumption, however, can lead to potential privacy concerns and is not feasible when wireless transmission links are of poor quality. To meet this challenge, a common approach is to quantize the data locally before transmission, which avoids exposure of raw data and significantly reduces the size of the data. Compared with perfect data, quantization poses fundamental challenges to maintaining data accuracy, which further impacts the convergence of the algorithms. To overcome this problem, we propose a DSG method with random quantization and flexible weights and provide comprehensive results on the convergence of the algorithm for (strongly/weakly) convex objective functions. We also derive the upper bounds on the convergence rates in terms of the quantization error, the distortion, the step sizes, and the number of network agents. Our analysis extends the existing results, for which special cases of step sizes and convex objective functions are considered, to general conclusions on weakly convex cases. Numerical simulations are conducted in convex and weakly convex settings to support our theoretical results.

It is significant to design a task scheduling scheme for the multi-objective task of autonomous underwater vehicle (AUV) network for target hunting and environmental exploration. Due to the limited communication and detection conditions, it is difficult for each individual AUV in the network to accurately obtain all environmental information without a central control node. Therefore, most centralized scheduling schemes are infeasible to a fully distributed AUV network. To address the aforementioned issues, a distributed multi-agent proximal policy optimization (DMAPPO) scheme is proposed in this work, where AUVs are efficiently scheduled to achieve target hunting and environmental exploration. The distributed scheduling scheme is able to adjust the number of AUVs for each task according to practical requirement. In addition, we design an intra-network cooperative multi-AUV environmental exploration method by introducing the age of information (AoI). Simulation results validate that the proposed algorithm can achieve an effective task scheduling in the distributed AUV network.

In this article, we aim to design a task scheduling scheme for the underwater multiobjective task of target hunting and environmental search. A distributed autonomous underwater vehicle (AUV) network is deployed to perform the task, where AUVs equipped with sensors can cooperatively search the environment and hunt the target by sharing local information. To achieve efficient exploration of the overall environment by the AUV network, we design an intranetwork cooperative searching approach based on the Age of Information (AoI). Besides, it is critical to conceive an energy-efficient mechanism due to the energy constraints of AUVs and the difficulty of sustainable energy supply. To address the aforementioned issues, we propose an energy-efficient distributed multiagent proximal policy optimization (DMAPPO) scheme to perform real-time AUV target hunting and environment searching in underwater turbulent fields. The proposed scheme can adjust the number of AUVs assigned to each objective according to practical requirement and residual energy. Distributed AUVs can make decisions autonomously and cooperatively complete the task efficiently through limited information interaction. In addition, we derive a lower bound on the policy improvement of MAPPO. Moreover, our simulation results demonstrate that the proposed scheme outperforms the standard algorithms in terms of hunting efficiency, degree of searching, and network energy efficiency.

With the development of marine exploitation, underwater acoustic sensor networks (UWA-SNs) have become a hot research field. However, the harsh environment poses a threat to the security of underwater communication, as most routing protocols ignore the curve transmission of acoustic wave, which are more susceptible to transmission interference with higher transmission delay. To cope with these problems above, this work exploits a model under the assumption that the sound curve propagation relies on positive sound speed gradient. In order to find the path with the shortest delay, we design a routing scheme inspired by artificial fish swarm (AFS) and ant colony optimization (ACO) algorithms. Furthermore, we establish the path comprehensive benefit (PCB) to make a tradeoff between transmission delay and the lifetime of network. The simulation results validate that the algorithm proposed in this work is capable of improving the system performance compared to the benchmark algorithms in terms of both transmission delay and load-balance, and meanwhile ensuring paths reliability and security of the entire network.

The autonomous underwater vehicle (AUV) has gradually become an important platform for performing various underwater tasks. Due to the shortcomings resulting from a single AUV's poor detection, information processing and moving capabilities, more and more tasks are completed in a cooperative manner by multiple AUVs. However, most of the existing works do not consider security factors in the process of multi-AUV cooperation. In this paper, we propose a novel cooperative tracking scheme towards an underwater moving target, performed by an intelligent AUV swarm. In this scheme, a cooperative multi-agent reinforcement learning (MARL) based tracking algorithm is proposed following a centralized training with distributed execution (CT-DE) manner. After centralized training in the designed secure private network, no information sharing is required during the mission execution. This feature ensures the security of the whole system, especially in a complex confrontation scenario. In addition, we build models of the AUV underwater dynamics and the target sonar detection, which make the algorithm applicable to real target tracking enabled AUV swarms. Then, based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, we design an end-to-end AUV control algorithm. Simulation results validate that the proposed algorithm can achieve competitive performance in tracking success rate and tracking stability against baselines, while ensuring the security of the entire system.

In recent years, unmanned aerial vehicles (UAV) have been widely adopted to support complex target tracking tasks for military and civilian applications, especially in open and unknown environments. In practical cases, the moving trajectory of the target cannot be known to the UAVs in advance, which brings great challenges to UAVs to realize real-time and effective tracking. In addition, the limited tracking ability of a single UAV can hardly meet the requirements of a high tracking success rate. To deal with these problems above, this paper establishes a multi-UAV cooperative target tracking system. Besides, a deep reinforcement learning (DRL) based algorithm is designed to enable UAVs to make flight action decisions intelligently to track the moving air target, according to the past and current position information of the target only. To further increase the detection coverage of the UAV network when tracking, spatial information entropy is introduced to the reward designing in this algorithm. Simulation results validate that the proposed algorithm yields impressive target tracking performances, and significantly outperforms several common DRL baselines in terms of the tracking success rate. The convergence of the algorithm is also verified by the simulations.

We examine the problem of Value of Information (VoI) based underwater information collection, which is the essence of many underwater applications such as depth surrounding oil platforms, monitoring of algal blooms and so on. Even if the information collection work has been carried out much in the terrestrial scenario, due to complicated physical, technological and economic differences between the terrestrial and underwater cases, it is not feasible to simply apply the existing terrestrial tricks. The existing cooperative Autonomous Underwater Vehicle (AUV) working paradigms are limited to omniscience of communication channel information among the AUVs, which is yet not practical in such a harsh communication environment. Therefore, we propose a contract based model which has little restriction on the communication channel to overcome information asymmetry and jointly optimizes energy consumption and VoI. Besides, we provide a concrete theoretical proof of the contract items and carry out a performance simulation which shows the mechanism we design is operative. At last, we summarize our work and give an insight of the future research directions.