In this paper, we propose a novel integrated sensing and communication (ISAC) scheme tailored for UAVs-enabled vehicular networks, which leverages the information coverage capabilities of multiple UAVs and addresses critical challenges posed by multiple moving users. Unlike many traditional scheme, our scheme efficiently leverages ISAC signal echoes and real-time data uploads to provide communication services while achieving accurate sensing, thereby overcoming issues of resource waste and low operational efficiency. In the scheme, we aim to optimize both communication and sensing indicators, taking into account practical issues such as energy saving and collision avoidance for UAVs. However, the inherent complexity of multi-objective stochastic optimization in dynamic environments and limited communication resources render centralized UAV control inconvenient. To address the above challenges, we propose a novel multi-agent reinforcement learning (MARL) algorithm based on local information to realize the distributed adaptive control of motion decision, power selection, and channel allocation for UAVs. The algorithm combines random network distillation (RND) and dynamic data augmentation with multi-agent deep deterministic policy gradient (MADDPG) to encourage agents to explore effectively under sparse rewards and improve MADDPG's policy learning ability in finite data, thus approaching the global optimal solution. Experimental results demonstrate that the proposed algorithm can improve communication and sensing performance by more than 16.71% and 68.26% compared with other baselines and satisfy the set constraints. Furthermore, by adjusting hyperparameters, we can optimize the ISAC performance while achieving different energy savings levels for UAVs, proving that the designed scheme can reduce the waste of resources and improve the ISAC operation efficiency.

Since seamless tracking of the underwater target is crucial for various underwater applications, we propose a fusion algorithm combining deep learning and reinforcement learning for multi-autonomous underwater vehicles (AUVs) to seamlessly track the underwater target. The framework of our proposed fusion algorithm consists of two stages. In the first stage, we propose an underwater target localization method based on convolutional neural network (CNN) that relies on shaft-rate electric fields, in which the data collected by underwater sensors is utilized to train CNN to achieve accurate target localization. In the second stage, we innovatively propose a multi-agent soft actor-critic (MASAC) reinforcement learning algorithm based on centralized training with decentralized execution, in which appropriate reward functions are designed to encourage multiple AUVs to cooperate in seamlessly tracking the target in unknown environments while avoiding obstacles. Simulation results show that the proposed fusion algorithm has excellent performance, while the real-time target localization accuracy is 97.8%, and AUVs can carry out seamlessly cooperative tracking of the target in unknown environment.

Unmanned underwater vehicles (UUVs) face challenges such as high hardware costs, security concerns, a lack of training data in the actual development and debugging. Creating a simulation platform for simulation verification, training, and learning presents a potential solution to address these challenges. However, this area has seen limited prior work, and existing underwater platforms lack accuracy, user-friendliness, and intelligence. Therefore, this paper introduces an intelligent simulation platform “UUVSim” based on the robot operating system and Gazebo. UUVSim modular integrates basic modules such as high-precision simulation scenarios, dynamic models, sensors and controllers, while reserving programming interfaces. In addition, UUVSim provides reinforcement learning environment for UUV intelligent learning, supplemented with scenario transfer training, multi-agent reinforcement learning, offline reinforcement learning techniques to realize efficiently training for complex tasks, multi-robot coordination, and simulation to reality (sim2real) deployment. Further, we validate these technologies through underwater target tracking benchmarks and sim2real experiments, demonstrating the platform’s practicality.

In this study, an autonomous underwater vehicle (AUV) is considered to collect data in Internet of Underwater Things (loUT) networks. The AUV is tasked with timely visits to sensor nodes (SNs) to collect data using a navigation-hover-communication protocol. Considering the AUV's limited energy, the dynamic data upload demand of SNs and the diminishing Value of Information (Vol) during data transmission, efficient path planning for AUV is required. Therefore, we formulate a multi-objective optimization problem and employ the deep deterministic policy gradient (DDPG) algorithm to address it. Our objectives encompass the maximization of the sum data rate, the maximization of the sum Vol, and the minimization of the AUV's energy consumption within specified task time constraints. To mitigate the challenges posed by sparse rewards, we enhance the DDPG algorithm with hindsight experience replay (HER). The simulation results show that the data collection policy trained by our proposed algorithm can converge quickly and has excellent generalization. When the communication range changes, it can still effectively reduce the AUV's energy consumption while ensuring the quality and timeliness of the data collection task.

Autonomous underwater vehicles (AUVs) can function as sensor nodes in Internet of Underwater Things (IoUT), contributing to ocean exploration and monitoring by collecting and transmitting data to the base station. Most of the routing algorithms applied to IoUT require the participation of stationary nodes and seldom consider the fluctuations of network topology, which cannot be directly applied to the IoUT composed of AUVs. The focus of this research is to examine the problem of packet routing in a dynamic AUV-assisted IoUT, with the ultimate goal of ensuring the effective transmission of underwater information. We analyze the transmission pattern of underwater acoustic signals and the consequent communication disruption between AUVs, which helps establish the Age of Information (AoI) and bit error rate (BER) of data through modeling. A routing algorithm that utilizes the branch-and-bound (BB) technique has been suggested, alongside the introduction of the Value of information (VoI) to enable the joint optimization of the AoI and BER. We describe two nearly optimal heuristic algorithms for networks with a high number of AUVs. The AFA-ACO-BB strategy is designed based on the above algorithms and the influence of AUV motion on link reliability is considered. Moreover, we have developed a power regulation mechanism that can effectively minimize the occurrence of network packet loss and energy waste. The simulation results demonstrate that the proposed scheme outperforms certain classically related schemes in terms of AoI and BER, while simultaneously maintaining superior packet loss rate (PLR) and energy consumption.

In recent years, the Internet of Underwater Things (IoUT) has garnered significant attention owing to its potential in ocean exploration and monitoring. However, environmental erosion and limited energy can cause node failures, leading to routing voids, communication congestion, and even IoUT breakdowns. Addressing these challenges, this work considers a node repair scheme for multiple autonomous underwater vehicles (AUVs) to search and repair faulty nodes to ensure the stable operation of the IoUT networks. Moreover, AUVs should adapt automatically to the unknown environment, working in cooperative or separative modes to balance repair efficiency and coverage. We propose a multiagent reinforcement learning-based AUV-assisted node repair (RANR) scheme, which considers limited underwater communication and scheduling between AUVs. To further enhance work efficiency, we introduce area information entropy to reduce redundant coverage among AUVs. Simulation results demonstrate that the RANR scheme is highly applicable to different working conditions.

In recent years, unmanned aerial vehicles (UAVs) have been widely used in ocean target tracking and image acquisition for processing. Due to the limited energy of the UAV and the high computational complexity associated with image processing tasks, a lightweight energy-saving target tracking scheme is designed for the UAV, and the unmanned surface vehicle (USV) based mobile edge computing (MEC) networks are adopted to share the computing load of the UAV. Due to the randomness of the environment, we formulate data processing, computation offloading, resource allocation, and target-tracking as a joint stochastic optimization problem. This paper investigates a two-stage optimization scheme to address the problem. Firstly, we employ a Lyapunov-based approach to convert the stochastic optimization problem into a deterministic per-time slot problem under communication and computing resources constraints. Then, we develop a real-time target tracking scheme for the UAV based on the Elman neural network. Numerical results validate that the designed tracking scheme can effectively minimize propulsion energy consumption while maintaining a high success rate in tracking. Furthermore, the proposed method balances data-related energy consumption, image detection accuracy, and stability of the data storage queue.

It is significant to design a task scheduling scheme for the multi-objective task of autonomous underwater vehicle (AUV) network for target hunting and environmental exploration. Due to the limited communication and detection conditions, it is difficult for each individual AUV in the network to accurately obtain all environmental information without a central control node. Therefore, most centralized scheduling schemes are infeasible to a fully distributed AUV network. To address the aforementioned issues, a distributed multi-agent proximal policy optimization (DMAPPO) scheme is proposed in this work, where AUVs are efficiently scheduled to achieve target hunting and environmental exploration. The distributed scheduling scheme is able to adjust the number of AUVs for each task according to practical requirement. In addition, we design an intra-network cooperative multi-AUV environmental exploration method by introducing the age of information (AoI). Simulation results validate that the proposed algorithm can achieve an effective task scheduling in the distributed AUV network.

In this article, we aim to design a task scheduling scheme for the underwater multiobjective task of target hunting and environmental search. A distributed autonomous underwater vehicle (AUV) network is deployed to perform the task, where AUVs equipped with sensors can cooperatively search the environment and hunt the target by sharing local information. To achieve efficient exploration of the overall environment by the AUV network, we design an intranetwork cooperative searching approach based on the Age of Information (AoI). Besides, it is critical to conceive an energy-efficient mechanism due to the energy constraints of AUVs and the difficulty of sustainable energy supply. To address the aforementioned issues, we propose an energy-efficient distributed multiagent proximal policy optimization (DMAPPO) scheme to perform real-time AUV target hunting and environment searching in underwater turbulent fields. The proposed scheme can adjust the number of AUVs assigned to each objective according to practical requirement and residual energy. Distributed AUVs can make decisions autonomously and cooperatively complete the task efficiently through limited information interaction. In addition, we derive a lower bound on the policy improvement of MAPPO. Moreover, our simulation results demonstrate that the proposed scheme outperforms the standard algorithms in terms of hunting efficiency, degree of searching, and network energy efficiency.

With the development of marine exploitation, underwater acoustic sensor networks (UWA-SNs) have become a hot research field. However, the harsh environment poses a threat to the security of underwater communication, as most routing protocols ignore the curve transmission of acoustic wave, which are more susceptible to transmission interference with higher transmission delay. To cope with these problems above, this work exploits a model under the assumption that the sound curve propagation relies on positive sound speed gradient. In order to find the path with the shortest delay, we design a routing scheme inspired by artificial fish swarm (AFS) and ant colony optimization (ACO) algorithms. Furthermore, we establish the path comprehensive benefit (PCB) to make a tradeoff between transmission delay and the lifetime of network. The simulation results validate that the algorithm proposed in this work is capable of improving the system performance compared to the benchmark algorithms in terms of both transmission delay and load-balance, and meanwhile ensuring paths reliability and security of the entire network.