Underwater acoustic sensor networks (UASNs) are extensively utilized in various sectors, including aquaculture, naval operations, and oceanic disaster alert systems. The protection of UASNs, with a specific focus on internal threats, has become an increasing priority. Attacks originating from within the network, involving compromised legitimate nodes, can be more harmful and covert compared to external threats, such as communication interception, data decryption, and identity impersonation. Trust models, which serve as mechanisms for detecting internal threats through interaction data, have proven effective in enhancing UASN security. However, traditional trust models often face scalability issues, particularly in environments characterized by mobile underwater devices, diverse network conditions, and evolving attack strategies. To address these challenges, this work presents a novel trust management scheme based on attention-weighted federated deep reinforcement learning (AFRTM). The AFRTM overcomes the limitations of existing approaches by first improving the evidence quantification methods-encompassing both environmental and behavioral evidence-to better adapt to the uncertainty of underwater scenarios. Subsequently, the acquired trust evidence is input into the respective deep reinforcement learning (DRL)-driven local trust framework to achieve trust estimation and model development. Finally, the model's parameters are periodically aggregated and updated using an attention-weighted federated learning method, ensuring adaptability to changing conditions. The experimental findings demonstrate that the suggested approach delivers commendable outcomes in enhancing trust estimation precision and energy efficiency, and further providing a robust solution to the security challenges faced by UASNs.

Advancements in Wireless Buoy Network (WBN) have significantly accelerated the development of marine exploitation and monitoring, acting as a relay between underwater and surface networks in emerging 6G scenarios. Due to unstable maritime communication environment, it is a challenging issue to deploy the optimal data routing or collection strategies to ensure the collected data to be delivered to the target point. By employing the Software-Defined Networking (SDN) technology, this paper proposes the paradigm of Software-Defined WBN (SDWBN) to improve the network management efficiency and provide a platform to embed the Multi-Agent Reinforcement Learning (MARL) framework (for data routing intelligence), respectively. On account of the proposed SDWBN, this paper proposes a Reward Backpropagation-enabled Multi-Agent Deep Q-learning algorithm (RBMADQ)-based active routing scheme, which aims to assist buoys in making routing decisions and navigating the challenges posed by the dynamic and unstable communication environment. Further, this paper proposes a dual replay buffer-based training method, to enhance the convergence speed of the proposed RBMADQ-based routing scheme. Evaluation results demonstrate that the proposed routing scheme performs better compared with recent research products, with a higher packet delivery rate, lower network latency, and simultaneously, less communication overhead, etc.

Routing-driven timely data collection in Underwater Acoustic Sensor Networks (UASNs) is crucial for marine environmental monitoring, disaster warning, and underwater resource exploration, etc. However, harsh underwater conditions, including high delays, limited bandwidth, and dynamic topologies, make efficient routing decisions challenging in UASNs. In this paper, we propose a smart interrupted routing scheme for UASNs to address dynamic underwater challenges. We first model underwater noise influences from real underwater routing features, e.g., turbulence and storms. We then propose a Software-Defined Networking (SDN)-based Interrupted Software-defined UASNs Reinforcement Learning (ISURL) framework, which ensures adaptive routing through dynamical failure handling (e.g., energy depletion of sensor nodes or link instability) and real-time interrupted recovery. Based on ISURL, we propose the MA-MAPPO algorithm, integrating multi-head attention mask mechanism with MAPPO to filter out infeasible actions and streamline training. Furthermore, to support interrupted data routing in UASNs, we introduce MA-MAPPO_i, MA-MAPPO with interrupted policy, to enable smart interrupted routing decisions in UASNs. The evaluations demonstrate that our proposed routing scheme achieves exact underwater data routing decisions with faster convergence speed and lower routing delays than existing approaches.

Recent advances in network technologies and Multi Agent Reinforcement Learning (MARL) have accelerated the development of Autonomous Underwater Vehicle (AUV) cluster networks, enabling intelligent applications such as target tracking and cooperative target encirclement. However, existing MARL models are typically designed for single-task scenarios, limiting their scalability in real-world multi-task environments. To ad dress this, we propose a lightweight MARL framework capable of handling multiple AUV tasks with reduced reliance on underwater sampling. Specifically, a unified state space representation is constructed to support task generalization, while a hybrid online offline MARL training paradigm is introduced by leveraging the logical reasoning and sample generation capabilities of Large Language Models (LLMs). This reduces the demand for real-time data collection. Furthermore, a supervised pretraining strategy is incorporated to improve convergence and learning stability. Based on these components, we develop the Large Language Model-driven Hybrid online-offline MARL algorithm towards Multi-Task scenarios (LLM-HMT), which supports intelligent deployment of multi-task AUV cluster systems with minimal state representation, reduced sample requirements, and limited training iterations. Extensive experiments demonstrate that LLM HMT outperforms mainstream MARL baselines in convergence speed, task success rate, and resource efficiency, highlighting its potential for practical underwater applications.

Autonomous Underwater Vehicles (AUVs) in wireless cluster networks have shown great promise for ocean exploration, particularly in multi-target tracking, with Critical applications in both military and civilian purposes such as environmental monitoring and underwater resource exploration. This paper proposes a novel framework for smart underwater AUV wireless cluster networks by integrating Software-Defined Networking (SDN) and Multi-Agent Reinforcement Learning (MARL) to achieve efficient, scalable multi-target tracking in dynamic underwater environments. Specially, this paper introduces a Software-Defined Multi-Teacher-Student Reinforcement Learning (SD-TSRL) architecture that synergizes SDN's centralized control with MARL's adaptive decision-making, enabling intelligent communication and dynamic resource management. To further enhance learning efficiency, a reciprocal teacher-student mechanism is proposed, which optimizes resource allocation and communication during training. On account of the mechanism, this paper presents the Reciprocal Teacher-Student-Inspired Centralized (RTSIC) MARL algorithm, which improves both communication and computation resource utilization in AUV wireless cluster network. Experimental results demonstrate that the proposed approach significantly enhances tracking accuracy and network performance compared to existing methods, validating the effectiveness of SDN-MARL integration for advanced underwater wireless networks.

Edge computing is fundamental to filling the various quality-of-service needs for Industrial Internet of Things (IIoT) applications. However, introducing edge computing to IIoT inevitably results in hybrid intrusion problems and fails to satisfy the security demands of IIoT. Fortunately, Lagrange coded computing has emerged as a low-complexity and low-overhead solution for resisting hybrid intrusions during data offloading and processing. However, how to make decentralized, accurate, and real-time encoding/offloading/decoding decisions remains challenging. This article designs a fully decentralized training and decision-making framework to address the joint secure data offloading and resource allocation problem against hybrid intrusions for dynamic and uncertain IIoT, attempting to minimize the long-run energy and delay costs while improving the data confidentiality, integrity, and availability. It is proposed a fully decentralized multiagent actor–critic-based secure data offloading (FM-SDO) algorithm to solve the secure data offloading subproblem, wherein each industrial end device utilizes its local information to learn and execute its policy independently. This algorithm improves the structure of actor and critic networks and designs a multiagent alternant updating mechanism to increase learning accuracy, convergence, and stability. Based on the received offloading decisions of each device, each edge server leverages the Lagrange multiplier approach and Karush–Kuhn–Tucker condition to make fast and decentralized resource allocation decisions. Finally, we employed an IIoT intelligent production line platform named iCandyBox to test the performance of the FM-SDO algorithm. Experiment results suggest that the FM-SDO algorithm effectively reduces the total energy and delay costs while increasing the capability of resisting hybrid intrusions.

The multiple Autonomous Underwater Vehicle (AUV)-assisted cooperative system or the AUV-based Underwater Ad-hoc Networks (UAN) system has been considered as a highly-potential future in underwater data surveillance. In this paper, we propose grid-based distributed data collection architecture and define two categories of navigation modes. Based on the proposed data collection model, we propose MADAC, a scheme based on AUV-based UAN to cooperatively collect data from 6G-driven underwater wireless networks. We utilize the Software-Defined Networking (SDN) technique to re-organize the architecture of AUV-based UAN and propose software-defined actor-critic MARL framework. Based on the proposed MARL framework, we present the paradigm of MADDPG algorithm with optimal similarity attention mechanism (MADDPG-SA), to plan the paths for the AUV-based UAN, especially the cooperative underwater obstacle avoidance, the task distribution balancing, the Value of Information (VoI) are concurrently taken into account. In particular, the proposed MADDPG-SA improves the running efficiency of the proposed MADDPG-SA by encouraging the agent to learn from the similar and better-performance agent. The evaluation results demonstrate that the proposed MADAC can schedule the AUV-based UAN to perform efficient underwater data collection, reduce data collection time and energy consumption, and balance data collection tasks in the AUV-based UAN.

In recent years, autonomous underwater vehicle (AUV) swarms are gradually becoming popular and have been widely promoted in ocean exploration or underwater tracking, etc. In this paper, we propose a multi-AUV cooperative underwater multi-target tracking algorithm especially when the real underwater factors are taken into account. We first give normally modelling approach for the underwater sonar-based detection and the ocean current interference on the target tracking process. Then, based on software-defined networking (SDN), we regard the AUV swarm as a underwater ad-hoc network and propose a hierarchical software-defined multi-AUV reinforcement learning (HSARL) architecture. Based on the proposed HSARL architecture, we propose the “Dynamic-Switching” mechanism, it includes “Dynamic-Switching Attention” and “Dynamic-Switching Resampling” mechanisms which accelerate the HSARL algorithm's convergence speed and effectively prevents it from getting stuck in a local optimum state. Additionally, we introduce the reward reshaping mechanism for further accelerating the convergence speed of the proposed HSARL algorithm in early phase. Finally, based on a proposed AUV classification method, we propose a cooperative tracking algorithm calledDynamic-Switching-BasedMARL (DSBM)-driven tracking algorithm. Evaluation results demonstrate that our proposed DSBM tracking algorithm can perform precise underwater multi-target tracking, comparing with many of recent research products in terms of various important metrics.

With the rapid development of underwater materials technology and underwater robot technology, human exploitation of marine resources has been increasingly advanced, which has given rise to various application scenarios for Autonomous Underwater Vehicle (AUV) cluster networks, such as cooperative data collection and target tracking. In this paper, we aim to explore how to utilize networking and swarm intelligence to improve the AUV cluster network’s target tracking performance in a time-saving manner. Specifically, on account of our previous work, we introduce an underwater interrupted mechanism and propose an Interrupted Software-Defined Multi-AUV Reinforcement Learning (ISD-MARL) architecture. For MARL algorithm in ISD-MARL, we propose a time-saving MARL algorithm, S-MADDPG, integrating our proposed action optimization model and action network loss function, to accelerate the convergence of the MARL algorithm. Furthermore, to further improve the AUV cluster network’s path planning performance during the target tracking, we propose an Interrupted Tracking Path Planning Scheme (ITPPS) for the AUV cluster network based on the proposed ISD-MARL and S-MADDPG. The evaluation results showcase that our proposed scheme can effectively plan the underwater target tracking path for the AUV cluster network in a shorter time and outperform various mainstream strategies in terms of convergence speed and training time, etc.

Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL’s scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL’s scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.

Aquatic environments confront mounting threats from diverse sources, among which the persistent migration of continuous objects (e.g., chemical contaminants) is a primary concern. While advances have been made in tracking these entities using underwater acoustic sensor networks (UASNs), full-scale monitoring is challenged by unpredictable sensor deployments. Though Autonomous underwater vehicles show promise, their prohibitive costs and operational complexities limit their broad adoption. To tackle these issues, this article introduces a novel search strategy named virtual mobile agent-assisted continuous object tracking (VMA-COT). Drawing inspiration from binary tree structures, VMA-COT refines the search from the entire network to targeted hierarchical boundary mapping cells. Each cell encompasses a Section of the object’s boundary. A designated node within each cell evaluates information entropy at specified locations using a meticulously designed search sequence. By utilizing a feedback mechanism, grounded in the information entropy and the search sequence, VMA-COT progressively pinpoints the boundary. This methodology can be likened to a central node dispatching mobile agents in each cell. These agents, limited by a specific step range, adjust their trajectories for precise boundary delineation. Empirical tests and simulations demonstrate the effectiveness of VMA-COT, highlighting its efficiency, accuracy, and boundary node utilization.

In the development of Internet of Underwater Things (IoUT), the unknown nature of the underwater environment is a challenging issue. In various domains related to IoUT, utilizing autonomous underwater vehicles (AUVs) for unmanned and autonomous missions has become an inevitable trend. Considering the particularity of underwater environments, this study proposes a hybrid-algorithm-based full coverage search approach to searching moving targets in unknown underwater environments. This approach combines the improved Voronoi clustering strategy, the improved artificial bee colony (ABC) algorithm, the improved line-of-sight (LOS) technique, and the artificial potential field (APF) method to enhance the efficiency of underwater full coverage search (FCS). First, the improved Voronoi clustering strategy is employed to partition the entire underwater region and allocate each part to an AUV. Second, to enhance the search capability of AUVs, a full-dimensional ABC algorithm with adaptive factor is designed to plan global paths for AUVs to search for targets, and the paths are further smoothed using the improved acrlong SLOS technique. During the navigation of the AUVs along the global paths, obstacles may be detected; thus, the APF method is utilized to dynamically plan local paths for AUVs to avoid obstacles. Experimental results demonstrate that the proposed approach significantly improves the efficiency of underwater FCS.

With the rapid development of underwater robots, underwater communication techniques, etc., the Autonomous Underwater Vehicle (AUV) cluster network has emerged as a candidate paradigm to perform underwater civil and military applications, e.g., underwater target tracking. In this paper, we focus on how to utilize networking and multi-agent artificial intelligence technique to improve underwater target tracking. In particular, to improve the flexibility and scalability of the AUV cluster network, we employ Software-Defined Networking (SDN) and Centralized Training with Decentralized Execution (CTDE)-based Multi-Agent Reinforcement Learning (MARL) technologies, to propose a Hierarchical Software-Defined Multiple AUVs Reinforcement Learning (HSD-MARL) framework. For the MARL mechanism in HSD-MARL, we propose an advantage-attention mechanism and present the architecture of Multi-AUV Advantage-Attention Actor-Critic (MA-A3C), to address slow convergence and poor scalability issues on the AUV cluster network of large-scale. Further, to improve the utilization rate of advantage samples especially when the MA-A3C is utilized to perform AUV cluster network-based underwater tracking, we propose an ‘advantage resampling’ method based on experience replay buffer. Evaluation results showcase that our proposed approaches can perform exact underwater target tracking based on AUV cluster network systems and outperform some recent research products in terms of convergence speed, tracking accuracy, etc.