Edge computing is fundamental to filling the various quality-of-service needs for Industrial Internet of Things (IIoT) applications. However, introducing edge computing to IIoT inevitably results in hybrid intrusion problems and fails to satisfy the security demands of IIoT. Fortunately, Lagrange coded computing has emerged as a low-complexity and low-overhead solution for resisting hybrid intrusions during data offloading and processing. However, how to make decentralized, accurate, and real-time encoding/offloading/decoding decisions remains challenging. This article designs a fully decentralized training and decision-making framework to address the joint secure data offloading and resource allocation problem against hybrid intrusions for dynamic and uncertain IIoT, attempting to minimize the long-run energy and delay costs while improving the data confidentiality, integrity, and availability. It is proposed a fully decentralized multiagent actor–critic-based secure data offloading (FM-SDO) algorithm to solve the secure data offloading subproblem, wherein each industrial end device utilizes its local information to learn and execute its policy independently. This algorithm improves the structure of actor and critic networks and designs a multiagent alternant updating mechanism to increase learning accuracy, convergence, and stability. Based on the received offloading decisions of each device, each edge server leverages the Lagrange multiplier approach and Karush–Kuhn–Tucker condition to make fast and decentralized resource allocation decisions. Finally, we employed an IIoT intelligent production line platform named iCandyBox to test the performance of the FM-SDO algorithm. Experiment results suggest that the FM-SDO algorithm effectively reduces the total energy and delay costs while increasing the capability of resisting hybrid intrusions.

With the advancement of marine resource exploration and exploitation technologies, autonomous underwater vehicles (AUVs) have shown significant potential to perform underwater tasks, such as pipeline maintenance. However, traditional control algorithms, such as proportional-integral-derivative, sliding mode, and model predictive control, struggle to adapt to nonlinear and current-disturbed underwater environments, which impedes their accuracy and stability in path-tracking tasks. To address these challenges, this paper proposes a curiosity-driven distributional soft actor-critic framework. The framework leverages distributional soft actor-critic algorithms to control the navigation direction of the AUV and employs traditional proportional-integral control to maintain navigation speed, creating a stable, high-precision control strategy for complex underwater environments. Building on this framework, this paper further advances its capabilities through two key improvements. First, a curiosity-driven automatic entropy adjustment technique is designed to enhance the exploration of unknown states and the utilization of similar states, thus optimizing the accuracy of path tracking. Second, a dynamic prioritized experience replay mechanism is developed to prioritize learning samples based on time-differential errors and historical rewards, thereby improving learning efficiency and stability. Simulation experiments are conducted in two different underwater environments with and without ocean currents. Compared with reinforcement learning-based methods and traditional control algorithms, the proposed method has significant advantages in terms of accuracy, stability, and adaptability.

Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL’s scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL’s scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.

Data collection and transmission is the foundation for Internet of Underwater Things (IoUT) applications. Currently, quite a few autonomous underwater vehicle (AUV)-assisted data collection technologies have been proposed. Most of them concentrate on AUV path planning or multipath routing for data transmission, although MAC protocol design is crucial for reliable and secure data transmission in IoUT; hence, a MAC protocol based on parity group-graph coloring (PGGC-MAC) is investigated for underwater AUV-aided data collection. First, the AUV broadcasts path packets before data collection, informing sensor nodes of the path to travel in advance. Then, sensor nodes perform location update before AUV arrives, and collect the location information of neighbor nodes. Based on the position and the path information, the dynamic network topology is analyzed and the interference graph is obtained, which is used to assign working time slots for sensor nodes to transmit packets to the AUV. Finally, simulation results demonstrate that PGGC-MAC outperforms other related techniques in terms of network throughput, packet delivery ratio, energy usage, etc.

In underwater acoustic sensor networks (UASNs), the underwater sensors perform underwater data collection tasks, such as data collection and transmission at different locations in the monitoring area. To support cooperative underwater missions among the underwater sensor nodes, such as cooperative data delivery, one of the challenges is how to design smart underwater routing protocols that can guarantee safe, reliable, and energy-efficient data transfer among the underwater sensors. In this article, we introduce the paradigm of software-defined networking (SDN) and propose an SDN-based network framework for UASNs. Based on the proposed network framework, a$Q$-learning-assisted trust routing scheme for SDN-based UASNs (SDN-QLTR) is proposed. The proposed SDN-QLTR aims to seek for a secure routing path for executing underwater data transmission. Note that, in SDN-QLTR, effective trust evaluation methods are designed to resist malicious attacks initiated by nodes in UASNs. And SDN-QLTR integrates the advantages of SDN and reinforcement learning algorithm, can be flexibly applied in UASNs with dynamic features. Simulation results show that SDN-QLTR performs better in network lifetime, latency, and reliability.

Edge computing has become increasingly important to fulfill the diversified Quality-of-Service (QoS) or Quality-of-Experience (QoE) demands for Industrial Internet of Things (IIoT) applications, such as machine condition monitoring, fault diagnosis, intelligent production scheduling, and production quality control. Due to the heterogeneity of IIoT systems, it is of urgent necessity to concentrate on the cloud–edge–end cooperative partial task offloading and resource allocation (CPTORA) problem for realizing workload balancing, efficient resource utilization, and better QoS/QoE of IIoT applications. However, the challenge lies in how to make real-time, accurate, decentralized task offloading (TO) and resource allocation (RA) decisions for dynamic and device-intensive IIoT. Therefore, this work examines the CPTORA problem for IIoT, aiming at minimizing its long-run overall delay and energy costs. To lower the problem complexity, this problem is decomposed into the TO subproblem and the RA subproblem. Then, an improved soft actor–critic-based decentralized multiagent deep reinforcement learning (MADRL) algorithm is proposed to address the TO subproblem, where each IIoT device can learn its globally optimal policy and make its decisions independently. This algorithm innovatively combines the divergence regularization, the distributional reinforcement learning, and the value function decomposition methods to improve convergence speed and accuracy of the existing MADRL methods. After receiving the TO decisions of every IIoT device, every edge server employs the Lagrange multiplier method and Karush–Kuhn–Tucker condition to solve its RA subproblem. The experimental results show that the proposed algorithm decreases the overall delay and energy costs more effectively, compared to the other state-of-the-art MADRL approaches.

Data-driven approaches have gained great success in the field of rotating machinery fault diagnosis for its powerful feature representation capability. However, in most of the current studies, model training process requires massive fault data which are costly to gather or even unavailable in some extreme operating conditions. At the same time, structural relationships between samples are not fully exploited to facilitate the model performance. In response to these problems, a novel GHOAN for rotating machinery fault diagnosis is proposed in this study. Specifically, the proposed approach incorporates the advantages of improved graph attention network model and the multiorder neighborhood feature perception to achieve richer feature representation by aggregating features from multiple neighborhood domains. In this way, effective fault diagnosis may be achieved by using fewer training samples based on vibration signal analysis. The results of experiments conducted on two benchmarking datasets and a practical experimental platform show that the proposed GHOAN achieve superior performance.

Unauthorized underwater vehicles (UUVs) pose a serious threat to maritime security. To preserve maritime security, it is essential to pursue these UUVs. The majority of traditional pursuit methods are based on known environmental dynamics. However, the underwater environment is too complicated and unpredictable to describe these dynamics accurately. This study developed a novel online decision-making technique called multi-agent distributional soft actor-critic (MADA) to handle the issue of underwater cooperative pursuit. The method constructs a control-oriented framework based on multi-agent reinforcement learning that can map autonomous underwater vehicle (AUV) observations to pursuit actions. Multiple AUVs can combine to make prompt pursuit decisions. Then, the proposed method combines distributional soft actor-critic and curriculum learning to improve the success rates of multiple AUVs in pursuing UUVs. Experimental results show that the MADA can obtain a better cooperative pursuit strategy.

Path planning is a basic requirement for Autonomous Underwater Vehicles (AUVs) to accomplish underwater missions. However, previous studies often have limitations, such as ignoring the basic condition that the AUV operates in an ocean current environment and discretizing its actions without considering the action space, which results in the simulation being far from the actual situation. To solve the above problems, this paper proposes a method of using a Parametrized Deep Q-Network (PDQN) to output hybrid actions for path planning, which can output a hybrid action space based on the current local observation, flexibly avoid obstacles under limited sensor observations, and realize the refined operation of AUV actions. According to the setup of the simulation environment, the AUV needs to visit multiple target nodes underwater and decelerate within the communication range of the nodes to have enough time to communicate with the nodes. The PDQN enables the AUV to easily learn the connection between the current state and discrete actions. It outputs the corresponding continuous actions based on the current discrete actions, which realizes a time-saving strategy of accelerating and then decelerating among the nodes. Meanwhile, we also utilize the actual current data and terrain data to restore the simulation environment as accurately as possible, and the simulation results prove the superiority and robustness of the algorithm.

The fifth-generation of wireless communication (5G) is a promising paradigm toward massive interconnectivity within Internet-of-Things (IoT) networks. However, because the data traffic throughput sharply increases with the number of IoT devices, a tremendous burden on the backhaul links and core networks results. With this in mind, mobile edge caching is an effective method that can relieve stress of the backhaul links, while decreasing the service latency. The purpose of this study is to analyze the problem of jointly optimizing cooperative edge caching and radio resource allocation in 5G-enabled massive IoT networks. For that, a joint optimization long-term nonlinear integer programming problem is posed. This class of problems is known to be NP-hard; thus, to reduce the problem complexity, a divide and conquer scheme will be applied—the task at hand will be divided into two subproblems: 1) cooperative edge caching and 2) radio resource allocation. The cooperative edge caching subproblem is formulated as a constrained Markov decision process. Herein, a deep reinforcement learning method to optimize the caching decisions for all the edge nodes. Then, based on the resulting optimal caching decisions, the radio resource allocation subproblem for each edge node is posed as an NLIP problem, and an improved branch-and-bound method is proposed to yield the optimal radio resource allocation decisions for each edge node. Extensive simulations were performed to confirm that the proposed methods have the capability of enhancing the content caching hit ratio, while lessening the content retrieving delays for 5G-enabled massive IoT networks—improving over various baseline algorithms.