Port underwater inspection is essential for ensuring maritime safety and operational continuity. However, low visibility and complex environments make it challenging to achieve reliable detection. This study addresses the issue of underwater target recognition based on multibeam sonar imagery. Specifically, we propose YOLO-ShuffleNet-BiFPN-EIOU, an enhanced YOLOv5 detection framework with three key improvements: first, the adoption of the lightweight backbone network ShuffleNetv2, second, the incorporation of a bidirectional feature pyramid network, and third, the optimization of the enhanced intersection over union loss function. Furthermore, we theoretically derive the maximum permissible velocity threshold for autonomous underwater vehicle (AUV), leading to a novel velocity-constrained controller that improves sonar imaging quality. A general transformation function is applied, and a functional dependence with quasi-linear characteristics between the independent and dependent variables is established, converting the partially constrained AUV system into an unconstrained one. Finally, experiments conducted in a pool and a real-world port demonstrate that the proposed method achieves significant improvements in accuracy and efficiency compared to YOLOv5m, with [email protected] increasing by 3.4%, [email protected]:0.95 improving by 5.3%, giga floating-point operations per second reduced by 89.8%, and the velocity-constrained AUV operation effectively enhances detection performance.

Underwater target tracking is a critical challenge in marine exploration and defense applications due to the unknown maneuvers of target and the complex marine environment. To overcome the above challenge, this paper develops a digital twin (DT)-driven unknown maneuver target tracking strategy via remotely operated vehicles (ROVs). In order to capture the maneuver characteristics of target, a state prediction-based DT framework is constructed, where the neural network learning strategy is designed to estimate the unknown state transition matrix of target. Based on the predicted target state, a reinforcement learning (RL)-based tracking controller is designed for the virtual ROVs in DT model, such that the optimal tracking policy from DT model can be implemented to physical ROVs. To reduce the matching error between virtual and physical ROVs, an RL-based optimization algorithm is conducted by using the data interaction between DT model and ROVs. Note that the DT-driven target tracking strategy not only can reduce the communication energy consumption by periodically feeding back the real-data of ROVs to the DT model, but also can relax the dependence of target maneuver model via the state prediction method. Finally, experimental results are provided to verify the effectiveness of our strategy.

Underwater acoustic sensor networks (UASNs), which are formed by a number of interconnected mobile vehicles and static sensors, have emerged as a promising solution to explore and utilize the ocean resources. Typically, the localization, communication and control are the fundamental services for the applications of UASNs. Although they are closely related, the localization, communication and control issues are usually separately tackled. The separate design directly affects the localization accuracy, transmission reliability and control efficiency, especially for resource-constrained UASNs. In this regard, it is essential and necessary to co-design the localization, communication and control systems for UASNs. At present, the theoretical framework of the above integration design is still in the construction phase, and some key problems remain unresolved. Therefore, this article aims to give a survey on the integration design of localization, communication and control for UASNs. We first present the communication architecture, through which the main challenges aiming at the integration design are analyzed. After that, a holistic survey on the underwater localization, communication and control basics is provided. Followed by this, the recent advances on the integration design are given. At last, we make an outlook to the future research directions on the integration design of localization, communication and control for UASNs.

Underwater three-dimensional (3D) reconstruction is of great significance for underwater structure detection. However, ocean current disturbance and color distortion pose significant challenges in achieving reliable 3D reconstruction of underwater scenes. This paper investigates 3D reconstruction based on movable stereo camera via an underwater teleoperator. Specifically, a vision system based on ORB-SLAM3 is constructed to calculate the depth maps and perform dense 3D reconstruction through real-time video frames. An improved Multi-Scale Retinex with Color Restoration-Color Correction (MSRCR-CC) algorithm is proposed to address color distortion in underwater image enhancement. A redundant point removal algorithm is designed for the point cloud mapping process, effectively reducing the system's computational burden. Furthermore, to ensure stable scanning and obtain a high-quality original video stream in disturbed underwater environments, a disturbance observer-based prescribed performance control scheme is proposed for the underwater teleoperation system. The proposed control strategy guarantees that the master- slave synchronization error is bounded within predefined boundaries, which can enhance the fast fourier transform response value of the original image, and thus improve the 3D reconstruction quality. A global prescribed performance function is applied, which removes the initial value restriction on synchronization errors. Finally, simulation and experimental results are provided to illustrate the effectiveness of the proposed methods.

Localization is a key premise for implementing the applications of underwater acoustic sensor networks (UASNs). However, the inhomogeneous medium and the open feature of underwater environment make it challenging to accomplish the above task. This paper studies the privacy-preserving localization issue of UASNs with consideration of direct and indirect data threats. To handle the direct data threat, a privacy-preserving localization protocol is designed for sensor nodes, where the mutual information is adopted to acquire the optimal noises added on anchor nodes. With the collected range information from anchor nodes, a ray tracing model is employed for sensor nodes to compensate the range bias caused by straight-line propagation. Then, a differential privacy (DP) based deep learning localization estimator is designed to calculate the positions of sensor nodes, and the perturbations are added to the forward propagation of deep learning framework, such that the indirect data leakage can be avoided. Besides that, the theory analyses including the Cramer-Rao Lower Bound (CRLB), the privacy budget and the complexity are provided. Main innovations of this paper include: 1) the mutual information-based localization protocol can acquire the optimal noise over the traditional noise-adding mechanisms; 2) the DP-based deep learning estimator can avoid the leakage of training data caused by overfitting in traditional deep learning-based solutions. Finally, simulation and experimental results are both conducted to verify the effectiveness of our approach.

Formation control of autonomous underwater vehicles (AUVs) has been regarded as the basis of many sophisticated marine missions. However, the complex marine environment and the weak acoustic communication on AUVs make it hard to achieve the formation task. This paper attempts to overcome the above challenge from graph theory and intelligent learning perspectives. A local topology estimator is first designed by observing the coupled state evolution of AUVs, such that the unknown interaction relationship of AUVs can be inferred on the basic of local sensing. Based on this, we adopt the graph direction and contraction to generate an optimally persistent topology for AUVs, whose aim is to reduce the communication redundancy and guarantee the topology connectivity. With the optimized network topology, a model-free inverse reinforcement learning (IRL) formation controller is developed for AUVs to keep the desired formation shape. The innovations can be summarized as follows: 1) the local topology estimator can reveal the interaction topology relationship of AUVs with multiple degrees of freedom (DOF); 2) the optimally persistent topology can balance energy efficiency and topology connectivity as compared to the neighboring rule-based solutions; 3) the IRL-based formation controller has better adaptability to the underwater unknown environment as compared to the traditional reinforcement learning solutions. Finally, simulation and experimental results are both conducted to verify the effectiveness of our solution.

Flocking control of autonomous underwater vehicles (AUVs) has been regarded as the basis of many sophisticated marine coordination missions. However, there is still a research gap on the flocking of AUVs in weak communication and complex marine environment. This article attempts to fill up the above research gap from graph theory and intelligent learning perspectives. We first employ the bearing rigidity graph to describe the topology relationships of AUVs, through which an iterative gradient decent-based localization estimator is provided to obtain the position information. In order to improve the localization accuracy and energy efficiency, a min-weighted bearing rigidity graph generation strategy is developed. Along with this, we adopt the semi-supervised broad learning system (BLS) to design the model-free flocking controllers for AUVs in obstacle environment. The innovations of this article are summarized as follows: 1) the min-weighted bearing rigidity-based localization strategy can balance the localization accuracy and communication consumption as compared to the neighboring rule-based solutions and 2) the semi-supervised broad learning-based flocking controller can decrease the training time and solve the label limit over the supervised learning-based controllers. Finally, simulation and experimental studies are provided to verify the effectiveness.

Formation control of autonomous underwater vehicles (AUVs) has been regarded as the basis of many sophisticated marine missions. However, the complex marine environment and the high communication energy consumption make it hard to achieve this task. This article is concerned with an energy-efficiency formation issue of AUVs via angle measurement. Particularly, the single vector hydrophone is used to measure the single-frequency signal emitted by AUVs, through which the relative angles among AUVs can be estimated. Based on this, a minimally angle rigid topology generation algorithm is designed to balance the tradeoff between communication energy efficiency and topology connectivity, while a model-free inverse reinforcement learning (IRL)-based formation controller is developed to steer AUVs to reach the target while maintaining a specific shape. The innovations are summarized as follows: 1) the angle measurement in this article can eliminate the reliance on the position information of AUVs; 2) the minimally angle rigid topology in this article can improve the formation stability and reduce the communication energy consumption as compared to the neighboring rule-based solutions; and 3) the IRL-based controller can avoid manually designing cost functions and improve environmental adaptability as compared to traditional-learning-based controllers. Finally, simulation and experimental results are both conducted to verify the effectiveness.

Formation control of remotely operated vehicles (ROVs) has been regarded as the basis of many sophisticated marine missions. However, the high communication energy consumption and weak environment perception ability on ROVs make it challenging to achieve this task. To overcome the above challenge, this article develops a digital twin (DT)-driven formation control approach for ROVs. We first establish a virtual twin model for each ROV by extracting the motion parameters and environment information. With the collected states from ROVs, an integral reinforcement learning (IRL) based formation controller is designed to drive the motion outputs of DT model. After that, the optimal control policy from the DT model is employed to accomplish formation task for each ROV. To reduce the matching error and ensure the formation stability, an IRL-based optimization algorithm is conducted by using the data interaction between DT model and ROVs. Note that the DT-driven formation solution not only can reduce the communication energy consumption by periodically feeding back the real-data of ROVs to the DT model, but also can improve the perception ability of ROVs by reconstructing a virtual twin environment. Finally, experimental results are provided to verify the effectiveness of our solution.

Due to energy constraint and reliability consideration, autonomous underwater vehicles (AUVs) have fewer independent actuators than their degrees of freedom (DOFs). Additionally, the position and velocity of AUV are sometimes limited due to physical constraints. The current solutions, such as Barrier Lyapunov Function (BLF) and Nonlinear State Dependent Function (NSDF), depend on the upper bounds of virtual controllers and dynamic surface control (DSC) technique. This paper develops a new trajectory tracking controller for underactuated AUV systems with state constraints. A quasi-linear relationship is established between the independent and dependent variables of the transformation function. The Nussbaum functions are employed to address algebraic loop problem, which avoids using the DSC and any approximator. Moreover, an auxiliary controller is introduced to deal with underactuation problem. The Lyapunov theory proves that the proposed controller can guarantee asymptotic tracking of desired trajectory while keeping the position and velocity of the AUV within their constrained bounds. The method can be extended to the nth-order parametric-strict-feedback nonlinear systems. Finally, both simulation and experimental results reveal that tracking performance can be guaranteed by the proposed control scheme.

Motion planning of underwater vehicles is regarded as a promising technique to make up the flexibility deficiency of underwater sensor networks (USNs). Nonetheless, the unique characteristics of underwater channel and environment make it challenging to achieve the above mission. This article is concerned with a communication-efficient and collision-free motion planning issue for underwater vehicles in fading channel and obstacle environment. We first develop a model-based integral reinforcement learning (IRL) estimator to predict the stochastic signal-to-noise ratio (SNR). With the estimated SNR, an integrated optimization problem for the codesign of communication efficiency and motion planning is constructed, in which the underwater vehicle dynamics, communication capacity, collision avoidance, and position control are all considered. In order to tackle this problem, a model-free IRL algorithm is designed to drive underwater vehicles to the desired position points while maximizing the communication capacity and avoiding the collision. It is worth mentioning that, the proposed motion planning solution in this article considers a realistic underwater communication channel, as well as a realistic dynamic model for underwater vehicles. Finally, simulation and experimental results are demonstrated to verify the effectiveness of the proposed approach.

Localization is an indispensable service for underwater sensor networks (USNs). Generally, the convex optimization method is adopted to solve the localization problem. However, the acoustic ray in water medium does not propagate along a straight line, which makes it difficult or impossible to transform the nonconvex optimization problem into a convex optimization problem. This article develops a broad learning (BL)-based localization solution for USNs with isogradient sound speed profile. We first employ the ray tracing model to compensate the range bias caused by straight-line propagation. On the basis of collected range information from anchor nodes, the localization optimization problem is transformed into supervised, unsupervised, and semisupervised learning frameworks. Correspondingly, three BL-based location estimators are developed to seek the position information of sensor nodes, where the incremental learning schemes are conducted for fast parameter tuning and remodeling. In addition, the Cramer–Rao lower bound (CRLB) of positioning error and the convergence to global optimality are both analyzed. Finally, simulation and experiment results are presented to show the effectiveness of our approach. It is demonstrated that the proposed solution in this article has the following nice features: 1) relax the dependence of convex relaxation over convex optimization-based location estimators and 2) reduce the training time and improve the localization efficiency over deep-learning-based location estimators.

Communication-aware motion planning of autonomous underwater vehicle (AUV) is regarded as an emergent requirement for marine intelligent transportation system. However, the fading acoustic channel and the complex underwater environment make it difficult to realize such task. This paper is concerned with a communication-aware motion planning issue for AUV in obstacle-dense environment. We first develop an intelligent AUV system, which includes binocular cameras for short-distance obstacle avoidance, sonars for long-distance detection, and modems for acoustic communication with buoys. For such system, the parallax angles from AUV to obstacles are utilized to construct an optimal motion planning problem by integrating our previously proposed channel estimation approach. In order to solve the above problem, a deep learning method called depth deterministic policy gradient (DDPG) is developed to minimize the cost function, such that a collision-free path can be planed for AUV while maintaining the communication quality. Note that the advantages of our solution are highlighted as: 1) balance the communication quality and motion stability over the disk model-based methods; 2) improve the collision-avoidance efficiency in path lengths and control efforts as compared with the distance-based methods. Finally, simulation and experimental studies are both provided to verify the effectiveness of our method.

Communication efficiency and flocking stability are two basic requirements for the application of autonomous underwater vehicles (AUVs) in maritime transportation systems. Although they are closely related, most existing flocking approaches focus on the control techniques and ignore the influence of communication efficiency. This paper presents a joint design solution to the channel estimation and flocking control for multi-AUV-based maritime transportation systems, with the consideration of path loss, shadow and multipath fading channels. A value iteration-based reinforcement learning (RL) estimator is first designed to predict the channel quality of AUVs in positions that have not yet visited. With the predicted channel quality, we construct an integrated optimization problem for the co-design of communication and flocking strategies. Along with this, a value iteration-based RL flocking controller is developed to achieve the co-design of channel estimation and flocking control for AUVs. It is worth mentioning that, the value iteration-based RL estimator in this paper can avoid local optimal in traditional least-squares methods, and meanwhile the flocking controller in this paper can make a balance between flocking stability and communication efficiency for AUVs. Finally, simulation and experimental results reveal that the proposed approach in this paper has superior performances by comparing with the other works. As such, our approach is more useful for marine engineer to understand and explore the maritime transportation system from the communication and control view points.

This article is concerned with a containment control issue for autonomous underwater vehicles (AUVs), subject to unavailable velocity signals in cyber side and stochastic environment disturbances in physical side. We first divide the environmental disturbances into deterministic and stochastic parts. Based on this, a terminal sliding mode observer is developed to estimate the velocities of AUVs in finite time. With the estimated velocities, a distributed containment controller is designed for each AUV to follow a convex hull spanned by trajectories of the leader AUVs. For the developed velocity observer, a double power reaching law is employed to reduce the chattering and improve the convergence rate. Besides that, an adaptive strategy is incorporated into the containment controller, such that the steady-state errors caused by stochastic environment disturbances can be compensated. Stability conditions for the velocity observer and containment controller are also provided. Finally, we conduct the simulation and experimental studies to verify the effectiveness.

Human-on-the-loop (HOTL) system is regarded as a promising technology to allow autonomous underwater vehicle (AUV) to track the most adequate target point as soon as possible. However, the unique characteristics of the underwater environment make it challenging to perform the tracking task. This article is concerned with a finite-time tracking control issue for AUV, subjected to unavailable velocity signals in the measurement side and uncertain model parameters in physical side. A HOTL system, including operator, buoys, AUV and sensors, is first provided to construct a cooperative tracking network. For such system, operator in surface control center decides the tracking mission based on all available data. Then, a buoy-assisted localization estimator is utilized by AUV to acquire its position, through which a fast terminal sliding mode observer is developed to estimate the velocity of AUV in finite time. With the estimated velocity information, an adaptive-nonsingular fast terminal sliding mode tracking controller is designed to drive AUV to the target point in finite time. For the proposed velocity observer and tracking controller, the signum and differential functions are employed together to improve the convergence speed and reduce the chattering. Besides that, the proposed solution can not only guarantee finite-time velocity observation, but also achieve finite-time tracking control. Finally, simulation and experimental results are both presented to verify the effectiveness.

This article studies the joint localization and tracking issue for the autonomous underwater vehicle (AUV), with the constraints of asynchronous time clock in cyberchannels and model uncertainty in physical channels. More specifically, we develop a reinforcement learning (RL)-based asynchronous localization algorithm to localize the position of AUV, where the time clock of AUV is not required to be well synchronized with the real time. Based on the estimated position, a scalable sampling strategy called multivariate probabilistic collocation method with orthogonal fractional factorial design (M-PCM-OFFD) is employed to evaluate the time-varying uncertain model parameters of AUV. After that, an RL-based tracking controller is designed to drive AUV to the desired target point. Besides that, the performance analyses for the integration solution are also presented. Of note, the advantages of our solution are highlighted as: 1) the RL-based localization algorithm can avoid local optimal in traditional least-square methods; 2) the M-PCM-OFFD-based sampling strategy can address the model uncertainty and reduce the computational cost; and 3) the integration design of localization and tracking can reduce the communication energy consumption. Finally, simulation and experiment demonstrate that the proposed localization algorithm can effectively eliminate the impact of asynchronous clock, and more importantly, the integration of M-PCM-OFFD in the RL-based tracking controller can find accurate optimization solutions with limited computational costs.

Underwater sensor networks (USNs) are envisioned to enable a large variety of marine applications. Such applications require accurate position information of sensor nodes. However, the openness and inhomogeneity characteristics of underwater medium make it much more challenging to solve the localization issue. This paper is concerned with a privacy-preserving localization issue for USNs in inhomogeneous underwater medium. An honest-but-curious model is considered to develop a privacy-preserving localization protocol. Based on this, a localization problem is constructed for sensor nodes to minimize the sum of all measurement errors, where a ray compensation strategy is incorporated to remove the localization bias from assuming the straight-line transmission. To make the above problem tractable, we consider the unsupervised, supervised and semisupervised scenarios, through which deep reinforcement learning (DRL) based localization estimators are utilized to estimate the positions of sensor nodes. It is noted that, the proposed localization solution in this paper can hide the private position information of USNs, and more importantly, it is robust to local optimum for nonconvex and nonsmooth localization problem in inhomogeneous underwater medium. Finally, simulation studies are given to show the position privacy can be preserved, while the localization accuracy can be enhanced as compared with the other existing works.

Most studies so far on trajectory tracking control of autonomous underwater vehicle (AUV) have assumed that the Euler angles are exactly known. However, the AUV inevitably suffers from external environmental disturbances which are driven by wind, density, and temperature gradients. The attitude transducers cannot derive accurate attitude information of the AUV. Additionally, the uncertain hydrodynamic parameters affect the stability of the system. Consequently, it is unknown whether tracking performance of the AUV can be guaranteed. In order to overcome these drawbacks, in this paper, a finite-time controller is developed by using the nonsingular fast terminal sliding mode control technique. A robust differentiator is proposed to estimate the external disturbances and uncertain parts. Simulations are performed to show that with the proposed control laws, the AUV converges to the desired trajectory even in the presence of external disturbances and system uncertainty.

Most research so far in teleoperation control has assumed that all information is transmitted continuously. Unfortunately, the damaged and electromagnetic interfered line cause communication link failure. In addition, the unreliable link further leads to port data congestion. The data packet will be discarded when the buffer overflows. Consequently, it is unknown whether stability of the teleoperator could be guaranteed in the presence of intermittent communications. In order to overcome these drawbacks, in this paper, we provide a solution to the formation control problem of a single-master-multislave teleoperator in the situation where each robot is allowed to communicate with its neighbors only at some irregular discrete time instants. The relationship among control gains, topology, and maximum-allowable connected interval is presented. Simulations are performed to show the validity of our proposed approach.

In this correspondence paper, an exact method is developed to guarantee asymptotic stability of a bilateral teleoperation system that is subjected for a time-delayed communication. This extends the prior art of searching for the maximum upper bound of time delay. In order to improve the flexibility in controller design and obtain better performance, a fractional-order PDαcontroller is proposed. The exactly stable regions of delays are explored for both integral-order and fractional-order controllers. Compared with conditions in most previous works which are deduced by the Lyapunov-Krasovskii functional and rely on the solution of some linear matrix inequalities, the stability conditions proposed in this paper are established from the frequency domain point of view, and thus, the results are not only sufficient but also necessary. To illustrate accuracy of the conditions, they are simulated on a delayed teleoperation system composed of a pair of robots.