Reliable fault diagnosis for autonomous underwater vehicle (AUV) propellers is critical yet highly challenging in complex underwater environments. Diagnostic methods based on single optical video or electrical signals exhibit inherent limitations. Moreover, traditional multimodal fusion strategies suffer from a sharp performance decline when facing dynamic degradation of optical data quality (e.g., blur, low-light) due to the lack of a reliability assessment mechanism. To address this bottleneck, this paper proposes the quality-aware conditional modality gating attention network (QACMANet). This network introduces a dual-dimensional reliability assessment framework that integrates external objective image-quality cues with internal subjective model uncertainty. The mechanism synergistically evaluates data quality and model confidence, adaptively reducing reliance on optical data when it is unreliable and intelligently shifting reliance to the more stable electrical signals. Extensive experiments on a multimodal dataset featuring 11 test conditions demonstrate that QACMANet achieves an average accuracy of 92.86%, outperforming the best baseline by a significant margin of 7.78 percentage points, with the advantage being particularly pronounced under severe visual degradation. This research provides a robust solution for multimodal diagnosis in non-ideal underwater environments and validates the critical value of explicit reliability assessment to enhance the environmental adaptability of the system.

Accurate and automatic underwater target recognition is a compelling challenge for autonomous underwater vehicles (AUVs) in intelligent marine surveys. This study proposed a seabed target correction model based on side-scan sonar (SSS) images and combined the navigation information of AUV to achieve pixel-level geocoding. Moreover, a lightweight multi-level attention adaptive segmentation framework$^{^{^{^{}}}}$(${\rm{M}}{{\rm{A}}^{\rm{2}}}{\rm{Net}}$) was proposed to achieve fine-grained recognition. It contains three new modules: 1) The lightweight attention network (LAN) is designed as the baseline to obtain dense feature maps and focus on interesting features based on a balanced attention mechanism. 2) the multi-scale feature pyramid (MASPP) was then constructed to capture the context of SSS images and extract rich semantic information at high levels. 3) Finally, the adaptive feature fusion module (AFF) effectively incorporates feature maps of MASPP and spatial information to improve the learned representations further. Extensive experiments are verified on six SSS categories and show the remarkable performance of the${\rm{M}}{{\rm{A}}^{\rm{2}}}{\rm{Net}}$compared with state-of-the-art methods. Furthermore, real sea trials were conducted by deploying${\rm{M}}{{\rm{A}}^{\rm{2}}}{\rm{Net}}$to the autonomous target recognition (ATR) system of AUV, which can achieve 29.7 fps and 81.23% MIoU for a ($512\times 512$) input on a single Nvidia Jetson Xavier.Note to Practitioners—This paper aims to provide a real-time semantic segmentation model for the autonomous target detection of AUV, which is suitable for the autonomous detection of underwater targets by underwater robots (ROV, AUV, ARV, et al). This paper proposes a lightweight, multi-scale attention-adaptive segmentation framework (${\rm{M}}{{\rm{A}}^{\rm{2}}}{\rm{Net}}$) incorporating pixel-level seabed targets rectification methods. The algorithm has high segmentation accuracy and fast operation speed. It can identify seabed targets in high-resolution sonar images online and realize precise positioning of small seabed targets, which is conducive to improving the intelligence level of marine survey unmanned equipment. This paper details the design of${\rm{M}}{{\rm{A}}^{\rm{2}}}{\rm{Net}}$and the hardware structure of the autonomous target recognition system (ATR). Plenty of simulation experiments and sea trials have proved the efficiency and practicability of the method for the autonomous detection of different seabed targets (sand waves, coral reefs, metal balls, threads, and artificial reefs). Future research will verify the generalization of the algorithm in more seabed targets.

Underwater thrusters, vital components in various underwater vehicles, have been extensively studied for fault identification and classification. However, the automatic and unsupervised assessment of fault levels remains largely unexplored. This article presents a novel approach integrating physical information into a data-driven architecture and training process, enabling automated and unsupervised fault identification and evaluation. The process begins with constructing a physical model of the thruster and estimating its low-fidelity current based on the vehicle's velocity and the thruster's rotational speed. An improved SimGAN is then utilized, in combination with the vehicle's motion state, to map this low-fidelity current to its high-fidelity counterpart in the physical space. Statistical features are extracted from the absolute errors between the high-fidelity current and the measured current as conditions for the discriminator. The framework achieves thruster malfunction identification and assessment by analyzing the discriminator's output. Ocean trial data validate the effectiveness of this framework, with experimental results showing its satisfactory performance in fault identification, level evaluation, computational complexity, and robustness when compared with advanced methods.

Trajectory reconstruction (TR) plays an important role in practical applications. The data collected for TR are often contaminated with non-Gaussian noise due to environmental factors, which reduces the accuracy of TR. This study proposes a novel method to suppress the effects of non-Gaussian measurement noise. The method consists of two steps: decomposing the measurement noise and performing a weighted fusion of the relevant states. In the first step, the measurement noise is decomposed into a weighted combination of multiple Gaussian distributions using the Gaussian mixture model. In the second step, the interacting multiple model is employed to perform the weighted fusion of the relevant states. The main idea of the proposed method is to transform the state estimation problem from a non-Gaussian and nonlinear case into a state estimation problem in the nonlinear and Gaussian case. Based on this idea, a new unscented Kalman filter and an unscented Rauch–Tung–Striebel smoother framework are developed. The TR simulations and experiments are conducted for an underwater unmanned vehicle to verify the effectiveness and superiority of the proposed algorithms. The results demonstrate that the performance of the proposed algorithms is significantly better than that of the prominent existing algorithms, especially in the presence of non-Gaussian noise.

In this article, we propose an intelligent collaborative system for robotic navigation and control (CNaC) governed by the Euler-Lagrange equation. First, a state reconstruction based on neural networks navigation (SR-NNN) law is designed to estimate the current position of the robot for intelligent CNaC. The SR-NNN makes full use of partial truth information and the mighty local fitting ability of neural networks. In the absence of landmark, SR-NNN still exhibits navigation performance with high precision. The maximum root-mean-squared error (RMSE) of DR is 0.096 and the maximum RMSE of SR-NNN is 0.053, which has been improved by 55%. In addition, the motion model obtained by SR-NNN online training can avoid the error introduced by the predetermined motion model and overcome the interference of the external environment. The intelligent CNaC still can achieve satisfactory control performance based on the estimated position given by the SR-NNN rather than the ground truth which is formed by postprocessing. The intelligent CNaC has been demonstrated by simulation tracking sample and real experiments, which verifies the effectiveness of the intelligent CNaC.

The underwater environment is often poorly illuminated, making the images captured during underwater exploration missions poorly visible. Therefore, artificial light sources are often used to assist in underwater imaging. However, artificial light sources usually change the light conditions to a large extent, resulting in nonuniform illumination. To solve this problem, we propose a pseudo-siamese network that enables the separation of the nonuniform illumination layer and the ideal illumination image. And cascading iterative operation is utilized to achieve enhancement of image details and improve the image quality. To better balance the estimation quality of both, we propose residual reconstruction loss and structure loss as further guidance for network training. In addition, we simulate a nonuniform illumination model to create a dataset containing nonuniform illumination layers and ideally illuminated uniform images to alleviate the problem of insufficient data. The comprehensive experiments show that the algorithm can effectively correct the underwater nonuniform illumination image and recover the details of images better than the existing correction algorithms.

We propose a human-in-the-loop reinforcement learning mechanism to help robots learn emotional behavior. Unlike the previous methods of providing explicit feedback via pressing keyboard buttons or mouse clicks, we provide a more natural way for ordinary people to train social robots how to perform social tasks according to their preferences - facial expressions. The whole experiment is carried out on the desktop robot Haru, which is mainly used for the research of emotion and empathy participation. Our experimental results show that through learning from implicit feedback of facial features, Haru can quickly understand and dynamically adapt to individual preferences, and obtain a similar performance to learning from explicit feedback. In addition, we observe that the recognition error of human feedback will cause a “temporary regress” of the robot's learning performance, which is more obvious at the beginning of the training process. This phenomenon is shown to be correlated with the accuracy of recognizing negative implicit feedback.

Social robots that are able to express emotions can potentially improve human’s well-being. Whether and how they can learn from interactions between them and human being in a natural way will be key to their success and acceptance by ordinary people. In this paper, we proposed to shape social robot Haru affective behaviors with predicted continuous rewards based on received implicit facial feedback via human-centered reinforcement learning. The implicit facial feedback was estimated with the valence and arousal of received implicit facial feedback using Russell’s circumplex model, which can provide a more accurate estimation of the subtle psychological changes of human user, resulting in more effective robot behavior learning. The whole experiment is conducted on the desktop robot Haru, which is primarily used to study emotional interactions with human in different scenarios. Our experimental results show that with our proposed method, Haru can obtain a similar performance to learning from explicit feedback, eliminating the need for human users to get familiar with training interface in advance and resulting in an unobtrusive learning process.

Deep reinforcement learning has achieved significant success in many fields, but will confront sampling efficiency and safety problems when applying to robot control in the real world. Sim-to-real transfer learning was proposed to make use of samples in the simulation and overcome the gap between simulation and real world. In this paper, we focus on improving Progressive Neural Network — an effective sim-to-real learning method, by proposing Interactive Progressive Network Learning (IPNL). IPNL integrates progressive network and interactive reinforcement learning (interactive RL) which learns from evaluative feedback provided by an observing human trainer. We test our method using five RL tasks with discrete or continuous actions in OpenAI Gym and a sinusoids curve following task with AUV simulator on the Gazebo platform. Our results suggest that while Progressive Network has good performance when transferring from tasks with low-dimensional state space to those with high-dimensional one but has little effect for transferring from high-dimensional tasks to low-dimensional ones, IPNL allows an agent to learn a more stable policy with better performance faster for both cases. More importantly, our further analysis indicate that there is a synergy between Progressive Network and interactive RL for improving the agent’s learning. Our results in the path following of AUV shed light on the potential of applying our method in the real world tasks.

In order to maintain the submarine equipment, autonomous underwater vehicle (AUV) is usually assigned to track the submarine cables or pipes. The capabilities of navigation and control are critical to track the target accurately. Ultra-short baseline (USBL) is essential equipment for AUV, which uses sound waves for positioning. Unfortunately, due to the low frequency of USBL, it inevitably limits the frequency of control and ultimately affects the tracking effect. In order to improve the aforementioned issue and achieve better tracking tasks, intelligent collaborative navigation and control (CNaC) was herein proposed in this article. First, we proposed nonlinear state reconstruction neural network navigation, which used the neural networks to reconstruct the state between two adjacent USBL valid values online. Combined with the valid USBL and reconstructed states, the online process model generated by neural networks are applied to give the estimate position for AUV. At last, intelligent CNaC use the estimated position and valid USBL as inputs to control AUV to achieve tracking tasks. This strategy makes the control frequency free from the limitation of the USBL frequency. The proposed intelligent CNaC is demonstrated by simulation and real experiments. Compared to mechanically combining the traditional navigation and control algorithm, the tracking accuracy of intelligent CNaC improves by 81.96%.

This paper proposed an improved incremental smoothing and mapping (iSAM) which combines flexible re-linearization threshold with error learning model to improve the efficiency and accuracy of navigation for autonomous underwater vehicle (AUV). The flexible threshold of the proposed method, which can avoid periodic re-linearization of iSAM, is controlled by adaptive threshold. Through the use of flexible re-linearization, the proposed method can reduce the running time and carry out re-linearization timely. Simultaneously, for the first time, the proposed method takes advantages of the fast training time of hidden-layer neural networks and Gaussian Process Regression which is more suitable for non-linearity to get error learning model for iSAM. The proposed method can achieve better accuracy with only a rough model and can be easily transplanted to other systems without cumbersome calculations of the precise model. Our algorithm has been demonstrated by a range of simulated and real datasets, especially in practical application it outperforms current available iSAM algorithms in efficiency and accuracy. The RMSE of proposed method increases by 21.7% and efficiency also has been improved by 56% than iSAM2 in practical application for AUV.

Underwater images suffer from low visibility caused by absorption and scattering, which leads to haze and some further limitations. To overcome these issues, we propose a novel dehaze method based on a universal observation that all pixels of most underwater images tend to distribute nearby a specific plane in RGB space. This observation is named the color space dimensionality reduction prior. By projecting all pixels to this plane, namely the UV color space, the color distribution of these pixels can be reduced from the three-dimensional space (RGB space) to a new two-dimensional space (UV space) without causing any excessive color shift. By carefully setting the haze-free boundary in UV space, we can obtain the image transmission and finally produce an excellent dehazed image. Experimental results show that our method has competitive results compared with mainstream underwater single image dehazing methods.

As a branch of reinforcement learning, interactive reinforcement learning mainly studies the interaction process between humans and agents, allowing agents to learn from the intentions of human users and adapt to their preferences. In most of the current studies, human users need to intentionally provide explicit feedback via pressing keyboard buttons or mouse clicks. However, in our paper, we proposed an interactive reinforcement learning method that facilitates an agent to learn from human social signals - facial feedback via a ordinary camera and gestural feedback via a leap motion sensor. Our method provides a natural way for ordinary people to train agents how to perform a task according to their preferences. We tested our method in two reinforcement learning benchmarking domains - LoopMaze and Tetris, and compared to the state of the art - the TAMER framework. Our experimental results show that when learning from facial feedback the recognition of which is very low, the TAMER agent can get a similar performance to that of learning from keypress feedback with slightly more feedback. When learning from gestural feedback with a more accurate recognition, the TAMER agent can obtain a similar performance to that of learning from keypress feedback with much less feedback received. Moreover, our results indicate that the recognition error of facial feedback has a large effect on the agent performance in the beginning training process than in the later training stage. Finally, our results indicate that with enough recognition accuracy, human social signals can effectively improve the learning efficiency of agents with less human feedback.

Human-centered reinforcement learning (RL), in which an agent learns how to perform a task from evaluative feedback delivered by a human observer, has become more and more popular in recent years. The advantage of being able to learn from human feedback for a RL agent has led to increasing applicability to real-life problems. This paper describes the state-of-the-art human centered RL algorithms and aims to become a starting point for researchers who are initiating their endeavors in human-centered RL. Moreover, the objective of this paper is to present a comprehensive survey of the recent breakthroughs in this field and provide references to the most interesting and successful works. After starting with an introduction of the concepts of RL from environmental reward, this paper discusses the origins of human-centered RL and its difference from traditional RL. Then we describe different interpretations of human evaluative feedback, which have produced many human-centered RL algorithms in the past decade. In addition, we describe research on agents learning from both human evaluative feedback and environmental rewards as well as on improving the efficiency of human-centered RL. Finally, we conclude with an overview of application areas and a discussion of future work and open questions.

Programing robots to perform tasks is difficult in the real world because of its richness and uncertainty. For robots and agents to be more useful, they must be able to learn quickly from ordinary people via natural interactions. In this paper, we investigate how an agent can learn from demonstration and positive and negative evaluative feedback provided by a human teacher. Specifically, we proposed a model-based method-IRL-TAMER-by combining learning from demonstration via inverse reinforcement learning (IRL) and learning from human reward via the TAMER framework. We tested our method in the Grid World domain and compared with the TAMER framework using different discount factors on human reward. Our results suggest that although an agent learning via IRL can learn a useful value function indicating which states are good based on the demonstration, it cannot obtain an effective policy navigating to the goal state with one demonstration. However, learning from demonstration can reduce the number of human reward needed to obtain an optimal policy, especially the number of negative feedback. That is to say, learning from demonstration can be a jump-start for agent's learning from human reward and reduce the number of mistakes-incorrect actions. Furthermore, our results show that learning from demonstration can only be useful for agent's learning from human reward when the discount factor is small, i.e., learning from myopic human reward.

In this paper, we propose three sets of multi-channel image patch features for monocular visual-IMU (Inertial Measurement Unit) odometry. The proposed feature sets extract image patch exemplars from multiple feature maps of an image. We also modify an existing visual-IMU odometry framework by using different salient point detectors and feature sets and replacing the inlier selection approach with a self-adaptive scheme. The modified framework is used to examine the proposed feature sets. In addition to the Root Mean Square Error (RMSE) metric, we use the Hausdorff distance to measure the inconsistency between the estimated and ground-truth trajectories. Compared to the point-wise comparison used by RMSE, the Hausdorff distance takes the shape inconsistency of two trajectories into account and is hence more perceptually consistent. Experimental results show that the multi-channel feature sets outperform, or perform comparably to, the single gray level channel feature sets examined in this study. Particularly, the multi-channel feature set that uses integral channels, i.e., ICIMGP (Integral Channel Image Patches), outperforms two state-of-the-art feature sets: SIFT (Scale Invariant Feature Transform) and SURF (Speed Up Robust Features). Besides, ICIMGP performs better than the two multi-channel feature sets that are designed based on derivative channels and gradient channels respectively. These promising results are attributed to the fact that the multi-channel features encode richer image characteristics than their single gray level channel counterparts.