Over prolonged evolution, natural fish possess exceptional maneuverability. For underwater robotic fish, maneuverability is a crucial performance metric during locomotion. This paper proposes a turning control optimization framework for a multi-flexible-joint bionic robotic fish. Firstly, three distinct turning strategies are devised based on kinematic analysis of the flexible robotic fish’s turning motion. Experimental results indicate that the control strategies need to be adjusted with respect to motion frequency to achieve optimal performance for the flexible robotic fish. Furthermore, an optimization problem incorporating dynamic constraints and cost functions of the flexible robotic fish is constructed. Based on a Constrained Iterative Linear Quadratic Regulator (CILQR), a turning performance optimization method is designed. Subsequently, optimal control strategies under both stationary and motion states are derived, effectively enhancing the turning performance of the flexible robotic fish. Finally, simulation and experimental results validate the effectiveness of the designed method. The developed flexible robotic fish could achieve a maximum swimming speed of 1.63 BL/s (body lengths per second) and a maximum average turning speed of 130.5°/s, offering guidance for the practical application of flexible robotic fish in aquatic environments.

Compared with traditional underwater vehicles, robotic fish have been receiving increasing attention in recent years due to their excellent maneuverability. However, the characteristics of fishlike undulatory motions and complex underwater working environment have posed significant challenges to robotic fish speed measurement, limiting their autonomy. To overcome these challenges, an artificial lateral line sensor (ALLS) was developed, drawing inspiration from the tactile system of fish. It captured the real-time speed of robotic fish through assessing the deformation of the stressed component under laminar flow impact. To mitigate turbulence disturbances near the ALLS, three flow control components, fairing, flow conditioner, and flow collector, were proposed to attenuate turbulence noise under the viscous effect. Furthermore, a physics-informed calibration method was presented to establish the nonlinear model of ALLS. Specifically, a physical model embedding algorithm based on data resampling was used to mitigate the risk of overfitting by the multilayer perceptron, considering the influence of turbulence disturbance and fishlike undulatory noise. Compared with the classical calibration method based on physical model fitting, the calibration method proposed in this paper reduced the error by 36.0%. Our ALLS’s final mean absolute error was 0.016 m/s with a linearity (R2) of 0.956. The experimental results indicated that the significant changes in the motion state of robotic fish reduced the accuracy of ALLS. The fusion with other sensors is expected to enhance the robustness of ALLS in the future. Note to Practitioners—The motivation of this paper is to design an artificial lateral line sensor based on surface flow field detection and turbulence noise suppression, providing a small-sized and high-precision solution to the speed measurement problem of bionic robotic fish. Most existing ALLS research focused on developing new types of sensors based on different measurement principles, without suppressing the noise caused by fishlike motions, and most experiments were conducted in environments with excessive controls rather than free-swimming robotic fish. To this end, we developed an ALLS based on deformation measurement and proposed three flow control components to make the measured flow more stable. Furthermore, a physics-informed overfitting suppression method was used for the calibration task of the ALLS. A series of simulations and experiments demonstrated that the proposed turbulence noise suppression and calibration method were practical and effective. Hopefully, our methods can provide theoretical and technical guidance to marine engineers for underwater vehicle speed measurement and flow sensing. The recommended flow control component is applicable for conditioning surface fluids in pneumatic control systems. Furthermore, the proposed biomimetic tactile sensor is poised to inspire tactile-based human-machine interaction methods.

Underwater collision-free navigation and dense reconstruction are essential for marine refined exploration. However, existing passive vision-based methods are difficult to apply in low-light and weak-feature underwater environments. In this article, a more adaptable three-dimensional (3-D) dense mapping robotic system based on self-designed scanning binocular structured light (BSL), named ROV-Scanner, is developed to address this challenge. First, the measurement principle based on the refraction model ensures its high accuracy. Second, an underwater 3-D dense mapping algorithm fusing the Doppler velocity log (DVL), inertial measurement unit (IMU), and pressure sensor multifrequency information is proposed to realize dense mapping during robot motion. Then, an air–water two-stage extrinsic calibration algorithm is proposed. In particular, the extrinsic parameters between DVL and camera are innovatively calibrated using BSL, enhancing robustness. Furthermore, for the first time, a framework of BSL-based collision-free navigation is presented to guarantee the safe movement of the system in unknown environments. Experimental results show that our system can simultaneously achieve autonomous collision-free navigation and dense mapping in dark underwater environments, which has great potential for application in marine refined exploration.

Due to the complex mechanism and fabrication process of flexible materials, it remains extremely challenging for a flexible robotic fish to achieve fast and efficient locomotion. In this article, taking advantage of the passive bending and energy storage properties of flexible materials, we propose an untethered robotic fish with multiple flexible joints to achieve high performance and low Cost of Transport (COT). First, combining rigid links and flexible materials, a compact flexible tail with a simple and efficient structure is proposed. Next, the pseudo-rigid body theory is applied to analyze the deformation of passive joints, and a full-state dynamic model is established. More importantly, an optimization method by adjusting the phase differences of the passive joints is used to obtain high aquatic performance. Finally, extensive simulations and experiments validate the effectiveness of the proposed method, and the robotic fish can achieve a maximum speed of 1.63 body length (BL) per second and a minimum COT of 4.8 J/m (2.87 J/m$\cdot$kg). Compared with the multi-joint robotic fish with a similar design, the COT is reduced by up to 81.05% with the basically same aquatic ability. Excitingly, the flexible robotic fish can achieve a COT of 7.36 J/m at 1.23 BL/s, which is 15.72%-36.34% lower than that of the bluefin tuna and is within the range of yellowfin tuna, offering valuable insight into high speed and long endurance applications for underwater robots.Note to Practitioners–This paper is motivated by the design and optimization of an efficient bionic flexible underwater robot with high aquatic abilities, which is conducive to aquatic tasks that require long-time and long-distance sailing, such as underwater topographic exploration, submarine archaeology, and underwater search and rescue. Existing studies of free-swimming bionic underwater robots usually focus on the improvement of swimming speed and rarely consider achieving both high swimming performance and low energy cost. Thus, this paper proposes a bionic underwater robot design with two joints made of flexible materials on the tail to address the problem. Based on detailed analyses of the hydrodynamic force and flexible joint deformation, we propose an effective optimization method for swimming performance. A series of simulations and experiments suggest that the mechatronic design and optimization method are practical and valid. Hopefully, our design and method can provide theoretical guidance for engineers to design and optimize robots with flexible joints.

Structured light systems are widely used in underwater dense reconstruction due to their excellent accuracy. However, the current related methods mainly focus on fixed positions. The reconstruction performance in motion is insufficient. Therefore, we propose an underwater movable binocular structured light (MBSL) based high-precision dense reconstruction framework, named WaterMBSL, to realize the robot reconstruction while moving. Specifically, an onboard binocular structured light system based on mirror-galvanometer is developed first. Then, a simplified underwater point cloud acquisition algorithm is presented to quickly obtain 3-D information of the scene. Besides, a new underwater motion compensation algorithm combining inertial measurement unit and uniform velocity model is proposed. Moreover, the generalized-ICP point cloud registration algorithm is introduced to achieve accurate motion estimation. Finally, an underwater movable reconstruction platform is developed by integrating the self-designed structured light system with the underwater robot BlueROV for validating the performance of our proposed Water-MBSL. Experimental results show that satisfactory motion reconstruction performance can be obtained.

The high speed and low energy cost are two conflicting objectives in the motion optimization of bio-inspired underwater robots, but playing a very important role. To this end, this paper proposes an optimization strategy for swimming speed and power cost using an improved NSGA-II for a flexible robotic fish. A dynamic model involving flexible deformation is established for speed prediction with the hydrodynamic parameters identified. A back propagation (BP) neural network is applied to perform compensation of power cost prediction with the dynamic model's prediction as input. In particular, an NSGA-II-AMS method is developed to improve the efficiency of solving the two-objective optimization problem based on NSGA-II. Finally, extensive simulations and experimental results demonstrate the effectiveness of the proposed optimization strategy, which offers promising prospects for the flexible robotic fish performing aquatic tasks with different performance constraints.

The past decade has seen the rapid development of the robotic fish in many aspects. However, the velocity measurement problem has not been fully addressed, which limits the autonomy of the robotic fish. To this end, an artificial lateral line (ALL) sensor, inspired by the sensory organs of fish, is developed in this article. By measuring the deformation of the sensitive element, the local flow field around the robotic fish is sensed. According to the characteristics of fishlike motions, a fairing structure is proposed to suppress the turbulence noise and yaw motion noise caused by fishlike oscillation of the tail. This structure ensure that the flow measured by the ALL sensor is closer to laminar flow under viscous effects. Furthermore, to measure the magnitude and direction of the robotic fish velocity, an ALL sensor array is assembled by mounting multiple sensors on the robot's surface to sense the flow field distribution. Next, a kinematic-based fusion method is proposed for the array system, which obtained the real-time velocity vector of the robotic fish by solving overdetermined motion equations. The proposed ALL array system is tested on a freely swimming robotic fish, and our method achieves a mean absolute error of 0.018 m/s, a linearity ($R^{2}$) of 0.951, and a position tracking error of 0.085 m. Additionally, the fairing structure is found to improve the signal-to-noise ratio by 116%.

Seam feature point acquisition is the premise of the intelligent welding process such as initial point guiding and seam tracking. However, conventional seam feature point acquisition methods based on geometric feature have shortcomings of poor flexibility and robustness. In this article, a seam feature point acquisition method based on efficient convolution operator (ECO) and particle filter (PF) is proposed, which could be applied to different weld types and could achieve fast and accurate seam feature point acquisition even under the interference of welding arc light and spatter noises. First, a structured light vision sensor is developed to acquire welding image. Second, the ECO algorithm is adopted to track the seam region and acquire seam feature point during gas metal arc welding process. Third, the state and measurement equations of the weld seam position are established, and PF is applied to improve seam feature point acquisition accuracy. Finally, a welding experiment system is built and a series of seam feature point acquisition experiments of butt joint, lap joint, and fillet joint are carried out to validate the performance of the proposed method. The experiment results demonstrate that the processing speed of the proposed method could reach up 35 Hz, and the seam feature point acquisition errors are smaller than 0.15 mm, which could meet the real-time and accuracy requirement for subsequent initial point guiding and seam tracking.

Recently, laser vision sensors are widely applied in initial point alignment and seam tracking to improve the level of intelligent welding because of good characteristics. However, since the deformation of laser stripe is unobvious at the narrow weld with 0.2 mm width, these methods are not applicable for the narrow weld. Moreover, there are rare researches that could achieve initial point alignment and seam tracking of narrow weld simultaneously. Therefore, an initial point alignment and seam tracking system for narrow weld is proposed in this paper. At first, a laser vision sensor with extra light emitting diode light is used to obtain laser and weld seam image. Besides, the seam feature point is extracted and three-dimensional coordinates can be obtained with vision model. In addition, three controllers including decision controller, initial point alignment controller, and seam-tracking controller are proposed to achieve initial point alignment and seam tracking control in X- and Z-axis directions. Moreover, feature verification, Kalman filter, and output pulse verification are designed to improve the accuracy and stability of this system. Finally, many initial point alignment and seam-tracking experiments of narrow weld are conducted. Experimental results demonstrate that proposed system can well achieve initial point alignment and seam tracking of planar and curved surface narrow weld.

In this correspondence paper, an image dynamics-based visual servoing for quadrotors is proposed to realize stable hovering and tracking. Four perspective image moments are adopted as visual features to control all the independent degrees of freedom of a quadrotor. The complicated interaction matrix is simplified by projecting original image to virtual image plane. On this basis, the dynamics of the system is determined by considering the dynamics of image features and the quadrotor simultaneously. Backstepping controllers are then designed to stabilize the visual servoing system of the quadrotor. In reality, it is unrealistic to have exact prior knowledge about the trajectory parameters of an unpredictable moving target. To solve this problem, a trajectory observer based on nonlinear tracking-differentiator to estimate trajectory parameters of the target is firstly integrated into the quadrotor with image dynamics, which guarantees a satisfactory performance. The effectiveness of the proposed approach is verified by simulations.

In this paper, we design and build a power line inspection robot capable of hybrid operation modes. Specifically, the developed robot is able to land on the overhead ground wire (OGW) and to move as the climbing robot. When to negotiate obstacles, it can vertically take off the wire and fly over the obstacles as the unmanned aerial vehicle (UAV). A customized trumpet-shaped undercarriage is used to guarantee that the robot can land and move safely. With the aid of a swingable 2D Laser Range Finder (LRF), the robot can not only determine whether there are obstacles but also detect the position and orientation of the OGW, making it suitable for automatic inspection of power lines. The outdoor experimental results1demonstrate the effectiveness of the robot in landing and obstacle negotiation. In addition, the average power consumption of the robot is much lower than that of traditional flying robots for power line inspection.

Ampoule injection is a routinely used treatment in hospitals due to its rapid effect after intravenous injection. During manufacturing, tiny foreign particles can be present in the ampoule injection. Therefore, strict inspection must be performed before ampoule injections can be sold for hospital use. In the quality control inspection process, most ampoule enterprises still rely on manual inspection which suffers from inherent inconsistency and unreliability. This paper reports an automated system for inspecting foreign particles within ampoule injections. A custom-designed hardware platform is applied for ampoule transportation, particle agitation, and image capturing and analysis. Constructed trajectories of moving objects within liquid are proposed for use to differentiate foreign particles from air bubbles and random noise. To accurately classify foreign particles, multiple features including particle area, mean gray value, geometric invariant moments, and wavelet packet energy spectrum are used in supervised learning to generate feature vectors. The results show that the proposed algorithm is effective in classifying foreign particles and reducing false positive rates. The automated inspection system inspects over 150 ampoule injections per minute (versus ~ 12 ampoule injections per minute by technologist) with higher accuracy and repeatability. In addition, the automated system is capable of diagnosing impurity types while existing inspection systems are not able to classify detected particles.

This correspondence paper proposes a fast orientation estimation approach of natural images without the help of semantic information. Different from traditional low-level features, our low-level features are extracted inspired by the biological simple cells of the visual cortex. Two approximated receptive fields to mimic the biological cells are presented, and a local rotation operator is introduced to determine the optimal output and local orientation corresponding to an image position, which serve as the low-level feature employed in this paper. To generate the low-level features, a bisection method is applied to the first derivative of the model of receptive fields. Moreover, the feature screener is introduced to eliminate too much useless low-level features, which will speed up the processing time. After all the valuable low-level features are combined, the overall image orientation is estimated. The proposed approach possesses several features suitable for real-time applications. First, it avoids the tedious training procedure of some conventional methods. Second, no specific reference such as the horizon is assumed and no a priori knowledge of image is required. The proposed approach achieves a real-time orientation estimation of natural images using only low-level features with a satisfactory resolution. The effectiveness of our proposed approach is verified on real images with complex scenes and strong noises.

This paper proposes a novel general framework for line segment perception, which is motivated by a biological visual cortex, and requires no parameter tuning. In this framework, we design a model to approximate receptive fields of simple cells. More importantly, the structure of biological orientation columns is imitated by organizing artificial complex and hypercomplex cells with the same orientation into independent arrays. Besides, an interaction mechanism is implemented by a set of self-organization rules. Enlightened by the visual topological theory, the outputs of these artificial cells are integrated to generate line segments that can describe nonlocal structural information of images. Each line segment is evaluated quantitatively by its significance. The computation complexity is also analyzed. The proposed method is tested and compared to state-of-the-art algorithms on real images with complex scenes and strong noises. The experiments demonstrate that our method outperforms the existing methods in the balance between conciseness and completeness.

Nanomanipulation under scanning electron microscopy (SEM) has been demonstrated as an enabling technique for the manipulation and characterization of nanomaterials. We recently developed nanomanipulation techniques for the extraction and identification of DNA contained within sub-nuclear locations of a single cell nucleus. In nanomanipulation of DNA, a key step is target identification through SEM-fluorescence correlative imaging. Existing image correlation techniques often require fiducial marks and/or manual feature selection or data training, which are unsuitable for DNA nanomanipulation. This paper presents an approach for correlating SEM-fluorescence microscopy images, proven effective in processing images taken under poor SEM imaging conditions imposed by the necessity of preserving DNA's biochemical integrity. The performance of the image correlation approach under different imaging conditions was quantitatively evaluated. Compared to manual correlation by skilled operators, the automated correlation approach demonstrated an order of magnitude higher speed. The SEM-fluorescence correlation approach enables targeted nanomanipulation of sub-cellular structures under SEM.

This paper proposes a kinematic modeling method for a bio-inspired robotic fish based on single joint. Lagrangian function of freely swimming robotic fish is built based on a simplified geometric model. In order to build the kinematic model, the fluid force acting on the robotic fish is divided into three parts: the pressure on links, the approach stream pressure and the frictional force. By solving Lagrange's equation of the second kind and the fluid force, the movement of robotic fish is obtained. The robotic fish's motion, such as propelling and turning are simulated, and experiments are taken to verify the model.

The swimming backward method for biomimetic carangiform robotic fish is analyzed in this paper based on the dynamic/kinematic model. The equation of Lagrange of multi-link carangiform robotic fish and simplified fluid force are inducted to calculate the dynamic and kinematic characteristics of the motions. A specific gait is calculated to make the profile of the carangiform robotic fishpsilas undulation fit the characteristics of European eelpsilas swimming backward, which is summarized from the motion sequence of European eel. The simulated and experimental data is given to verify the method.

In this paper, a method for the posture control of a biomimetic robot fish TPF-I is proposed. In this method, the position of the robot fish's gravity centre can be changed by a barycenter-adjustor, which leads to the pitching angle changing. Propelled by coordinating a multi-link body and a tail, the robot fish can complete the posture control and 3-D Locomotion. The 3-D locomotion and posture control are implemented by synthesizing three basic control methods speed control, orientation control and pitching control, which are described in detail respectively. Finally, the experimental results of the robot fish's motion control are given and the performance is analyzed