Underwater humanoid robots hold great potential for complex marine tasks thanks to their dexterous and versatile hands. However, their perception capabilities are severely hindered in turbid and low-light environments, where vision-only sensing becomes unreliable. To address this challenge, we present HydroPalm, the first bionic dual-modal visual-tactile sensor designed for hands of underwater humanoid robots. HydroPalm integrates a wide-field binocular vision module with a high-resolution soft tactile interface. Specifically, an iterative concentric angular topology sorting (ICATS) algorithm is proposed to resolve marker-matching ambiguity caused by background distortions. A contact-based refractive stereo ray tracing (CRSRT) method is introduced to perform accurate 3D reconstruction in water with variable refractive indices. Experiments across 0-1285.2 NTU demonstrate that HydroPalm improves reconstruction quality by over 230% compared to vision-only baselines, while maintaining a mean absolute error below 5% in waters with varying refractive indices. When deployed on a robotic hand, HydroPalm further enables reliable grasping inside a fully dark and highly turbid underwater cavity. The results suggest a new dual-modal sensing paradigm tailored for underwater humanoid robots, with promising applications in seafood harvesting, delicate ecological sampling, and archaeological excavation.

Multi-robot collision avoidance in complex environments poses a significant challenge, as robots must not only avoid collisions with obstacles but also with other robots. However, most existing methods only use neighbor robot states or raw sensor data as input, which is difficult to deal with scenarios containing both unknown obstacles and neighbor robots. To address this problem, we propose a novel hierarchical perception spatio-temporal network (HPSTN) to capture the spatio-temporal features of the environment from both the sensor level and the agent level. At the agent level, we use the reciprocal velocity obstacle (RVO) to represent the neighbor robot information. At the sensor level, we propose a geometric representation method called lidar obstacle (LO) to describe the lidar-detected occluded sectors, and use it to represent the collision-relevant information of obstacles, so as to achieve a compact and informative observation representation unified with RVO. The robot’s ego state and the above two types of observation information are encoded and fused through spatial and temporal encoders of the HPSTN, enabling the robot to perceive the surroundings more comprehensively and adaptively focus on more important information in complex environments. Extensive simulation experiments with various scenarios demonstrate that our approach outperforms several state-of-the-art methods in terms of safety and efficiency. Furthermore, we also conduct physical experiments with multiple differential-drive robots to validate the effectiveness of our approach in real-world scenarios.

Robust and efficient tracking and mapping are critical for underwater vehicles, but remain challenging due to degraded visual quality, ambiguous features, and limited computational resources. Although recent deep learning-based stereo matching methods have significantly improved geometric perception for robots, most existing approaches struggle to simultaneously achieve high speed and strong generalization. To address these challenges, we propose SAFT, a tracking and mapping framework based on self-supervised, robust, and real-time stereo matching. SAFT introduces three key innovations: 1) SAFT-Stereo, a novel stereo matching network that integrates cost aggregation with iterative optimization to enable efficient disparity estimation in feature-sparse regions; 2) a spatiotemporal self-supervised loss that leverages both spatial and temporal constraints to provide stable training signals in textureless regions; and 3) SAFT-DSOL, a real-time tracking and mapping algorithm that integrates the self-supervised models to achieve robust localization and dense reconstruction. Extensive experiments on both public and custom underwater datasets demonstrate that SAFT-Stereo achieves the best generalization performance among all real-time methods, while requiring only 1/6 of the inference time of RT-IGEV++. Moreover, the proposed SAFT-DSOL enables stable and efficient tracking and achieves real-time dense reconstruction in indoor shipwreck scenarios. The code is available at github.com/c237814486/SAFT-Stereo.

Panoptic 3D reconstruction from a monocular video is a fundamental perceptual task in robotic scene understanding. However, existing efforts suffer from inefficiency in terms of inference speed and accuracy, limiting their practical applicability. We present EPRecon, an efficient real-time panoptic 3D reconstruction framework. Current volumetric-based reconstruction methods usually utilize multi-view depth map fusion to obtain scene depth priors, which is time-consuming and poses challenges to real-time scene reconstruction. To address this issue, we propose a lightweight module to directly estimate scene depth priors in a 3D volume for reconstruction quality improvement by generating occupancy probabilities of all voxels. In addition, compared with existing panoptic segmentation methods, EPRecon extracts panoptic features from both voxel features and corresponding image features, obtaining more detailed and comprehensive instance-level semantic information and achieving more accurate segmentation results. Experimental results on the ScanNet V2dataset demonstrate the superiority of EPRecon over current state-of-the-art methods in terms of both panoptic 3D reconstruction quality and real-time inference. Code is available at https://github.com/zhen6618/EPRecon.

Navigating a mobile robot through crowded environments presents a significant challenge. Despite extensive research, existing methods often fail to ensure consistent effectiveness across different scenarios. To enhance the ability of the navigation policy to be applied to different scenarios, we propose a crowd navigation policy architecture based on state prediction and value estimation, which includes a state predictor based on pedestrian trajectory prediction and a value estimator using a graph convolutional network, which can navigate the robot to the target position safely and efficiently. Additionally, we propose a knowledge distillation-based training framework to distill the state prediction model, thereby improving its adaptability to human motions in different scenarios. Our approach is evaluated in a simulation environment through multi-scenario experiments using both simulated data and real pedestrian datasets. The experimental results demonstrate the superiority of our approach in terms of success rate and navigation efficiency, indicating that our approach can effectively improve the generalization performance of the navigation policy across different scenarios.

As an important branch of embodied artificial intelligence, mobile manipulators are increasingly applied in intelligent services, but their redundant degrees of freedom also limit efficient motion planning in cluttered environments. To address this issue, this paper proposes a hybrid learning and optimization framework for reactive whole-body motion planning of mobile manipulators. We develop the Bayesian distributional soft actor-critic (Bayes-DSAC) algorithm to improve the quality of value estimation and the convergence performance of the learning. Additionally, we use a quadratic programming method to calculate and constrain joint velocities, thereby improving the safety of the whole-body motion planning. We conduct experiments and make comparison with standard benchmark. The experimental results verify that our proposed framework significantly improves the efficiency of reactive whole-body motion planning, reduces the planning time, and improves the success rate of motion planning. Additionally, the proposed reinforcement learning method ensures a rapid learning process in the whole-body planning task. The novel framework allows mobile manipulators to adapt to complex environments more safely and efficiently.

Tactile sensing is indispensable for underwater operations, as it provides critical feedback on contact information and environmental interactions. Although existing sensors, such as optical, piezoelectric devices, have been employed for underwater tactile perception, they exhibit some limitations in multi-physical field contact perception. To address these challenges, this study introduces the ViTac-Gripper, a vision-based tactile sensing underwater gripper designed to enhance underwater perception and grasping capabilities. The Finite Element Method (FEM) is utilized to simulate the deformation and force distribution on the tactile surface, providing a high-fidelity model for contact behavior analysis. A domain-aligned multi-physical information perception network framework is proposed, which effectively bridges the simulation-to-reality gap and enables robust extraction of tactile information, including normal/shear force, contact position, 3D Reconstruction and dense force distribution. Experimental results demonstrate the system’s ability to accurately reconstruct multi-physical contact information, while the adaptive grasping strategy ensures stable and reliable object manipulation. The ViTac-Gripper represents a significant advancement in underwater robotics, offering a cost-effective and versatile solution for underwater manipulation tasks.

For field tasks, it is quite challenged to operate in a complex environment for the underwater robots, especially for those with multiple control surfaces due to different response and gain characteristics. To this end, this paper develops a highly integrated robotic dolphin followed by a robust motion control system. For better maneuverability and fault-tolerant capabilities, a newly-designed robotic dolphin is presented, owning a wide array of sensors and multiple control surfaces, in which passive flukes are particularly applied. On this basis, a robust motion control system is proposed, including a depth controller based on velocity-related allocation strategies and a heading controller based on clearance compensation. In detail, considering the degradation of motion performance caused by passive flukes, a sliding mode controller for gain uncertainty and an allocation-related parameter tuning strategy for inputs response characteristics are designed. Extensive simulations and aquatic experiments are conducted, and the obtained results demonstrate the satisfied maneuverability of the designed prototype and the effectiveness of the proposed methods. This study can lay a foundation for further development of robotic dolphins with a robust motion system to execute complex tasks in the field.Note to Practitioners—This paper is inspired by the issue of robust motion control system for a newly-designed practical robotic dolphin that possesses a passive tail and redundant control surfaces. The traditional methods are usually susceptible to uncertainties in the passive tail gain, exhibiting degraded control performance. Moreover, control oscillations and slow convergence speed often occur caused by neglecting the characteristics of different control surfaces, including response patterns and clearance. This paper suggests a robust depth controller based on velocity-related allocation strategies and a robust heading controller based on clearance compensation. Specifically, an allocation-related parameter tuning strategy is given by considering inputs response characteristics, including response speed, saturations, and hydrodynamic force variation patterns. To guarantee fine regulations of heading control, a nonlinear disturbance observer (NDOB)-based clearance compensation is proposed. Extensive aquatic experiments on the newly-designed robotic dolphin verified the effectiveness of the proposed methods. It is envisioned that all these presented results can provide valuable engineering practice insights for industry practitioners.

Over prolonged evolution, natural fish possess exceptional maneuverability. For underwater robotic fish, maneuverability is a crucial performance metric during locomotion. This paper proposes a turning control optimization framework for a multi-flexible-joint bionic robotic fish. Firstly, three distinct turning strategies are devised based on kinematic analysis of the flexible robotic fish’s turning motion. Experimental results indicate that the control strategies need to be adjusted with respect to motion frequency to achieve optimal performance for the flexible robotic fish. Furthermore, an optimization problem incorporating dynamic constraints and cost functions of the flexible robotic fish is constructed. Based on a Constrained Iterative Linear Quadratic Regulator (CILQR), a turning performance optimization method is designed. Subsequently, optimal control strategies under both stationary and motion states are derived, effectively enhancing the turning performance of the flexible robotic fish. Finally, simulation and experimental results validate the effectiveness of the designed method. The developed flexible robotic fish could achieve a maximum swimming speed of 1.63 BL/s (body lengths per second) and a maximum average turning speed of 130.5°/s, offering guidance for the practical application of flexible robotic fish in aquatic environments.

The Five Hundred Meter Spherical Radio Telescope (FAST) is the largest spherical radio telescope in the world, and there are more than 2,000 prism targets distributed on its reflector that require regular maintenance. These prism targets are screwed into the corresponding threaded holes by target bolts. At present, manual maintenance is mainly used, which is inefficient and unsafe. To address this issue, we develop an autonomous prism target maintenance robotic system, called APTMRS. This system integrates a mobile robot with an assembly robot and a collaborative robot, enabling it to move and perform the prism target replacement task on the reflector. Its technical development is threefold. First, a robotic end-effector that can be used for screwing the non-standard target bolt from the side is designed. Second, a novel pose measurement framework, which incorporates a feature point extraction and matching method, is utilized to measure the pose of the prism target. Third, the manipulation policies for the challenging steps in the workflow are presented, including picking and placing, initial thread mating, and final tightening. Independent experiments indicate that the performances of these methods are satisfactory. Moreover, tests in both simulated and real FAST scenarios confirm the effectiveness of APTMRS in achieving autonomous prism target maintenance.Note to Practitioners—Due to its excellent versatility, threaded connections have been widely adopted in a multitude of scenarios. Threads are not only present on standard bolts and nuts but also on a myriad of non-standard components. The robotic system proposed in this paper offers a solution for assembling and disassembling target bolts for FAST. Specifically, the scenario applied in this paper covers several extremely challenging problems in autonomous threaded assembly. Firstly, the object of assembly is a non-standard bolt, with obstacles above it, rendering traditional tools inapplicable. Secondly, the surface of the object is weakly textured and highly reflective. These pose significant challenges for the vision system to accurately acquire its pose. Lastly, the assembly environment is harsh. The threaded holes reside on an incline that is hard to access, with a maximum slope of$56^\circ$. To address these challenges, this paper presents a robotic system supplemented by three techniques including a novel robotic end-effector, a new pose measurement framework, and manipulation policies. The experimental results indicate that this robotic system is capable of autonomously disassembling and assembling non-standard target bolts. We explore the potential for generalizing these designs and methods to other scenarios, with the hope of contributing new insights to the development of threaded assembly systems.

Directly learning the swimming behaviors of real fish can significantly enhance the swimming performance of bionic robotic fish. This paper presents a novel transfer learning method based on a dynamic trajectory control approach for the robotic fish to learn swimming skills from real fish. First, we develop a fish motion capture system and a crucial motion extraction approach to realize precise decomposition of fish motions and collect abundant meaningful features from a snakehead fish as pre-training data. Next, we construct a two-stage transfer learning method based on Deep Deterministic Policy Gradient (DDPG), including an offline and an online stage. Specifically, in the offline stage, the obtained pre-training data is processed for experience learning within a DDPG-based network, whereas in the online stage, a dynamic trajectory tracking method is utilized to refine the robotic fish’s motions in real time based on the learned strategies. Experimental results on a self-developed four-joint robotic fish show that the proposed method effectively extracts and transfers biological motion features into the motion control of the robotic fish. Compared to the conventional CPG method, the proposed approach exhibits stronger acceleration capabilities and more efficient swimming, resulting in enhanced maneuverability of the robotic fish. Overall, this approach provides a technical foundation for bionic robotics to learn from nature.

Compared with traditional underwater vehicles, robotic fish have been receiving increasing attention in recent years due to their excellent maneuverability. However, the characteristics of fishlike undulatory motions and complex underwater working environment have posed significant challenges to robotic fish speed measurement, limiting their autonomy. To overcome these challenges, an artificial lateral line sensor (ALLS) was developed, drawing inspiration from the tactile system of fish. It captured the real-time speed of robotic fish through assessing the deformation of the stressed component under laminar flow impact. To mitigate turbulence disturbances near the ALLS, three flow control components, fairing, flow conditioner, and flow collector, were proposed to attenuate turbulence noise under the viscous effect. Furthermore, a physics-informed calibration method was presented to establish the nonlinear model of ALLS. Specifically, a physical model embedding algorithm based on data resampling was used to mitigate the risk of overfitting by the multilayer perceptron, considering the influence of turbulence disturbance and fishlike undulatory noise. Compared with the classical calibration method based on physical model fitting, the calibration method proposed in this paper reduced the error by 36.0%. Our ALLS’s final mean absolute error was 0.016 m/s with a linearity (R2) of 0.956. The experimental results indicated that the significant changes in the motion state of robotic fish reduced the accuracy of ALLS. The fusion with other sensors is expected to enhance the robustness of ALLS in the future. Note to Practitioners—The motivation of this paper is to design an artificial lateral line sensor based on surface flow field detection and turbulence noise suppression, providing a small-sized and high-precision solution to the speed measurement problem of bionic robotic fish. Most existing ALLS research focused on developing new types of sensors based on different measurement principles, without suppressing the noise caused by fishlike motions, and most experiments were conducted in environments with excessive controls rather than free-swimming robotic fish. To this end, we developed an ALLS based on deformation measurement and proposed three flow control components to make the measured flow more stable. Furthermore, a physics-informed overfitting suppression method was used for the calibration task of the ALLS. A series of simulations and experiments demonstrated that the proposed turbulence noise suppression and calibration method were practical and effective. Hopefully, our methods can provide theoretical and technical guidance to marine engineers for underwater vehicle speed measurement and flow sensing. The recommended flow control component is applicable for conditioning surface fluids in pneumatic control systems. Furthermore, the proposed biomimetic tactile sensor is poised to inspire tactile-based human-machine interaction methods.

Underwater vehicle-manipulator systems (UVMSs) play crucial roles in the fields of underwater target monitoring and pipeline maintenance. However, achieving accurate tracking for underwater pipelines is challenging due to the complexity of UVMSs in terms of nonlinearity, strong coupling and underactuation. To solve the aforementioned problems, an underwater biomimetic vehicle-manipulator system (UBVMS) and an underwater pipeline tracking control method based on the robot vision are proposed. The UBVMS is equipped with the biomimetic undulatory fin propulsors and the biomimetic flipper propulsors, which are inspired by the median and/or paired fin propulsion mode and the body and/or caudal fin propulsion mode of fishes, respectively. The biomimetic undulatory fin propulsors provide the UBVMS with advantages of maneuverability and stability, while the biomimetic flipper propulsors enable the UBVMS to have improved acceleration ability. A tracking control algorithm with adaptive weight coefficients is designed to improve the pose stability of the UBVMS. A fuzzy rule mapping model is constructed to describe the nonlinear relationship between the biomimetic propulsors' control parameters and the propulsive force/torque. Finally, four types of pipeline tracking experiments are conducted to verify the effectiveness and feasibility of the proposed UBVMS and control algorithm.

In this article, we propose a tightly coupled visual-inertial-acoustic sensor fusion method to improve the autonomous localization accuracy of underwater vehicles. To address the performance degradation encountered by existing visual or visual-inertial simultaneous localization and mapping systems when applied in underwater environments, we integrate the Doppler velocity log (DVL), an acoustic velocity sensor, to provide additional motion information. To fully leverage the complementary characteristics among visual, inertial, and acoustic sensors, we perform multimodal information fusion in both frontend tracking and backend mapping processes. Specifically, in the frontend tracking process, we first predict the vehicle's pose using the angular velocity measurements from the gyroscope and linear velocity measurements from the DVL. Thereafter, measurements performed by the three sensors between adjacent camera frames are utilized to construct visual reprojection error, inertial error, and DVL displacement error, which are jointly minimized to obtain a more accurate pose estimation at the current frame. In the backend mapping process, we utilize gyroscope and DVL measurements to construct relative pose change residuals between keyframes, which are minimized together with visual and inertial residuals to further refine the poses of the keyframes within the local map. Experimental results on both simulated and real-world underwater datasets demonstrate that the proposed fusion method improves the localization accuracy by more than 30% compared to the current state-of-the-art ORB-SLAM3 stereo-inertial method, validating the potential of the proposed method in practical underwater applications.

In nature, fish have evolved sophisticated muscular systems that enable them to dynamically regulate their body movements for efficient and agile swimming, which has inspired the development of compact and fast flexibility regulation mechanisms in robotic fish. While existing robotic fish have primarily relied on passive flexible mechanisms and tunable stiffness mechanisms, these approaches often lack the dynamic adjustment capabilities that are characteristic of living fish. This article proposes a novel biomimetic flexible fishtail capable of dynamically controlling its deformation through artificial muscles made from macrofiber composite. In detail, the fishtail is equipped with a servo motor as the sole driving joint, while the artificial muscles regulate the deformation to indirectly adjust stiffness. A dynamic model considering both flexibility and hydrodynamics is established, and a partial differential equation observer is particularly developed to estimate the tail's full states. Subsequently, a deformation control framework incorporating a deep reinforcement learning strategy is constructed and successfully deployed on an embedded platform via lightweight design. Simulation and experimental results validate the accuracy and effectiveness of the dynamic model, observer, and control strategy. Especially, the proposed fishtail demonstrates the ability to enhance propulsion in fishlike swimming modes across various frequencies, ranging from 15% to 203%. When assembled into an untethered robotic prototype, deformation control allows the prototype's swimming speed to vary, achieving up to 42% slower or 37% faster speeds compared to passive compliance. Its rapid adjustability and adaptability to different frequencies represent significant advancements not widely reported in previous studies. The obtained results will offer some significant insights for flexible robotic systems to enhance their agility and interactivity.

The complex underwater environment presents numerous challenges for the design of soft grippers, which often suffer from limited load capacity, poor stability, low portability, and imprecise control. This paper proposes a novel rigid-soft hybrid gripper specifically designed for underwater use. The gripper's finger is constructed from silicone, reinforced with a multi-link rigid exoskeleton on the outside, and actuated by tendons. This design provides three key advantages: compliance (capable of handling fragile objects such as a piece of tofu), heavy lifting (demonstrated by lifting an 80 kg barbell with three fingers), and precise, stable operation (the hybrid gripper maintains its shape despite water flow disturbances). Additionally, the gripper is compact and lightweight, with the driving system powered by just four 23g servo motors, making it easy to mount on various underwater robots. To enable precise control, both specialized kinematic and mechanics models were developed, allowing accurate predictions of the relationships among tendon displacement, exoskeleton deformation, soft material deformation, and tendon tension. This study thoroughly considers the challenges of underwater environments, offering new insights for advancing the field of underwater soft grasping.

Underwater collision-free navigation and dense reconstruction are essential for marine refined exploration. However, existing passive vision-based methods are difficult to apply in low-light and weak-feature underwater environments. In this article, a more adaptable three-dimensional (3-D) dense mapping robotic system based on self-designed scanning binocular structured light (BSL), named ROV-Scanner, is developed to address this challenge. First, the measurement principle based on the refraction model ensures its high accuracy. Second, an underwater 3-D dense mapping algorithm fusing the Doppler velocity log (DVL), inertial measurement unit (IMU), and pressure sensor multifrequency information is proposed to realize dense mapping during robot motion. Then, an air–water two-stage extrinsic calibration algorithm is proposed. In particular, the extrinsic parameters between DVL and camera are innovatively calibrated using BSL, enhancing robustness. Furthermore, for the first time, a framework of BSL-based collision-free navigation is presented to guarantee the safe movement of the system in unknown environments. Experimental results show that our system can simultaneously achieve autonomous collision-free navigation and dense mapping in dark underwater environments, which has great potential for application in marine refined exploration.

Benefiting from its hyper-redundant structure, the biomimetic snake-like manipulator retains its remarkable flexibility even within confined spaces. However, its motion planning and control pose significant challenges. This paper imitates the winding uncoiling behavior of snakes to achieve controllable constrained path following. Firstly, based on control points, a recursive computational model and an equivalent planning angle model are established, enabling efficient and analytical determination of joint positions, collision regions, and motion parameters during the path following. Subsequently, the sliding control point algorithm and motion smoothing restriction algorithm are designed. The former ensures that the remaining segments during following strictly remain within the collision-free regions defined by the base and path controls, while the latter smooths the control parameters based on velocity and acceleration limitations. Finally, simulation and practical experiments demonstrate the feasibility of the proposed methods. The prototype that applied our method can reach targets and accomplish tasks, further validating the applicability of the snake-like manipulator.

This article addresses an autonomous manipulation problem for an underwater vehicle-manipulator system (UVMS) operating in a free-floating way while subjecting to unknown continuous disturbance. More specifically, a composite control scheme composed of disturbance observer (DOB), predictor model network (PM-Net), and nonlinear model predictive control (NMPC), is devised to improve the control performance of UVMS (i.e., unicycle-like UVMS actuated only in the surge, heave, and yaw for vehicle body) in the case of disturbance, model mismatch, and input saturation. A RBF-DOB is formulated by combining a DOB and a Radial Basis Function (RBF) neural network to estimate disturbance at the current step. Then, the PM-Network, composed of a disturbance predictor network and state predictor network, is developed based on long short-term memory (LSTM) network that predicts UVMS state sequences considering model mismatch and disturbance. The NMPC is deployed as a feedback control law to endow the input saturation of the UVMS and produce optimal control action. Compared with conventional DOB control methods using feed-forward compensation of disturbance, the primary merit of the proposed approach is that the disturbance estimated by RBF-DOB is utilized in the PM-Net to predict future UVMS state sequences, which are exploited on the NMPC’s receding optimization. Finally, realistic simulation and relevant experiment are conducted to demonstrate the effectiveness of the proposed method. Note to Practitioners—The motivation behind this article is the autonomous manipulation of an underwater vehicle-manipulator system subjected to unknown disturbance. However, it is not always feasible or straightforward to obtain the external disturbance and unmodeled dynamics for designing robust controllers. On the one hand, how to manipulate the disturbance into the designed controller to generate optimal control action rather than by using feed-forward compensation. On the other hand, the control input saturation often occurs in the UVMS control, especially under the disturbance rejection conditions, where it should be considered in the controller design. Currently, the predominant methods for UVMS control lack a control scheme that provides a complete and credible control strategy that takes the aforementioned issues into consideration. Motivated by the above analysis, this study provides a composite control scheme to deal with the dynamic uncertainties, unknown disturbance, and input saturation. The results of realistic simulation and relevant experiments demonstrate the effectiveness of the proposed method. Hopefully, our control method can provide valuable theoretical and technical guidance to practicing marine engineers for controller design.

Due to the complex mechanism and fabrication process of flexible materials, it remains extremely challenging for a flexible robotic fish to achieve fast and efficient locomotion. In this article, taking advantage of the passive bending and energy storage properties of flexible materials, we propose an untethered robotic fish with multiple flexible joints to achieve high performance and low Cost of Transport (COT). First, combining rigid links and flexible materials, a compact flexible tail with a simple and efficient structure is proposed. Next, the pseudo-rigid body theory is applied to analyze the deformation of passive joints, and a full-state dynamic model is established. More importantly, an optimization method by adjusting the phase differences of the passive joints is used to obtain high aquatic performance. Finally, extensive simulations and experiments validate the effectiveness of the proposed method, and the robotic fish can achieve a maximum speed of 1.63 body length (BL) per second and a minimum COT of 4.8 J/m (2.87 J/m$\cdot$kg). Compared with the multi-joint robotic fish with a similar design, the COT is reduced by up to 81.05% with the basically same aquatic ability. Excitingly, the flexible robotic fish can achieve a COT of 7.36 J/m at 1.23 BL/s, which is 15.72%-36.34% lower than that of the bluefin tuna and is within the range of yellowfin tuna, offering valuable insight into high speed and long endurance applications for underwater robots.Note to Practitioners–This paper is motivated by the design and optimization of an efficient bionic flexible underwater robot with high aquatic abilities, which is conducive to aquatic tasks that require long-time and long-distance sailing, such as underwater topographic exploration, submarine archaeology, and underwater search and rescue. Existing studies of free-swimming bionic underwater robots usually focus on the improvement of swimming speed and rarely consider achieving both high swimming performance and low energy cost. Thus, this paper proposes a bionic underwater robot design with two joints made of flexible materials on the tail to address the problem. Based on detailed analyses of the hydrodynamic force and flexible joint deformation, we propose an effective optimization method for swimming performance. A series of simulations and experiments suggest that the mechatronic design and optimization method are practical and valid. Hopefully, our design and method can provide theoretical guidance for engineers to design and optimize robots with flexible joints.

This paper addresses a learning-based path following control scheme for a biomimetic underwater vehicle (BUV) driven by undulatory fins. A dynamic line-of-sight (DLOS) guidance system is designed, which uses a virtual ball with a dynamic radius to detect the reference path. This DLOS system guides our BUV in the path following control and extracts essential information for the Markov decision process (MDP) of the control task. A deep reinforcement learning (DRL) algorithm, sample-observed soft actor-critic (SOSAC) is proposed. The can train out control policy with greater cumulative reward and higher success rate by using two tricks: sample observation and sample diversification. Based on the DLOS system, the MDP of the control task, and a multilayer perceptron (MLP) trained by the SOSAC, our control scheme is established. Experiments show that our BUV can successfully achieve path following control in an indoor pool environment by using this control scheme.Note to Practitioners—The motivation of this paper is to design a practical end-to-end path following control scheme for the BUV driven by undulatory fins, and verify this scheme in a real-world environment. Unlike common autonomous underwater vehicles (AUVs) using axial propellers, the BUVs apply biomimetic propellers such as the undulatory fin. Multimodel wave patterns can be implemented by the undulatory fin, which generates nonlinear thrust and lateral force simultaneously. This propulsive feature makes the driving force on different directions of the BUV to be strong coupled, and it is complicated to convert the outputs of a common controller into waveform parameters of the undulatory fins to control the BUV. Therefore, in this paper, we proposed an end-to-end learning-based path following controller, which observes environmental information and directly generates waveform parameters to control our BUV. Experiments suggest that our control scheme is practical and valid.

Artificial lateral line (ALL) sensors hold the potential to enhance the perception abilities of robotic fish by capturing surface pressure gradients and identifying near-field object, such as dipole source. However, the robotic fish’s free-swimming motion introduces periodic low-frequency noise into the ALL data, while dipole sources with time-varying positions generate pressure signals with complex time-frequency characteristics. This paper proposes a complete solution to these challenges that would enable freely swimming robotic fish to locate dipole source. Firstly, an ALL system consisting of pressure sensors is integrated into the robotic fish, further constructing a real-time data acquisition and processing system. Secondly, to effectively estimate and remove the swimming-induced noise from the ALL data, a noise estimation model is developed based on the bionic motion mode and unsteady Bernoulli equation. Subsequently, short-time Fourier transform is applied to the high-quality data after noise elimination, followed by developing a convolution regression neural network for feature extraction and dipole source localization. Finally, extensive simulations and experiments are conducted to validate the effectiveness of the proposed methods and perform the positive impact of the noise estimation model. Remarkably, within the range of perception, the average accuracy of dipole source location can reach 13.6 mm, providing a promising reference for improving the perception abilities of underwater robots.Note to Practitioners—This paper is motivated by the problem of blind zones in near-field perception of underwater robots. The existing perception methods as visual sensing are limited by the dark and cloudy underwater environment, and are powerless in near-field localization. In addition, the artificial lateral line, as a potential near-field sensor, is challenging to be applied in self-propelled robots due to the swimming noise. This paper proposes an integrated near-field sensory system that includes an ALL sensor, a swimming noise elimination method and a dipole source localization method. Specifically, a fish-inspired ALL sensor is designed by high-accuracy pressure sensors and integrated into the robotic fish. To enhance localization performance, a swimming noise elimination model is constructed based on unsteady Bernoulli equation. Furthermore, a convolution regression network is developed for accurate localization of near-field objects. A series of simulations and experiments demonstrate the effectiveness and superiority of the proposed near-field sensory system. Hopefully, our proposed methods can provide valuable guidance and support for near-field object localization to improve the intelligent operation ability of underwater bionic robots, such as cooperative control, underwater navigation, environment exploration, and so forth.

The application potential of robotic fish embedded with intelligent visual navigation algorithms in underwater autonomous operation is on full display recently. However, the existing visual navigation methods are limited by the underwater visual conditions and the motion characteristics of robotic fish. To this end, this paper proposes a novel autonomous navigation framework integrated with visual stabilization control. In practice, a stereo vision-based navigation network is proposed to generate the guidance law. On this basis, a biomimetic robotic hammerhead shark with a controllable cephalofoil is developed, and a nonlinear model predictive controller for cephalofoil stabilization relying on the dynamic model is elaborately designed. Extensive simulations and underwater experiments are conducted to validate the effectiveness and superiority of the proposed methods, which significantly enhance exploration efficiency and reduce image jitter by 26.02% compared to the traditional methods. The obtained results provide a new idea for underwater robots to autonomously explore the ocean.Note to Practitioners—This paper is motivated by the problem of vision-based underwater autonomous navigation for a biomimetic robotic fish that possesses underwater visual stability and good maneuverability. The existing visual navigation networks usually generate unexpected navigation instructions when dealing with complex or ambiguous underwater scenes. Additionally, image jitter caused by the rhythmic motion of robotic fish can lead to navigation failure. This paper suggests an integrated navigation framework that includes a biomimetic platform design, a visual stabilization controller, and an intelligent underwater navigation network. Specifically, a novel sphyrnidae-inspired robotic shark is designed as a new platform with superior motion performance. To enhance underwater visual stability, a nonlinear model predictive control-based visual stabilization controller is proposed. Furthermore, a deep stereo attention navigation network based on a parallax attention mechanism is proposed to improve the generalization of vision-based autonomous navigation. A series of underwater search experiments on the robotic shark demonstrate the effectiveness and superiority of the proposed navigation framework. Hopefully, our proposed methods can provide valuable guidance and support for universal underwater robot navigation to accomplish practical marine tasks, such as underwater rescue, resource exploitation, biological observation, and so on.

The broad and powerful pectoral fins of manta rays are crucial to their efficient and maneuverable swimming. However, very little is currently known about the pectoral-fin-driven 3-D locomotion of manta-inspired robots. This study is focused on the development and 3-D path-following control of an agile robotic manta. First, a novel robotic manta with 3-D mobility is constructed, of which the distinctive pectoral fins provide the only propulsion. Specifically, the unique pitching mechanism is detailed in which the time-coupled coordination movement of the pectoral fins is applied. Second, based on a 6-axis force measuring platform, the propulsion characteristics of the flexible pectoral fins are analyzed. Then, the force-data-driven 3-D dynamic model is further established. Third, a control scheme combined with a line-of-sight (LOS) guidance system and a sliding-mode fuzzy controller is conceived, addressing the 3-D path-following task. Finally, various simulated and aquatic experiments are conducted, demonstrating the superior performance of our prototype and the effectiveness of the proposed path-following scheme. This study will hopefully generate fresh insights into the updated design and control of agile bioinspired robots performing underwater tasks in dynamic environments.

Cable-driven hyper-redundant manipulators (CDHRMs) enable unique tasks in confined spaces while presenting challenges for collision-free path planning. This article introduces a novel planning method called the stepwise follow-the-leader (SFTL) algorithm. SFTL consists of a local planner and a global planner. The local planner utilizes reinforcement learning to obtain a collision-free policy, optimizing target error, path length, angle fluctuation, and TE. The global planner incorporates an observation tree and reachability estimator to dynamically optimize extended path nodes for the local planner. The proposed sliding control points algorithm enables sequential movement along the planned path. The SFTL algorithm is applied to a coiled CDHRM and validated through simulations and practical experiments. Results show an average success rate of over 96% in various scenes, maintaining appropriate joint angles and cable tension. SFTL generates sensor-informed feasible paths, providing a robust planning framework for industrial CDHRM applications.

Structured light systems are widely used in underwater dense reconstruction due to their excellent accuracy. However, the current related methods mainly focus on fixed positions. The reconstruction performance in motion is insufficient. Therefore, we propose an underwater movable binocular structured light (MBSL) based high-precision dense reconstruction framework, named WaterMBSL, to realize the robot reconstruction while moving. Specifically, an onboard binocular structured light system based on mirror-galvanometer is developed first. Then, a simplified underwater point cloud acquisition algorithm is presented to quickly obtain 3-D information of the scene. Besides, a new underwater motion compensation algorithm combining inertial measurement unit and uniform velocity model is proposed. Moreover, the generalized-ICP point cloud registration algorithm is introduced to achieve accurate motion estimation. Finally, an underwater movable reconstruction platform is developed by integrating the self-designed structured light system with the underwater robot BlueROV for validating the performance of our proposed Water-MBSL. Experimental results show that satisfactory motion reconstruction performance can be obtained.

Video-based referring expression comprehension is a challenging task that requires locating the referred object in each video frame of a given video. While many existing approaches treat this task as an object-tracking problem, their performance is heavily reliant on the quality of the tracking templates. Furthermore, when there is not enough annotation data to assist in template selection, the tracking may fail. Other approaches are based on object detection, but they often use only one adjacent frame of the key frame for feature learning, which limits their ability to establish the relationship between different frames. In addition, improving the fusion of features from multiple frames and referring expressions to effectively locate the referents remains an open problem. To address these issues, we propose a novel approach called the Multi-Stage Image-Language Cross-Generative Fusion Network (MILCGF-Net), which is based on one-stage object detection. Our approach includes a Frame Dense Feature Aggregation module for dense feature learning of adjacent time sequences. Additionally, we propose an Image-Language Cross-Generative Fusion module as the main body of multi-stage learning to generate cross-modal features by calculating the similarity between video and expression, and then refining and fusing the generated features. To further enhance the cross-modal feature generation capability of our model, we introduce a consistency loss that constrains the image-language similarity and language-image similarity matrices during feature generation. We evaluate our proposed approach on three public datasets and demonstrate its effectiveness through comprehensive experimental results.

For few-shot object detection, this work proposes a binary similarity detector (BSDet), which realizes a novel similarity-based multiple binary classification and enhances the feature margin between positive and hard negative samples. First, we revisit the classification paradigm, concluding that multiple binary classification paradigm is more suitable than multi-class classification paradigm for the few-shot task. Hence, we propose a binary similarity head (BSH) by posing the classification task as multiple binary similarity measurements rather than a multi-class prediction. Second, focusing on the hard negative samples, we propose a feature enhancement module (FEM). During training phase, the FEM can push the features of positive and hard negative samples far away from each other, and thus effectively suppresses false positives. Abundant experiments and visualizations indicate that our method achieves state-of-the-art performances on few-shot object detection tasks.

As a key technology for autonomous underwater operations, precise tracking control in tight space environments is a great challenge. With the aid of high maneuverability of the underwater bionic robot, this article proposes an integrated tracking control framework for a robotic dolphin to move through narrow areas, including top-level planning, middle-level tracking, and bottom-level control allocation. First, a nonlinear model predictive control-based planning method is presented with full consideration of tracking accuracy and obstacle avoidance safety. Second, in order to improve the anti-interference ability, we derive a nonlinear path tracking control law by combining the backstepping technique with a nonlinear disturbance observer. More importantly, through hydrodynamic analysis of the bionic multimodal motions under flippers and flukes, a fuzzy-based nonlinear control allocation system is particularly adopted to convert calculated control forces into bionic motion parameters. Finally, extensive simulations and aquatic experiments are conducted, and the obtained results validate the effectiveness of proposed methods, providing a new idea to further ocean exploration.

The high speed and low energy cost are two conflicting objectives in the motion optimization of bio-inspired underwater robots, but playing a very important role. To this end, this paper proposes an optimization strategy for swimming speed and power cost using an improved NSGA-II for a flexible robotic fish. A dynamic model involving flexible deformation is established for speed prediction with the hydrodynamic parameters identified. A back propagation (BP) neural network is applied to perform compensation of power cost prediction with the dynamic model's prediction as input. In particular, an NSGA-II-AMS method is developed to improve the efficiency of solving the two-objective optimization problem based on NSGA-II. Finally, extensive simulations and experimental results demonstrate the effectiveness of the proposed optimization strategy, which offers promising prospects for the flexible robotic fish performing aquatic tasks with different performance constraints.

This paper proposes a tightly-coupled visual-Doppler-Velocity-Log (visual-DVL) fusion method for underwater robot localization through integrating the velocity measurements from a DVL into a visual odometry (VO). Considering that employing the DVL measurements in dead-reckoning systems easily leads to error accumulation and suboptimal results in previous works, we directly integrate them into the visual tracking process. Specifically, the velocity measurements are utilized to improve the initial estimation of camera pose during visual tracking, aiming to provide a better initial value for pose optimization. Thereafter, these velocity measurements are also directly employed to constrain the position change of the camera between two adjacent frames by constructing a novel DVL error term, which is optimized jointly with the visual constrains to obtain a more accurate camera pose. Various experiments are carried out in the datasets collected from several scenarios of the underwater simulation environment HoloOcean, and the results illustrate that the proposed fusion method can effectively improve the localization accuracy for underwater robots by about 20% compared to pure visual odometry. The proposed method provides valuable guidance for the accurate localization of underwater robots.

Operating in narrow spaces is an important challenge in the development of robots. Redundant manipulators are one way to solve this problem, but their mechanism design and control method still have much room for improvement. In this paper, we propose a coiled cable-conduit-driven hyper-redundant manipulator (C-CDHRM) with great slenderness and flexibility. In terms of mechanism design, it considers both compactness and operability. By imitating the structure and behavior of a constricting snake, it can be uncoiled sequentially from a coiled storage state, led by the head. In terms of control methods, we propose a multi-layer control system that can make remote operations more accurate and reliable. On the one hand, guiding, segmenting, and following the path overcome the planning ambiguity caused by redundancy. On the other hand, conduit transmission modeling and cable length correction overcome the nonlinear mapping of cable-driven joints and were verified in experiments. Through tests, the mobile integrated system composed of C-CDHRM has an excellent performance in operation precision and accuracy, ensuring safety and accessibility in narrow spaces. Finally, in field experiments, the inspection and cleaning of various types of electrical equipment have been successfully completed, showing excellent application prospects.

This article proposes a novel barrier-based adaptive line-of-sight (ALOS) three-dimensional (3-D) path-following system for an underactuated multijoint robotic fish. The framework of the developed path-following system is established based on a detailed dynamic model, including a barrier-based ALOS guidance strategy, three integrated inner-loop controllers, and a nonlinear disturbance observer (NDOB)-based sideslip angle compensation, which is employed to preserve a reliable tracking under a frequently varying sideslip angle of the robotic fish. First, a barrier-based convergence strategy is proposed to deal with probable along-track error disruption and suppress the error within a manageable range. Meanwhile, an improved adaptive guidance scheme is adopted with an appropriate look-ahead distance. Afterward, a novel NDOB-based sideslip angle compensation is put forward to identify the varying sideslip angle independent of speed estimation. Subsequently, inner-loop controllers are intended for regulation about the controlled references, including a super-twisting sliding-mode control (STSMC)-based speed controller, a global fast terminal sliding-mode control (GFTSMC)-based heading controller, and a GFTSMC-based depth controller. Finally, simulations and experiments with quantitative comparison in 3-D linear and helical path following are presented to verify the effectiveness and robustness of the proposed system. This path-following system provides a solid foundation for future marine autonomous cruising of the underwater multijoint robot.

Dynamic object perception is an important yet challenging direction in the field of robot navigation. Without any prior knowledge about motion and objects, a novel dynamic rigid bodies mining and motion estimation method based on monocular camera is proposed in this article. Different from the existing works based on sampling that associate feature points to motion hypotheses according to the reprojection errors, our work endeavors to find the intrinsic relevance among motion hypotheses. To represent this relevance, the concept of the probabilistic field on the Lie group Sim(3) manifold is introduced, which is established using random sampling. It provides a computable way for the regions on the manifold where rigid bodies possibly appear. The probability of a motion hypothesis falling on a region is expressed by its confidence. The regions with large confidences in the probabilistic field are selected as potential rigid bodies, whose corresponding feature points are further sampled for pose calculation. As a result, the randomness of sampling is reduced and the inliers for possible rigid bodies are enhanced, which guarantees the accuracy of motion estimation. On this basis, the tracking of rigid bodies is achieved. The proposed method distinguishes the feature points of dynamic objects with 3-D motion from those in the static background, thus enabling simultaneous localization and mapping (SLAM) to be initialized in dynamic environments. The experimental results on the KITTI, Hopkins 155, and MTPV62 datasets demonstrate the effectiveness. Comparison experiments indicate that our method outperforms the other methods in sensitivity of dynamic objects perception.

To realize high-quality robotic welding, an efficient and robust complex weld seam feature point extraction method based on a deep neural network (Shuffle-YOLO) is proposed for seam tracking and posture adjustment. The Shuffle-YOLO model can accurately extract the feature points of butt joints, lap joints, and irregular joints, and the model can also work well despite strong arc radiation and spatters. Based on the nearest neighbor algorithm and cubic B-spline curve-fitting algorithm, the position and posture models of the complex spatially curved weld seams are established. The robot welding posture adjustment and high-precision seam tracking of complex spatially curved weld seams are realized. Experiments show that the method proposed in this article can extract weld seam feature points quickly and robustly, which enables welding robots to accurately track the weld seams and adjust the welding torch postures simultaneously.

This article introduces a flexible hand–eye calibration technique for a 3-D sensor from a reconstruction perspective, with no need for a specialized and accurate calibration rig. Our intention is to find the hand–eye relation that simultaneously aligns multiview point clouds of a common scene into the robot base frame, namely simultaneous calibration and reconstruction. To achieve this goal, a novel variant of iterative closest point (ICP) algorithm based on the Gauss–Newton method and Lie algebra is proposed, which iteratively transforms multiview point clouds into the robot base frame, estimates point-to-point correspondences between point clouds then refines the hand–eye relation to minimize the Euclidean distance between corresponding points. In addition to the calibration result, it returns a preliminary reconstruction as a byproduct. Cases of degeneracy and applicable conditions are given and proved. Using arbitrary daily objects with no prior information and a real robotic eye-in-hand system, we verify our method feasible and effective.

Robust vision restoration of underwater images remains a challenge. Owing to the lack of well-matched underwater and in-air images, unsupervised methods based on the cyclic generative adversarial framework have been widely investigated in recent years. However, when using an end-to-end unsupervised approach with only unpaired image data, mode collapse could occur, and the color correction of the restored images is usually poor. In this paper, we propose a data- and physics-driven unsupervised architecture to perform underwater image restoration from unpaired underwater and in-air images. For effective color correction and quality enhancement, an underwater image degeneration model must be explicitly constructed based on the optically unambiguous physics law. Thus, we employ the Jaffe-McGlamery degeneration theory to design a generator and use neural networks to model the process of underwater visual degeneration. Furthermore, we impose physical constraints on the scene depth and degeneration factors for backscattering estimation to avoid the vanishing gradient problem during the training of the hybrid physical-neural model. Experimental results show that the proposed method can be used to perform high-quality restoration of unconstrained underwater images without supervision. On multiple benchmarks, the proposed method outperforms several state-of-the-art supervised and unsupervised approaches. We demonstrate that our method yields encouraging results in real-world applications.

The past decade has seen the rapid development of the robotic fish in many aspects. However, the velocity measurement problem has not been fully addressed, which limits the autonomy of the robotic fish. To this end, an artificial lateral line (ALL) sensor, inspired by the sensory organs of fish, is developed in this article. By measuring the deformation of the sensitive element, the local flow field around the robotic fish is sensed. According to the characteristics of fishlike motions, a fairing structure is proposed to suppress the turbulence noise and yaw motion noise caused by fishlike oscillation of the tail. This structure ensure that the flow measured by the ALL sensor is closer to laminar flow under viscous effects. Furthermore, to measure the magnitude and direction of the robotic fish velocity, an ALL sensor array is assembled by mounting multiple sensors on the robot's surface to sense the flow field distribution. Next, a kinematic-based fusion method is proposed for the array system, which obtained the real-time velocity vector of the robotic fish by solving overdetermined motion equations. The proposed ALL array system is tested on a freely swimming robotic fish, and our method achieves a mean absolute error of 0.018 m/s, a linearity ($R^{2}$) of 0.951, and a position tracking error of 0.085 m. Additionally, the fairing structure is found to improve the signal-to-noise ratio by 116%.

This article presents the design and implementation of an innovative biomimetic underwater vehicle (BUV) and its locomotion controller. Through mimicking a dactylopteridae, the hybrid propulsion BUV is designed with two symmetrical bio-inspired long-fins and a double-joint fishtail. The mechatronic design of the dactylopteridae-inspired BUV with the pectoral long-fins and a double-joint fishtail is first provided. The two flexible long-fins compose the median and/or paired fin (MPF) propulsion, while the fishtail acts as the body and/or caudal fin (BCF) propulsion. Through the coordination of BCF and MPF propulsion modes, the BUV obtains excellent low-speed locomotion stability and also keeps high maneuverability. Moreover, the locomotion control methods based on central pattern generators (CPGs) model and fuzzy adaptive proportion integral differential (PID) are proposed for this BUV. In the end, the experimental results of the multimode motion and closed-loop motion control demonstrate the feasibility and effectiveness of the mechanism and the locomotion control system.Note to Practitioners—The motivation behind this article is the design of a novel biomimetic underwater vehicle (BUV) that possesses low-speed locomotion stability and fast swimming ability, which is suitable for carrying relevant sensors to complete water quality monitoring, biological observation, underwater equipment inspection, underwater structure detection, and other marine tasks. Currently, BUVs are usually designed as only one propulsion mode by caudal fin or paired fins, which makes it difficult to have the advantages of both modes. In order to further study the problem, we designed a dactylopteridae-inspired BUV with the bilateral pectoral long-fins (providing low-speed locomotion stability) and a double-joint fishtail (providing fast swimming ability). A hybrid-driven motion control framework is presented for the BUV based on a central pattern generators (CPGs) model and fuzzy adaptive proportion integral differential (PID). A series of experiments suggests that the mechanism and the locomotion control system are practical and valid. Hopefully, our mechanism and control framework can provide valuable theoretical and technical support guidance to the practicing marine engineer for the codesign of propulsion mode and control.

Category-level 6D object pose estimation has gained popularity and it is still challenging due to the diversity of different instances within the same category. In this paper, a novel category-level 6D object pose estimation framework with structure encoder and reasoning attention is proposed. A structure autoencoder is introduced to mine the shared structure features in the color images within the same category, via a distinct learning strategy that recovers the image of another instance but with the most similar pose to the input. On this basis, a reasoning attention decoder and full connected layers are stacked to form a rotation prediction network, where the structure features and 3D shape features are integrated and projected to a semantic space. The semantic space includes observed patterns and learnable patterns, which are better learned by adding a shortcut connection branch parallel to reasoning attention decoder with gradient decouple. Further reasoning based on these patterns endows the decoder with powerful feature representation. Without 3D object models, the proposed method models the attributes of category implicitly in the semantic space and better performance of 6D object pose estimation is guaranteed by reasoning on this space. The effectiveness of the proposed method is verified by the results on public datasets and actual experiments.

Pavement defect segmentation is a fundamental task in the field of transport infrastructure inspection. Existing methods mainly focus on detection/segmentation for long and thin cracks. However, there are many other types of defects with various sizes and shapes that are also essential to segment, which brings more challenges toward detailed road inspection. To address the above problems and provide a more comprehensive understanding of the overall road conditions, we propose a three-stream neural network that combines spatial, contextual and boundary information for fine-grained defect segmentation. Specifically, the spatial stream captures rich low-level spatial features. The contextual stream utilizes an attention mechanism and models high-level contextual relationships over local features. To further refine the segmentation results, the boundary stream encodes detailed boundaries using a global gated convolution and generates additional boundary maps. By combining the above different information, our model can effectively produce pixel-wise predictions for fine-grained road inspection. The network is trained using a dual-task loss in an end-to-end manner, and experiments were performed on three newly collected datasets, i.e., a fine-grained defect dataset and two crack datasets, which shows that the proposed method achieves favorable segmentation results on complex multi-class defects, and is also able to segment single-class cracks. Specifically, on the fine-grained dataset, it achieved state-of-the-art performance over other competing baselines (mPA of 0.54, mIoU of 0.38, Mic_F of 0.78 and Mac_F of 0.65), where each image is resized to 512$\times 512$and the processing speed is 21 FPS on average.

In this article, the underwater target tracking control problem of a biomimetic underwater vehicle (BUV) is addressed. Since it is difficult to build an effective mathematic model of a BUV due to the uncertainty of hydrodynamics, target tracking control is converted into the Markov decision process and is further achieved via deep reinforcement learning. The system state and reward function of underwater target tracking control are described. Based on the actor-critic reinforcement learning framework, the deep deterministic policy gradient actor-critic algorithm with supervision controller is proposed. The training tricks, including prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, are presented. Indirect supervision training is designed to address the issues of low stability and slow convergence of reinforcement learning in the continuous state and action space. Comparative simulations are performed to show the effectiveness of the training tricks. Finally, the proposed actor-critic reinforcement learning algorithm with supervision controller is applied to the physical BUV. Swimming pool experiments of underwater object tracking of the BUV are conducted in multiple scenarios to verify the effectiveness and robustness of the proposed method.

Biomimetic underwater robots have attracted considerable research attention globally, owing to their quieter actuations, higher propulsion efficiency, and stronger maneuverability when compared with conventional underwater vehicles equipped with axial propellers. This article provides a comprehensive survey of current research in this field. First, we review the development status of biomimetic underwater robots in both body/caudal fin (BCF), median/paired fin (MPF), and their hybrid propulsion modes. Then, we outline the motion control methods employed in biomimetic underwater robots, including open-loop swimming control and typical closed-loop control strategies. In particular, we detail our latest studies on the RobCutt series underwater robots. On this basis, some critical issues and future directions are summarized. We predict that biomimetic underwater robots will have excellent prospects in underwater environment exploration and resource utilization.

This paper proposes a multi-objective visionbased navigation network for biomimetic underwater robots to cope with scientific observation, target selection, and obstacle avoidance in marine missions. Structurally, a stereo block attention module is first constructed to serially extract the channel and spatial attention portion of the real-time visual feedback. Next, the parallax attention mechanism is introduced to enable the network to excavate implicit parallax information in stereo pairs, effectively eliminating the oscillation of the network output in the presence of ambiguous visual input. Further, with the assistance of other low-cost sensors, the proposed navigation network can be expanded in some largescale application scenarios, such as sparse coral observation. Finally, underwater simulations reveal that the proposed method obtains significantly improved control effect and real-time ability, compared with other related works. In particular, based on a self-developed biomimetic robotic dolphin, collision-free simulations with a cumulative distance beyond 1000 m were carried out and validated the effectiveness and the superiority of the navigation network, where both dense and sparse targets were fully tested. The robotic dolphin can not only successfully conduct accurate coral observation without collision, but also quest the observation targets as much as possible in the area where the observation targets are concentrated. The proposed network provides an intelligent and efficient navigation scheme for autonomous underwater operation of small-size underwater robots.

Underwater visual localization is an essential technique for the autonomous operation of underwater robots. However, the unique underwater image characteristics, including refraction, sparse features, and severe noise, pose an enormous challenge to it. For addressing these issues, this paper proposes an open-source fiducial-based underwater stereo visual-inertial localization method under the extended Kalman filter (EKF) framework, which is called FBUS-EKF. First, the refraction is corrected by the refractive camera model and akin triangulation. Second, the fiducial marker and a novel marker pose estimation method are applied to alleviate the adverse effect of sparse features. Third, the EKF is utilized to fuse the inertial and visual information so as to reject the serious noise. Finally, extensive experiments on a test bench demonstrate the effectiveness of the FBUS-EKF method, where the typical localization error is less than 3%, namely, the average error is lower than 3 cm within one meter. The obtained results reveal that the FBUS-EKF method has the prospect to be applied in the precise short-range operation and the localization for underwater robots, which offers a valuable insight for further autonomous underwater task.

Regular pavement inspection plays a significant role in road maintenance for safety assurance. Existing methods mainly address the tasks of crack detection and segmentation that are only tailored for long-thin crack disease. However, there are many other types of diseases with a wider variety of sizes and patterns that are also essential to segment in practice, bringing more challenges towards fine-grained pavement inspection. In this paper, our goal is not only to automatically segment cracks, but also to segment other complex pavement diseases as well as typical landmarks (markings, runway lights, etc.) and commonly seen water/oil stains in a single model. To this end, we propose a three-stream boundary-aware network (TB-Net). It consists of three streams fusing the low-level spatial and the high-level contextual representations as well as the detailed boundary information. Specifically, the spatial stream captures rich spatial features. The context stream, where an attention mechanism is utilized, models the contextual relationships over local features. The boundary stream learns detailed boundaries using a global-gated convolution to further refine the segmentation outputs. The network is trained using a dual-task loss in an end-to-end manner, and experiments on a newly collected fine-grained pavement disease dataset show the effectiveness of our TB-Net.

Seam feature point acquisition is the premise of the intelligent welding process such as initial point guiding and seam tracking. However, conventional seam feature point acquisition methods based on geometric feature have shortcomings of poor flexibility and robustness. In this article, a seam feature point acquisition method based on efficient convolution operator (ECO) and particle filter (PF) is proposed, which could be applied to different weld types and could achieve fast and accurate seam feature point acquisition even under the interference of welding arc light and spatter noises. First, a structured light vision sensor is developed to acquire welding image. Second, the ECO algorithm is adopted to track the seam region and acquire seam feature point during gas metal arc welding process. Third, the state and measurement equations of the weld seam position are established, and PF is applied to improve seam feature point acquisition accuracy. Finally, a welding experiment system is built and a series of seam feature point acquisition experiments of butt joint, lap joint, and fillet joint are carried out to validate the performance of the proposed method. The experiment results demonstrate that the processing speed of the proposed method could reach up 35 Hz, and the seam feature point acquisition errors are smaller than 0.15 mm, which could meet the real-time and accuracy requirement for subsequent initial point guiding and seam tracking.

This article addresses a problem of seabed terrain following control (STFC) for an underwater vehicle-manipulator system (UVMS). The motivation is to perform a visual search of marine products closely to seabed in unknown environment. In terms of this issue, we propose a novel and robust STFC framework for our UVMS to maintain an appropriate height to seabed. A nonlinear model predictive control (NMPC) method is formulated to solve the STFC problem. To relieve online computational burden and system noisy influence, Ohtsuka's continuation/generalized minimal residual (C/GMRES) algorithm incorporated with a tracking differentiator (TD) is investigated. In order to improve the following accuracy, the system state prediction part of the NMPC and a long short-term memory (LSTM) network are elaborated to predict future seabed terrain using a depth gauge and an altimeter, respectively. Finally, the three different physical scenarios for STFC problem are established using ROS to demonstrate the robustness and efficiency of the proposed algorithm.

This article presents a coordinated vehicle-manipulator control method for an underwater biomimetic vehicle-manipulator system (UBVMS) to implement floating autonomous manipulation in practice. An algorithm framework composed of adaptive tracking differentiator (ATD), extended state observer (ESO), improved nonsingular terminal sliding-mode control (I-NTSMC), fuzzy-logic controller (FLC), and estimator of manipulator disturbances, is proposed. The ATD is designed to generate desired motion state and alleviate noise. The ESO is developed to estimate the motion state, systematic uncertainties, and external disturbances. The proposed I-NTSMC method assures the finite-time convergence of the system states and alleviate chattering. The estimation of the manipulator disturbances is incorporated into the control strategy to enhance the station keeping of the vehicle. Finally, underwater autonomous free floating manipulation experiments about opening a door and grasping objects are conducted to validate the theoretical results and confirm the feasibility of the proposed control strategy.

This paper presents a three-dimensional (3-D) path planning method that combines the gliding with dolphin-like motions for the gliding robotic dolphin. A specific task that the robot uses the gliding motion for long-distance cruise and the dolphin-like motion for maneuverable obstacle avoidance is employed. The results of simulations and aquatic experiments validate the full-state dynamic model and the specific task, further offer some theoretical supports for 3-D path planning. Further, the 3-D path planning method is composed of three main components: 1) gliding path generation; 2) improved Astar (A*) algorithm; and 3) segmented Bezier curve smoothing. First, the gliding path is generated autonomously with the kinematic constraints that are obtained via the simulations of dynamic model. Furthermore, when the obstacles are detected by the sonar, an improved A* algorithm is employed to avoid the obstacles. Afterward, considering the path planned by A* is unsmoothed, a segmented Bezier curve method is presented. Simulation results demonstrate the effectiveness of the method, offering valuable insight into the utilization of hybrid underwater robots in the context of real-time task execution.

This article proposes a model predictive control (MPC)-based depth control system for the gliding motion of a gliding robotic dolphin. An injector-based buoyancy-driven mechanism is employed to achieve more precise control of net buoyancy. In the system, a novel framework of depth control is proposed on the basis of a simplified model, including a depth controller with improved MPC, a heading controller with velocity-based proportional-integral-derivative, and a sliding mode observer. Extensive simulation and experimental results demonstrate the effectiveness of the proposed control methods. In particular, a variety of slider-based experiments are also conducted to explore the performance of a movable slider in the depth control so as to better govern the gliding angle. The results obtained reveal that it is feasible to realize regular gliding angles via regulating the slider, which offers promising prospects for bio-inspired gliding robots playing a key role in ocean exploration.

This paper aims to investigate optimal formation solutions of multiple biomimetic underwater vehicles (BUVs). The BUV is propelled by undulatory fins on both sides, and can perform various locomotion patterns, especially turning in situ and diving vertically. Firstly, the optimal formation problem is formulated, followed by theoretical analysis of a special case of optimal line formation. Then, a solution is proposed from the perspective of evolutionary computation. In particularly, the coordinates and the slope of the desired line formation, together with the pairings between initial positions and target positions, are obtained based on particle swarm optimization. Furthermore, we demonstrate the validity of this method by comparing the simulation results with the results of theoretical analysis. Finally, simulations results of multiple BUVs verify the feasibility of the proposed optimal formation methods.

This article presents the comprehensive framework for a hybrid-driven underwater vehicle-manipulator system (HD-UVMS) to grasp marine products on the seabed. The purpose of the proposed hybrid-driven propulsion system is to improve the swimming ability of the HD-UVMS by using thrusters and enhance the stability of its pose adjustment mechanism via two unique long fin propulsors. The control mode for the thrusters and long fin propulsors is based on a fuzzy logic control method. Subsequently, a lightweight manipulator is developed to grasp marine products. The open-closed angle and current controls for the gripper help to avoid damaging marine products. A vision system is installed to enable the HD-UVMS to gradually approach marine products with the aid of monocular vision and grasp them with the aid of binocular vision. A detailed method for monocular passive ranging and stereo matching, in accordance with real-time metrics, is elaborated. Finally, relevant experiments are conducted in an indoor pool and under real sea condition to assess the effectiveness of the proposed framework. Note to Practitioners-The motivation behind this article is the design of an underwater vehicle-manipulator system that can grasp marine products on the real seabed and perform other underwater intervention tasks. Currently, the predominant method of fishing for marine products relies on human divers, which has disadvantages for human divers' health due to the long periods of time spent working underwater. In order to further study the problem, this article develops a hybrid-driven underwater vehicle-manipulator system (HD-UVMS) to work in a real seabed environment. A hybrid-driven motion control framework is presented using the thrusters to achieve effective cruising and searching for marine products and long fin propulsors for the fine pose adjustment required to grasp marine products. The proposed lightweight underwater manipulator can grasp marine products on the seabed with the aid of a vision system. A series of experiments suggests that the HD-UVMS is practical and valid.

Recently, laser vision sensors are widely applied in initial point alignment and seam tracking to improve the level of intelligent welding because of good characteristics. However, since the deformation of laser stripe is unobvious at the narrow weld with 0.2 mm width, these methods are not applicable for the narrow weld. Moreover, there are rare researches that could achieve initial point alignment and seam tracking of narrow weld simultaneously. Therefore, an initial point alignment and seam tracking system for narrow weld is proposed in this paper. At first, a laser vision sensor with extra light emitting diode light is used to obtain laser and weld seam image. Besides, the seam feature point is extracted and three-dimensional coordinates can be obtained with vision model. In addition, three controllers including decision controller, initial point alignment controller, and seam-tracking controller are proposed to achieve initial point alignment and seam tracking control in X- and Z-axis directions. Moreover, feature verification, Kalman filter, and output pulse verification are designed to improve the accuracy and stability of this system. Finally, many initial point alignment and seam-tracking experiments of narrow weld are conducted. Experimental results demonstrate that proposed system can well achieve initial point alignment and seam tracking of planar and curved surface narrow weld.

In this correspondence paper, an image dynamics-based visual servoing for quadrotors is proposed to realize stable hovering and tracking. Four perspective image moments are adopted as visual features to control all the independent degrees of freedom of a quadrotor. The complicated interaction matrix is simplified by projecting original image to virtual image plane. On this basis, the dynamics of the system is determined by considering the dynamics of image features and the quadrotor simultaneously. Backstepping controllers are then designed to stabilize the visual servoing system of the quadrotor. In reality, it is unrealistic to have exact prior knowledge about the trajectory parameters of an unpredictable moving target. To solve this problem, a trajectory observer based on nonlinear tracking-differentiator to estimate trajectory parameters of the target is firstly integrated into the quadrotor with image dynamics, which guarantees a satisfactory performance. The effectiveness of the proposed approach is verified by simulations.

The application of the fast sliding-mode control technique on solving consensus problems of fractional-order multiagent systems is investigated. The design and analysis are based on a combination of the distributed coordination theory and the knowledge of fractional-order dynamics. First, a sliding-mode manifold (surface) vector is defined, and then the fractional-order multiagent system is transformed into an integer-order (namely, first-order) multiagent system. Second, based on the fast sliding-mode control technique, a protocol is proposed for the obtained first-order multiagent system. Third, a new Lyapunov function is presented. By suitably estimating the derivative of the Lyapunov function, the reachability of the sliding-mode manifold is derived. It is proved that the exponential finite-time consensus can be achieved if the communication network has a directed spanning tree. Finally, the effectiveness of the proposed algorithms is demonstrated by some examples.

We develop two cooperative robots for power transmission lines (PTLs) inspection - a light climbing robot (CBR) which can stably move on the overhead ground wire (OGW) for sensor data collection and an unmanned aerial vehicle (UAV) with a grabbing mechanism, which can automatically put the CBR on the OGW and take it off. In order to guarantee the safety, the mechanical structures of the connectors are designed in the shape of a trumpet. Further, a self-locked structure of the CBR is developed to automatically seize and release the OGW. For autonomous navigation, the UAV is equipped with a movable sliding rail and a 2D Laser Range Finder (LRF). The LRF can not only detect the position and orientation of the OGW but also detect the top beam of the CBR and the grabbing position in it. Furthermore, the action of the grabbing mechanism is automatically triggered by a microswitch. Finally, by the developed UAV and CBR platforms, we test the whole loading and unloading strategy in an artificially constructed PTLs environment outdoors and achieve an encouraging result1. Combining the flexible motion of the UAV and the high inspection accuracy of the CBR, the CBR can negotiate any obstacle by flying and abandon the traditional heavy obstacle crossing mechanism to effectively realize close-proximity inspection. Due to the light weight and low power consumption, the CBRs can be deployed once in many power corridors to conduct a long-term inspection.

Most industrial robots are not capable of teaching by hand and require path points to be specified by teaching pendants. To enable the teaching of industrial robots by hand without any force sensors, this paper proposes a scheme to minimize the external force estimation error and reduce disturbance in the guiding task by using the virtual mass and virtual friction model. In this case, the maximum velocity and acceleration of the robot end effector shall be limited to ensure safety. Thus, the operator is allowed to guide the robot by hand. The joint torque is obtained from the motor current. The inertial force and friction of the links and driving systems are analyzed. The nonlinear dynamic model of the industrial robot is built and its parameters are calibrated by a nonlinear method. The force estimation is referenced to set the virtual friction and to design the force-following controller. Hence the end effector can follow the direction of external force compliantly and suppress jitters. Finally, several experiments on a six degrees of freedom industrial robot demonstrate the validity of the proposed control scheme.

This paper addresses the problem of path following for biomimetic underwater vehicles (BUVs) propelled by undulatory ribbon-fins. First, the general kinematics and dynamics models of underwater vehicles are presented, followed by a fuzzy logic model for dealing with a nonlinear relationship between the propulsive force/torque and the control parameters of the undulatory fins of the BUV. Then the path following problem of the BUV is formulated. A path following control paradigm integrating the line-of-sight guidance system with backstepping (BP) technique is proposed to maneuver the BUV to follow a predefined parameterized curve without time constraints. The stability of the BP controller is analyzed and guaranteed by Lyapunov stability theory. Finally, simulations and experimental results illustrate the performance of the proposed path following control paradigm.

This paper proposes a unique and robust UAV autonomous navigation approach along one side of overhead transmission lines for inspection. To this end, we establish a perspective model and develop a novel Pan/Tilt monocular-based navigation scheme. Simultaneously, the following three key issues are addressed. First, to locate the effective landmark - transmission tower timely and reliably, we customize a neural network for tower detection and combine it with a fast and smooth tracking. Second, to provide UAV with a robust and precise heading, we detect the transmission lines and compute and optimize their vanishing point. Third, to keep a safe distance from transmission lines, we optimize a homography matrix to restore the parallel nature of transmission lines and perceive the distance variation by a point set registration model. Finally, by the designed UAV platform, we test the whole system in a real-world transmission-line inspection scenario under different weather condition and achieve an encouraging result. Our approach provides great flexibility for refined inspection and effectively improves inspection safety.

Ampoule injection is a routinely used treatment in hospitals due to its rapid effect after intravenous injection. During manufacturing, tiny foreign particles can be present in the ampoule injection. Therefore, strict inspection must be performed before ampoule injections can be sold for hospital use. In the quality control inspection process, most ampoule enterprises still rely on manual inspection which suffers from inherent inconsistency and unreliability. This paper reports an automated system for inspecting foreign particles within ampoule injections. A custom-designed hardware platform is applied for ampoule transportation, particle agitation, and image capturing and analysis. Constructed trajectories of moving objects within liquid are proposed for use to differentiate foreign particles from air bubbles and random noise. To accurately classify foreign particles, multiple features including particle area, mean gray value, geometric invariant moments, and wavelet packet energy spectrum are used in supervised learning to generate feature vectors. The results show that the proposed algorithm is effective in classifying foreign particles and reducing false positive rates. The automated inspection system inspects over 150 ampoule injections per minute (versus ~ 12 ampoule injections per minute by technologist) with higher accuracy and repeatability. In addition, the automated system is capable of diagnosing impurity types while existing inspection systems are not able to classify detected particles.

This paper studies the containment control of multiagent systems (MASs) with multiple dynamic leaders in both continuous-time domain and discrete-time domain. The leaders' motions are described by the nth-order polynomial trajectories. This setting makes practical sense because given some critical points, the leaders' trajectories are usually planned by the polynomial interpolations. In order to drive all followers into the convex hull spanned by the leaders, a PIn-type containment algorithm is proposed (P and I are short for proportional and integral, respectively; Inimplies that the algorithm includes up to the n-thorder integral terms). It is theoretically proved that the PIn-type containment algorithm is able to solve the containment problem of MASs where the followers are described by any order integral dynamics. Compared to the previous results on the MASs with dynamic leaders, the distinguished features of this paper are that: 1) the containment problem is studied not only in the continuoustime domain but also in the discrete-time domain while most existing results only work in the continuous-time domain; 2) to deal with the leaders with the nth-order polynomial trajectories, existing results require the follower's dynamics to be the (n + 1)th-order integral while the followers considered in this paper can be described by any-order integral dynamics; 3) the “sign” function is not employed in the proposed algorithm, which avoids the chattering phenomenon; and 4) both disturbance and measurement noise are taken into account. Finally, some simulation examples are given to demonstrate the effectiveness of the proposed algorithm.

The optimal formation problem of multirobot systems is solved by a recurrent neural network in this paper. The desired formation is described by the shape theory. This theory can generate a set of feasible formations that share the same relative relation among robots. An optimal formation means that finding one formation from the feasible formation set, which has the minimum distance to the initial formation of the multirobot system. Then, the formation problem is transformed into an optimization problem. In addition, the orientation, scale, and admissible range of the formation can also be considered as the constraints in the optimization problem. Furthermore, if all robots are identical, their positions in the system are exchangeable. Then, each robot does not necessarily move to one specific position in the formation. In this case, the optimal formation problem becomes a combinational optimization problem, whose optimal solution is very hard to obtain. Inspired by the penalty method, this combinational optimization problem can be approximately transformed into a convex optimization problem. Due to the involvement of the Euclidean norm in the distance, the objective function of these optimization problems are nonsmooth. To solve these nonsmooth optimization problems efficiently, a recurrent neural network approach is employed, owing to its parallel computation ability. Finally, some simulations and experiments are given to validate the effectiveness and efficiency of the proposed optimal formation approach.

In this brief, we investigate the parameter optimization issue of a central pattern generator (CPG) network governed forward and backward swimming for a fully untethered, multijoint biomimetic robotic fish. Considering that the CPG parameters are tightly linked to the propulsive performance of the robotic fish, we propose a method for determination of relatively optimized control parameters. Within the framework of evolutionary computation, we use a combination of dynamic model and particle swarm optimization (PSO) algorithm to seek the CPG characteristic parameters for an enhanced performance. The PSO-based optimization scheme is validated with extensive experiments conducted on the actual robotic fish. Noticeably, the optimized results are shown to be superior to previously reported forward and backward swimming speeds.

This correspondence paper proposes a fast orientation estimation approach of natural images without the help of semantic information. Different from traditional low-level features, our low-level features are extracted inspired by the biological simple cells of the visual cortex. Two approximated receptive fields to mimic the biological cells are presented, and a local rotation operator is introduced to determine the optimal output and local orientation corresponding to an image position, which serve as the low-level feature employed in this paper. To generate the low-level features, a bisection method is applied to the first derivative of the model of receptive fields. Moreover, the feature screener is introduced to eliminate too much useless low-level features, which will speed up the processing time. After all the valuable low-level features are combined, the overall image orientation is estimated. The proposed approach possesses several features suitable for real-time applications. First, it avoids the tedious training procedure of some conventional methods. Second, no specific reference such as the horizon is assumed and no a priori knowledge of image is required. The proposed approach achieves a real-time orientation estimation of natural images using only low-level features with a satisfactory resolution. The effectiveness of our proposed approach is verified on real images with complex scenes and strong noises.

In order to implement model-based recognition of human motion intention, dynamics modeling and identification of a lower limb rehabilitation robot named iLeg is investigated. Due to the relatively strong motion constraints, the traditional identification methods become insufficient for iLeg in three aspects: (1) the coupling factors among joints have not been considered in the traditional joint friction models, which makes the structural error and the torque estimation errors relatively large; (2) because of the small and complicated feasible region caused by the motion constraints, the traditional initialization strategy, for searching the valid initial solutions of the optimization problem for the exciting trajectories, becomes very inefficient; and (3) the condition number of the observation matrix, calculated from the preliminary dynamic model and the associated optimized exciting trajectory, is too large for the identification, and, however, further reduction of the condition number has not been considered in the literature. Therefore, corresponding contributions are presented to overcome the limitation. First, the coupling factors among joints are considered in the joint friction model by using the Palmgren empirical formulation and a polynomial fitting method. Then, an indirectly generating strategy is designed, by which the valid initial solutions of the optimization problem can be found with good efficiency. Moreover, a recursive optimization method based on the optimization of the dynamic model and the exciting trajectories, is proposed to further reduce the condition number. Finally, the performance of the proposed methods is demonstrated by several experiments.

This paper proposes a novel general framework for line segment perception, which is motivated by a biological visual cortex, and requires no parameter tuning. In this framework, we design a model to approximate receptive fields of simple cells. More importantly, the structure of biological orientation columns is imitated by organizing artificial complex and hypercomplex cells with the same orientation into independent arrays. Besides, an interaction mechanism is implemented by a set of self-organization rules. Enlightened by the visual topological theory, the outputs of these artificial cells are integrated to generate line segments that can describe nonlocal structural information of images. Each line segment is evaluated quantitatively by its significance. The computation complexity is also analyzed. The proposed method is tested and compared to state-of-the-art algorithms on real images with complex scenes and strong noises. The experiments demonstrate that our method outperforms the existing methods in the balance between conciseness and completeness.

A lower limb rehabilitation robot, namely iLeg, has been developed recently. Since active exercises have been proven to be effective for neurorehabilitation and motor recovery, they are suggested to be implemented on iLeg. To this goal, patients' motion intention should be recognized. Therefore, a method based on the dynamic model of the human-robot interface (HRI) is designed to recognize the human motion intention. This paper is devoted to modeling and identifying the dynamics of the HRI. Firstly, the dynamic model of the HRI is designed by combining the dynamic models of the human leg and iLeg, where the human leg dynamic model (HLDM) is mainly concerned. By considering the motion trajectories during the rehabilitation exercises provided by iLeg, the human leg can be taken as a manipulator with two degrees of freedom; meanwhile, the joint angles and torques of the human leg can be measured indirectly by using the position and torque sensors mounted on the joints of iLeg. As a result, an 8-parameter HLDM can be designed by using the Lagrangian method. Then, the dynamic model of the HRI is identified by respectively and independently identifying the undetermined dynamic parameters of iLeg and the HLDM, where the dynamic parameters of the HLDM are mainly considered. Finally, the feasibility of the dynamic model of the HRI is validated by experiments.

This paper surveys the developments of the last 20 years in the field of central pattern generator (CPG) inspired locomotion control, with particular emphasis on the fast emerging robotics-related applications. Functioning as a biological neural network, CPGs can be considered as a group of coupled neurons that generate rhythmic signals without sensory feedback; however, sensory feedback is needed to shape the CPG signals. The basic idea in engineering endeavors is to replicate this intrinsic, computationally efficient, distributed control mechanism for multiple articulated joints, or multi-DOF control cases. In terms of various abstraction levels, existing CPG control models and their extensions are reviewed with a focus on the relative advantages and disadvantages of the models, including ease of design and implementation. The main issues arising from design, optimization, and implementation of the CPG-based control as well as possible alternatives are further discussed, with an attempt to shed more light on locomotion control-oriented theories and applications. The design challenges and trends associated with the further advancement of this area are also summarized.

This paper proposes a method of hierarchical WSN deployment and routing for fast localization. Position-known anchor nodes form a static upper-layer network organization and several mobile nodes constitute the lower-layer network which need dynamic message exchange with the upper network. In the upper static network, near-optimal routing table is allocated to each anchor node based on their known-position and the strength of signal for communication during network initialization. Mobile nodes in the lower-layer can access to the upper network dynamically and at the same time report the RSSI value of nearby anchors to the control-center. At the center, the computer will calculate the real-time location of the target based on the reported RSSI and node-IDs. Algorithm presented in this paper is low cast, easy to implement and can be used for fast location estimation and motion tracking indoors and outdoors.

This paper is devoted to the active turn control of a free-swimming multilink dolphin-like robot, with emphasis on yaw and pitch controls. With full consideration of both mechanical configuration and propulsive principle of the robot consisting of a yaw joint and multiple pitch joints, a viable approach to perform yaw maneuvers via laterally directed biases is formed, providing an advantage in qualitative and quantitative assessment. Meanwhile, based on the feedback of the pitch angle measured by an onboard gyroscope, a closed-loop control strategy in dorsoventral motions is proposed to achieve agile and swift pitch maneuvers. More remarkably, two hybrid acrobatic stunts, i.e., frontflip and backflip, are first implemented on the physical robot. The latest results obtained demonstrate the effectiveness of the proposed methods. It is also confirmed that the dolphin robot achieves better performance for pitch maneuvers than it does for yaw maneuvers, agreeing well with the biological observations.

The trajectory tracking problem of a closed-chain five-bar robot is studied in this paper. Based on an error transformation function and the backstepping technique, an approximation-based tracking algorithm is proposed, which can guarantee the control performance of the robotic system in both the stable and transient phases. In particular, the overshoot, settling time, and final tracking error of the robotic system can be all adjusted by properly setting the parameters in the error transformation function. The radial basis function neural network (RBFNN) is used to compensate the complicated nonlinear terms in the closed-loop dynamics of the robotic system. The approximation error of the RBFNN is only required to be bounded, which simplifies the initial "trail-and-error" configuration of the neural network. Illustrative examples are given to verify the theoretical analysis and illustrate the effectiveness of the proposed algorithm. Finally, it is also shown that the proposed approximation-based controller can be simplified by a smart mechanical design of the closed-chain robot, which demonstrates the promise of the integrated design and control philosophy.

This paper presents the behavior design and multimodal locomotion control of a biomimetic amphibious robot based on a bio-inspired CPG (central pattern generator). A set of four key parameters are introduced serving as external stimuli to shape the CPG rhythmic activities where necessary speed and orientation modulation as well as 3-D locomotion can be obtained. In terms of the built parameter set, a library of movement primitives based on finite state machine is established to facilitate rapid and smooth gait transitions. To enhance adaptive behaviors, well-integrated sensory feedback by means of two liquid-level detectors enables the gait transition between ground and water autonomously. Simulations and experiments are also conducted to demonstrate the feasibility of a behavior based control architecture governed by CPGs.

In this paper, we present the formulation of a dynamic model of a free-swimming multi-joint robotic fish with a pair of wing-like pectoral fins, in which the whole robot is regarded as a moving multilink rigid body in fluids. Considering that the thrust of fish mainly results from the force of trailing vortex, added lateral pressure, and leading-edge suction force, the dynamic equations of the swimming fish have been derived by summing up the longitudinal force, lateral force, and yaw moment on each propulsive component in the framework of Lagrangian mechanics. Furthermore, using the bio-inspired Central Pattern Generators (CPGs) as the swimming data generator, the overall dynamic propulsive characteristics of the swimming robot are estimated in a mathematical environment (i.e., Mathematica). As a case study, the created dynamic model offers a good guide to seeking pragmatic backward swimming patterns for a carangiform robotic fish, which exemplifies the validity of the CPG-coupled dynamic model.

Presented in this paper is our effort to create a multifunctional swimming robot, i.e., robotic fish, inspired by the well-integrated, configurable multiple control surfaces existing in real fish. By virtue of the hybrid propulsion capability in the tail plus the caudal fin and the maneuverability in accessory fins, a novel, synthesized propulsion scheme composed of multiple artificial control surfaces is proposed, involving the tail plus the caudal fin, pectoral fins, pelvic fin, and dorsal fin. Multimodal locomotion is then accomplished by manipulation of control surfaces, separately or cooperatively, allowing the robot to maneuver more diversely and agilely. In particular, bio inspired Central Pattern Generators (CPGs) based locomotion control is adopted for online swimming gait generation. Aquatic testing has been carried out to demonstrate the improved maneuverability and stability of the robotic fish underwater as well as the effectiveness of the conceived multi-fin mechatronic design.

An analytic flying model that can well represent the physical behavior is derived, where the ball's self-rotational velocity changes along with the flying velocity. Based on the least square method, a rebound model that represents the relation between the velocities before and after rebound is established. The initial trajectory is fitted to three second order polynomials of the flying time with the measured positions of the ball. The initial velocities of the ball in the analytic flying model, including the flying velocity and the self-rotational velocity, are computed from the polynomials. The ball's landing position and velocity is predicted with the model. The velocities after rebound are determined with the rebound model. By taking the velocities after rebound as new initial ones, the flying trajectory after rebound is described with the model again. In other words, the ball's trajectory is predicted. Experimental results verify the effectiveness of the proposed method.

A recurrent neural network is proposed for solving the non-smooth convex optimization problem with the convex inequality and linear equality constraints. Since the objective function and inequality constraints may not be smooth, the Clarke's generalized gradients of the objective function and inequality constraints are employed to describe the dynamics of the proposed neural network. It is proved that the equilibrium point set of the proposed neural network is equivalent to the optimal solution of the original optimization problem by using the Lagrangian saddle-point theorem. Under weak conditions, the proposed neural network is proved to be stable, and the state of the neural network is convergent to one of its equilibrium points. Compared with the existing neural network models for non-smooth optimization problems, the proposed neural network can deal with a larger class of constraints and is not based on the penalty method. Finally, the proposed neural network is used to solve the identification problem of genetic regulatory networks, which can be transformed into a non-smooth convex optimization problem. The simulation results show the satisfactory identification accuracy, which demonstrates the effectiveness and efficiency of the proposed approach.

In this paper, the firing rate of the neuron based on temporal coincidence coding is estimated for correlation detection. Two cases are considered: the independent inputs are stochastic spike trains that are modeled by the homogeneous Poisson process or the renew process. The situation that the inputs are correlated is also considered and the conditions for the neuron to detect the correlation among inputs are discussed. The results are demonstrated by the simulations.

This paper presents a control architecture for the underwater locomotion control of a biomimetic amphibious robot with multi-mobility mechanism. In view of both hydrodynamic problem and engineering approach, we develop a robotic prototype capable of multi-mode motion. A robust gait control for steady swimming using the central pattern generator (CPG) is proposed and has been successfully applied to the robot. The CPG can produce coordinated patterns of rhythmic activity while being simply modulated by control parameters including input drive, frequency, amplitude, threshold, etc., which will be suitable for manually interactive modulation. Using the CPG model, the robot is capable of performing and switching between various locomotion modes such as swimming forwards and backwards, turning and pitching, with the speed, direction and gait types modulated accordingly. A test-bed is provided and results are presented demonstrating interesting properties of the CPG-based control approach and feasibility of the CPG control for efficient propulsion.

This paper proposes a fuzzy logic PID based control method for a biomimetic underwater vehicle on swimming speed and yaw angle. The vehicle has a pair of long-fins installed symmetrically on its body. A set of inertial sensors are applied for collecting its velocity and pose information. And an embedded control system and a driving system based on FPGA are designed for generating and switching motion modes. Based on its motion discipline and system architecture, black-box identification is employed for system modeling. Therefore, according to the control system and driving system, a fuzzy logic PID control scheme is proposed for the underwater vehicle. A fuzzy logic controller is applied for the vehicle before the error is reduced to a given range. Then PID controller is applied when the error is within the given range. Here, velocity control is considered, which involves swimming speed and yaw angle. The fuzzy logic PID control scheme is used for the goals respectively. And Yaw angle control is prior to the swimming speed control. Finally, simulation results show the proposed control scheme is valid.

This paper deals with a novel closed-loop maneuvering control method to enhance the turning precision and turning response speed of a robotic fish propelled via the body and/or caudal fin (BCF) mode. Although the BCF propulsion is favorable for the cases requiring greater thrust and accelerations, its maneuverability can be compensated by effective turning control. In our method, the turning maneuver is divided into three phases: the bending, holding, and unbending phases. After much consideration on turning details, the functions of each phase and the basic control laws are further identified. Results of experiments on in-situ direction tracking and direction maintaining verify the effectiveness of the proposed turning control.

A neural-network-based adaptive approach is proposed for the leader-following control of multiagent systems. The neural network is used to approximate the agent's uncertain dynamics, and the approximation error and external disturbances are counteracted by employing the robust signal. When there is no control input constraint, it can be proved that all the following agents can track the leader's time-varying state with the tracking error as small as desired. Compared with the related work in the literature, the uncertainty in the agent's dynamics is taken into account; the leader's state could be time-varying; and the proposed algorithm for each following agent is only dependent on the information of its neighbor agents. Finally, the satisfactory performance of the proposed method is illustrated by simulation examples.

A dual neural-network method for the coordination of kinematically redundant robots is proposed in this paper. The performance criteria for single robots provided by Nedungadi and Kazerounian are generalized to a multicriteria form for the coordinated-manipulation system composed of multiple serial manipulators. By optimizing the local joint torques and generalized forces applied on the object/workpiece using a designed weighting matrix, the proposed method achieves the global stability during the coordinated-manipulation process. Moreover, the proposed algorithm has an explicit physical meaning, i.e., both the global kinetic energy of the coordination system and the two-norm of the generalized forces applied on the object are minimized simultaneously. In addition, the physical limits of both joint torques and the generalized forces applied on the object are considered, which makes the original coordination problem become a complicated optimization problem subject to both equality and inequality constraints. Compared with numerical optimization algorithms used in existing literatures, the dual neural-network method has better computational capability to deal with the complicated optimization problem. Finally, illustrative examples are given to show that the proposed method is effective and efficient for the multirobot coordinated-manipulation system.

This paper focuses on deriving a robust backstepping control approach to solve the active vibration isolation problem using a Stewart platform. The dynamics of the Stewart platform driven by the linear voice coil motors is developed with the Newton-Euler method. By fully considering the characteristics of vibration isolation, the properties of the dynamics of the Stewart platform are applied to transform the coupled dynamics into six independent single-input single-output (SISO) channels. Furthermore, in the procedure of controller design, the influence factors of vibration isolation are taken into account, such as the parameter perturbation and the unmodeled dynamics, etc. Meanwhile, high-gain design method is employed to deal with the problem introduced by input unmodeled dynamics of the system. It is demonstrated that a sufficiently small L2gain from disturbance to output can be obtained in Lyapunov synthesis. The simulation results show that the controller can effectively attenuate low frequency vibrations in six degrees of freedom (DOFs) and a satisfactory vibration isolation performance can be achieved.

This paper presents a new turning maneuver generation method for a multilink biomimetic robotic fish, in which smooth step functions are introduced to dynamically trigger directed offsets in active and asymmetric swimming. With the proposed method, three basic turning modes can be unified into a general framework by choosing appropriate step-function combinations and dynamic bias. Furthermore, this method can be employed to maneuver the robotic fish agilely in the path planning, which promises more flexibility and steadiness in potential applications to bio-inspired autonomous underwater vehicles.

A recurrent neural network is proposed to deal with the convex optimization problem. By employing a specific nonlinear unit, the proposed neural network is proved to be convergent to the optimal solution in finite time, which increases the computation efficiency dramatically. Compared with most of existing stability conditions, i.e., asymptotical stability and exponential stability, the obtained finite-time stability result is more attractive, and therefore could be considered as a useful supplement to the current literature. In addition, a switching structure is suggested to further speed up the neural network convergence. Moreover, by using the penalty function method, the proposed neural network can be extended straightforwardly to solving the constrained optimization problem. Finally, the satisfactory performance of the proposed approach is illustrated by two simulation examples.

In this paper, a radial basis function disturbance observer (RBFDO) based controller is developed to solve the control problem of an electrically driven Stewart platform for multiple degree-of-freedom (DOF) active vibration isolation. The RBFDO's are employed to monitor the modeling errors and external disturbances, etc. And on-line tuning rules for updating the weights of the RBFDO's are designed based on the e1-modification algorithm. Meanwhile, by considering the dynamics of the Stewart platform and its voice coil actuators, the developed RBFDO's are integrated with the backstepping method to design the active vibration isolation controller. In the presence of external vibrations and model uncertainties, the uniformly ultimately boundedness of the stabilization errors and the weight estimation errors can be guaranteed by the Lyapunov theory. Finally, simulation results demonstrate the proposed controller can effectively attenuate external low-frequency vibrations in six DOFs.

In this paper, a delayed projection neural network is proposed for solving a class of linear variational inequality problems. The theoretical analysis shows that the proposed neural network is globally exponentially stable under different conditions. By the proposed linear matrix inequality (LMI) method, the monotonicity assumption on the linear variational inequality is no longer necessary. By employing Lagrange multipliers, the proposed method can resolve the constrained quadratic programming problems. Finally, simulation examples are given to demonstrate the satisfactory performance of the proposed neural network.

A robust adaptive control approach is proposed to solve the consensus problem of multiagent systems. Compared with the previous work, the agent's dynamics includes the uncertainties and external disturbances, which is more practical in real-world applications. Due to the approximation capability of neural networks, the uncertain dynamics is compensated by the adaptive neural network scheme. The effects of the approximation error and external disturbances are counteracted by employing the robustness signal. The proposed algorithm is decentralized because the controller for each agent only utilizes the information of its neighbor agents. By the theoretical analysis, it is proved that the consensus error can be reduced as small as desired. The proposed method is then extended to two cases: Agents form a prescribed formation, and agents have the higher order dynamics. Finally, simulation examples are given to demonstrate the satisfactory performance of the proposed method.

Recent publications and developments based on SVM have shown that using multiple kernels instead of a single one can enhance interpretability of the decision function and improve classifier performance, which motivates researchers to explore the use of homogeneous model obtained as linear combinations of kernels. However, the use of multiple kernels faces the challenge of choosing the kernel weights, and an increased number of parameters that may lead to overfitting. In this paper we show that MKL problem with a enhanced spatial pyramid match kernel can be solved efficiently using projected gradient method. Weights on each kernel matrix (level) are included in the standard SVM empirical risk minimization problem with a L2constraint to encourage sparsity. We demonstrate our algorithm on classification tasks, which is based on a linear combination of the proposed kernels computed at multiple pyramid levels of image encoding, and we show that the proposed method is accurate and significantly more efficient than current approaches.

An adaptive neural network controller is proposed to deal with the task-space tracking problem of manipulators with kinematic and dynamic uncertainties. The orientation of manipulator is represented by the unit quaternion, which avoids singularities associated with three-parameter representation. By employing the adaptive Jacobian scheme, neural networks, and backstepping technique, the torque controller is obtained which is demonstrated to be stable by the Lyapunov approach. The adaptive updating laws for controller parameters are derived by the projection method, and the tracking error can be reduced as small as desired. The favorable features of the proposed controller lie in that: (1) the uncertainty in manipulator kinematics is taken into account; (2) the unit quaternion is used to represent the end-effector orientation; (3) the “linearity-in-parameters” assumption for the uncertain terms in dynamics of manipulators is no longer necessary; (4) effects of external disturbances are also considered in the controller design. Finally, the satisfactory performance of the proposed approach is illustrated by simulation results on a PUMA 560 robot.

This paper proposes a kinematic modeling method for a bio-inspired robotic fish based on single joint. Lagrangian function of freely swimming robotic fish is built based on a simplified geometric model. In order to build the kinematic model, the fluid force acting on the robotic fish is divided into three parts: the pressure on links, the approach stream pressure and the frictional force. By solving Lagrange's equation of the second kind and the fluid force, the movement of robotic fish is obtained. The robotic fish's motion, such as propelling and turning are simulated, and experiments are taken to verify the model.

A recurrent neural network is proposed to deal with the nonlinear variational inequalities with linear equality and nonlinear inequality constraints. By exploiting the equality constraints, the original variational inequality problem can be transformed into a simplified one with only inequality constraints. Therefore, by solving this simplified problem, the neural network architecture complexity is reduced dramatically. In addition, the proposed neural network can also be applied to the constrained optimization problems, and it is proved that the convex condition on the objective function of the optimization problem can be relaxed. Finally, the satisfactory performance of the proposed approach is demonstrated by simulation examples.

Object recognition and categorization are considered as fundamental steps in the vision based navigation for inspection robot as it must plan its behaviors based on various kinds of obstacles detected from the complex background. However, current approaches typically require some amount of supervision, which is viewed as a expensive burden and restricted to relatively small number of applications in practice. For this purpose, we present an computationally efficient approach that does not need supervision and is capable of learning object categories automatically from unlabeled images which are represented by an set of local features, and all sets are clustered according to their partial-match feature correspondences, which is done by a enhanced Spatial Pyramid Match algorithm (E-SPK). Then a graph-theoretic clustering method is applied to seek the primary grouping among the images. The consistent subsets within the groups are identified by inferring category templates. Given the input, the output of the approach is a partition of the images into a set of learned categories. We demonstrate this approach on a field experiment for a powerline inspection robot.

This paper proposes a new forward passageway (FP) based real-time collision-free target tracking approach for a mobile robot with local sensing. After the position of the target is estimated and localized in robot coordinate system through the combination of vision system and encoder, the sonar information and the target position are converted to a uniform environment model framework called decision-making space. Based on the space, a FP based decision-making is given to endow the robot with the ability to avoid possible obstacles and track the target in unknown environments. Experiment results show the validity of the proposed approach.

The swimming backward method for biomimetic carangiform robotic fish is analyzed in this paper based on the dynamic/kinematic model. The equation of Lagrange of multi-link carangiform robotic fish and simplified fluid force are inducted to calculate the dynamic and kinematic characteristics of the motions. A specific gait is calculated to make the profile of the carangiform robotic fishpsilas undulation fit the characteristics of European eelpsilas swimming backward, which is summarized from the motion sequence of European eel. The simulated and experimental data is given to verify the method.

An adaptive control approach is proposed to deal with the multi-manipulator system consensus problem based on the multi-agent theory. In the current multi-agent literature, agents are assumed to have determined models. However, the real manipulator's dynamics contains uncertain parameters. According to the “linearity-in-parameters” property, the adaptive updating law for uncertain dynamics parameters is derived by the projection method. Then, a decentralized controller is designed based on the backstepping scheme, which only utilizes the information of connected manipulators. By the proposed controller, all the manipulators' joints move towards the same configuration to achieve certain coordination tasks. In addition, performance of the control system is analyzed by the Lyapunov method, and the consensus error is proved to approach zero. Finally, the effectiveness of the proposed scheme is illustrated by simulations on a multiple two-link manipulators system.

A controller is proposed for the robust backstepping control of a class of nonlinear pure-feedback systems using fuzzy logic. The proposed control scheme utilizes fuzzy logic systems to learn the behavior of the unknown plant dynamics. Filtered signals are employed to circumvent algebraic loop problems encountered in the implementation of the usual controllers, and the approximation errors can be efficiently counteracted by employing smooth robust compensators. Most importantly, the uniform ultimate boundedness of all signals in the closed-loop system can be guaranteed, anda prioriknowledge of the plant dynamics is no longer required. Furthermore, the proposed method can be used for adaptive control of a large class of single-input--single-output nonlinear systems in both strict-feedback and pure-feedback forms, and has great potential in many diverse applications. The performance of the proposed approach is demonstrated through three simulation examples, including one nonlinear pure-feedback and two nonlinear strict-feedback systems.

This paper deals with maneuver issues of a multilink biomimetic robotic fish, particularly focusing on turning control in free swimming. The characteristic parameters determining turning performance involve magnitude, position, and time of the deflections applied to the links, which are discussed via a series of simulation calculations and actual experiments.

In this paper, a new active visual system is developed, which is based on bionic vision and is insensitive to the property of the cameras. The system consists of a mechanical platform and two cameras. The mechanical platform has two degrees of freedom of motion in pitch and yaw, which is equivalent to the neck of a humanoid robot. The cameras are mounted on the platform. The directions of the optical axes of the two cameras can be simultaneously adjusted in opposite directions. With these motions, the object's images can be located at the centers of the image planes of the two cameras. The object's position is determined with the geometry information of the visual system. A more general model for active visual positioning using two cameras without a neck is also investigated. The position of an object can be computed via the active motions. The presented model is less sensitive to the intrinsic parameters of cameras, which promises more flexibility in many applications such as visual tracking with changeable focusing. Experimental results verify the effectiveness of the proposed methods.

This paper presents an approach for sonar feature map building. The approach is composed of extracting features at the data-level fusion stage and fusing the extracted features with the registered features in the map at the feature-level fusion stage. A data-level fusion model, termed three measurements association model (TMAM), has been developed for associating three measurements with a line or a point feature. By use of TMAM, different sets of measurements obtained from a single sonar sensor at consecutive steps are associated with the line and point features. Subsequently, the parameters of the identified features are estimated by use of the iterated least square estimation method. Finally, when a feature is extracted, a simple feature-level fusion strategy is used to update the map. The proposed approach has been tested both in simulation and on real data.

A recurrent neural network is proposed for solving non-smooth nonlinear programming problems, which can be regarded as a generalization of the smooth nonlinear programming neural network used in (X.B. Gao, 2004). Based on the non-smooth analysis and the theory of differential inclusions, the proposed neural network is demonstrated to be globally convergent to the exact optimal solution of the original optimization problem. Compared with the existing neural networks, the proposed approach takes both equality and inequality constraints into account, and no penalty parameters have to be estimated beforehand. Therefore, it can solve a larger class of non-smooth programming problems. Finally, several illustrative examples are given to show the effectiveness of the proposed neural network.

A recurrent neural network for the optimal control of a group of interconnected dynamic systems is presented in this paper. On the basis of decomposition and coordination strategy for interconnected dynamic systems, the proposed neural network has a two-level hierarchical structure: several local optimization subnetworks at the lower level and one coordination subnetwork at the upper level. A goal-coordination method is used to coordinate the interactions between the subsystems. By nesting the dynamic equations of the subsystems into their corresponding local optimization subnetworks, the number of dimensions of the neural network can be reduced significantly. Furthermore, the subnetworks at both the lower and upper levels can work concurrently. Therefore, the computation efficiency, in comparison with the consecutive executions of numerical algorithms on digital computers, is increased dramatically. The proposed method is extended to the case where the control inputs of the subsystems are bounded. The stability analysis shows that the proposed neural network is asymptotically stable. Finally, an example is presented which demonstrates the satisfactory performance of the neural network.

This paper focuses on the design of fishlike underwater robots using an optimization approach to choose relative link lengths. Considering both ichthyologic characteristics and mechatronic constraints, the optimal link-length ratios are numerically calculated by an improved constrained cyclic variable method. Comparative results, before and after the optimization, demonstrate the enhanced performance

With the development of multicast service in the Internet, much attention has been drawn to multicast congestion control and analysis. Multicast traffic poses new challenges to the design of Internet congestion control protocols and system stability analysis. The rate control problem of feedback-based sessions on the coexistence of both unicast and multirate multicast traffic architecture networks is focused upon in this paper. First, a fairness problem is discussed in detail, and a reasonable consumption strategy is proposed. In the reasonable consumption strategy, scaling functions are adaptively adjusted based on a relationship between the session rates. Second, contraposing the case that available link capacities are changing with time for these feedback-based unicast and multicast sessions, stability analysis of a closed-loop rate control system under the modified rate mechanism is made based on Lyapunov stable theory. Finally, the simulations illustrate the effectiveness and goodness of the reasonable consumption strategy

High-voltage power transmission line inspection robot must plan its behavior to detect the obstacles from the complex background according to their types when it is crawling along the power transmission line in order to negotiate reliably. In most cases, robot fulfills the task by its vision system. However, motion blur due to camera motion caused by wind or other unknown causes can significantly degrade the quality of the image acquired. This is a typical kind of the so called image restoration problem, which is a hard problem since no prior knowledge of the motion is available. For this purpose, a novel approach for image restoration is proposed. The restoration procedure consists of two stages: estimation of blur function parameters and reconstruction of images. Image degradation model is proposed first to identify blur function parameters, then a recurrent neural network is used to restore the blurred image. Experiments on real blurred images on power transmission line prove the feasibility and reliability of this algorithm. Our experiments show that the restoration procedure consumes only small amount of computation time.

Real-time control of multi-robot coordination system has attracted a lot of attention in recent years. Traditional numerical algorithm is ineffective to perform this task. In this paper, a dual neural network approach is applied to resolve the coordination problem of two redundant robots. By this approach, the joint torque and distributed load can be obtained by optimizing a multiple criteria, and the physical limits of the joint torque and distributed load can be also incorporated into the control scheme. The dual neural network has a simple structure which is composed of only one layer of neuron array. The network configuration is updated by the command signals of desired acceleration of the grasped object, and the output of the network is the manipulator's joint torque. A simulation example is presented to demonstrate the effectiveness of the dual neural network method.

Inspection robot must plan its behavior to detect the obstacles from the complex background according to their types when it is crawling along the power transmission line in order to negotiate reliably. However, in most instances, detecting the obstacles from the complex background is a hard task. For this purpose, a novel and fast visual obstacle recognition algorithm is designed based on the structure of the 220 KV power transmission line. Basic principle and architecture of the algorithm are given. By this approach, three typical obstacles on the power transmission line such as insulator strings, counterweights and suspension clamps can be recognized with high accuracy. Experiments in the real power transmission line show its effectiveness. This method can contribute to the process of the mobile robot negotiating obstacles

In this paper, a new visual positioning method based on corresponding points at two adjacent views is developed for mobile robots. A camera is mounted on a wheeled mobile robot with nonholonomic constraints. The camera whose intrinsic parameters are well calibrated can rotate around an axis perpendicular to the ground plane. The relative orientation and scaled position offsets of the mobile robot are computed from the corresponding points in the common part of the views despite their unknown positions in Cartesian space. The relative orientation is used in a simple visual dead reckoning method to modify the odometry information. Then, based on the relative orientation and modified odometry information, equations are given to determine the position and orientation of the mobile robot. Experiments are performed to verify the effectiveness of the proposed methods

In this paper a seminal system for the topographic survey with an unmanned aerial robot is presented. The proposed system has demonstrated the feasibility of acquiring initial 3D ground models using active laser range sensors on a low-flying helicopter platform. The robot system consists of a remote control helicopter with a laser sensor and a GPS (global positioning system) for collecting ground point data and a post-processing data sub-system for drawing a topographic map. This robot system has many potential applications, such as terrain modeling, structure inspection or climate and weather measurement, etc. The experiment results verify the proposal robot system for topographic survey

In this paper, a method for the posture control of a biomimetic robot fish TPF-I is proposed. In this method, the position of the robot fish's gravity centre can be changed by a barycenter-adjustor, which leads to the pitching angle changing. Propelled by coordinating a multi-link body and a tail, the robot fish can complete the posture control and 3-D Locomotion. The 3-D locomotion and posture control are implemented by synthesizing three basic control methods speed control, orientation control and pitching control, which are described in detail respectively. Finally, the experimental results of the robot fish's motion control are given and the performance is analyzed

This paper proposes a distributed control approach called local interactions with local coordinate systems (LILCS)to multirobot hunting tasks in unknown environments, where a team of mobile robots hunts a target called evader, which will actively try to escape with a safety strategy. This robust approach can cope with accumulative errors of wheels and imperfect communication networks. Computer simulations show the validity of the proposed approach.

In this paper, a new type of six-degrees of freedom (DOF) flexible parallel mechanism (FPM) is presented. This type of parallel mechanism possesses several favorable properties: (1) its number of DOFs is independent of the number of serial chains which make up the mechanism; (2) it has no kinematical singularities; (3) it is designed to move on rails, and therefore its workspace is much larger than that of a conventional parallel manipulator; and (4) without changing the number of DOFs and the kinematics of the mechanisms, the number of the serial chains can be reconfigured according to the needs of the tasks. These properties make the mechanism very preferable in practice, especially for such tasks as joining huge ship blocks, in which the manipulated objects vary dramatically both in weights and dimensions. Furthermore, the mechanism can be used as either a fully actuated system or an underactuated system. In the fully actuated case, the mechanism has six DOF motion capabilities and manipulation capabilities. However, in the underactuated case, the mechanism still has six DOF motion capabilities, but it has only five DOF manipulation capabilities. In this paper, both the inverse and forward kinematics are studied and expressed in a closed form. The workspace and singularity analysis of the mechanism are also presented. An example is presented to illustrate how to calculate the kinematics of the mechanism in both fully-actuated and underactuated cases. Finally, an application of such a mechanism to manufacturing industry is introduced.

In this paper, a new method is proposed to detect the pose of an object with two cameras. First, the intrinsic parameters of the cameras are self-calibrated with two pairs of parallel lines that are orthogonal. Then, the poses of the cameras relative to the parallel lines are deduced, and the rotational transformation between the two cameras is calculated. With the intrinsic parameters and the relative pose of the two cameras, a method is proposed to obtain the poses of a line, plane, and rigid object. Furthermore, a new visual-control method is developed using a pose detection rather than a three-dimensional reconstruction. Experiments are conducted to verify the effectiveness of the proposed method.

A method of image process and features extraction for structured light image of welding seam with arc and splash disturbance is proposed. The seam area is detected by search with large step. The adaptive thresholds of image enhancement are determined in the frequency domain of the gray image. Then, the target image is pre-processed using image enhancement and binarization. After thinning the seam with its both edges, main characteristic line is obtained using Hotelling transform and Hough transform. Finally, the feature points in the seam are found according to its second derivative. Experimental results show its effectiveness, good performance in real time and adaptability to different seams.

This paper is concerned with the design of a robotic fish and its motion control algorithms. A radio-controlled, four-link biomimetic robotic fish is developed using a flexible posterior body and an oscillating foil as a propeller. The swimming speed of the robotic fish is adjusted by modulating joint's oscillating frequency, and its orientation is tuned by different joint's deflections. Since the motion control of a robotic fish involves both hydrodynamics of the fluid environment and dynamics of the robot, it is very difficult to establish a precise mathematical model employing purely analytical methods. Therefore, the fish's motion control task is decomposed into two control systems. The online speed control implements a hybrid control strategy and a proportional-integral-derivative (PID) control algorithm. The orientation control system is based on a fuzzy logic controller. In our experiments, a point-to-point (PTP) control algorithm is implemented and an overhead vision system is adopted to provide real-time visual feedback. The experimental results confirm the effectiveness of the proposed algorithms.

This paper explores the application of the behavior-based approach to path planning for multiple mobile robots performing a formation control task in unknown environments. To predict the positions of moving obstacles for the purpose of collision avoidance, parabola prediction model whose parameters are estimated by the recurrence least square algorithm with restricted scale is adopted. Then, on the basis of the task and environment, we adopt five primitive behaviors and design a series of generation functions to generate control parameters for behaviors' combination. Furthermore, as the outputs of these functions can be adjusted according to the current situation, thus robots can achieve a motion strategy by reasonably combining behaviors and the adaptability to the environment is improved. We illustrate the validity of the approach by the simulations.

In this paper, we present a new foraging algorithm applied to multi-robot system. The algorithm classifies robots into two groups; one group is engaged in navigating, while the other in collecting. Due to different duties of robots in foraging, the robots team displays diversity during the foraging process. The results of simulation show that the team of greater diversity would be more efficient than the team without diversity in accomplishing the foraging task. Under the consideration of such factors as the robot team's size and the density of attractors in environment, the relationship between the duty-based diversity and the performance of multi-robot system is discussed based on simulation results.

Dead reckoning is an important method for mobile robot. If its accuracy can be improved, navigation tasks will be simplified. The mobile robot concerned in this paper is with five wheels, in which there are dual driving wheels and a castor. A measurement wheel with encoder is fixed beside each driving wheel. There are two encoders fixed on the castor. One measures the castor's movement, and another indicates the angle of yawing. A new kind of kinematics equations derived from the robot's turning radius and angle of movement trajectory is presented. We can get a group of turning radius and angle from the data of four encoders, in which there are four different values for radius and six different values for angle. The estimated values of radius and angle can be gotten by fusion using fuzzy algorithm. The improved dead reckoning method has the advantages of simplicity, less computation cost, cheapness and good performance in uneven road. The simulation shows its effectiveness.