An increasing number of underwater robots inspired by Carangidae are developed, which is characterized by high efficiency and flexibility. However, estimating the swimming direction of these robotic fish is challenging due to the constant swinging of the head during movement, which complicates precise control. In this study, we installed two low-cost inertial measurement unit (IMU) sensors separately on the head and tail parts of a double-joint robotic fish and presented a method for accurately and timely estimating the swimming direction. Firstly, we effectively compensated for the yaw angle drift of the IMU sensors through a fused Kalman Filter. Furthermore, we propose the Anti-Shake Estimation (ASE) algorithm to calculate the real-time swimming direction using filtered yaw angles at a high updating rate of 100Hz. Finally, we applied the method to swimming direction feedback control for evaluation and comparison. The results show that our ASE method performs better than other existing methods in straight-line swimming experiments. The experiment of S-curve swimming also demonstrates the effectiveness of our method in complex missions.

In this article, the pose regulation control problem of a robotic fish is investigated by formulating it as a Markov decision process (MDP). Such a typical task that requires the robot to arrive at the desired position with the desired orientation remains a challenge, since two objectives (position and orientation) may be conflicted during optimization. To handle the challenge, we adopt the sparse reward scheme, i.e., the robot will be rewarded if and only if it completes the pose regulation task. Although deep reinforcement learning (DRL) can achieve such an MDP with sparse rewards, the absence of immediate reward hinders the robot from efficient learning. To this end, we propose a novel imitation learning (IL) method that learns DRL-based policies from demonstrations with inverse reward shaping to overcome the challenge raised by extremely sparse rewards. Moreover, we design a demonstrator to generate various trajectory demonstrations based on one simple example from a nonexpert helper, which greatly reduces the time consumption of collecting robot samples. The simulation results evaluate the effectiveness of our proposed demonstrator and the state-of-the-art (SOTA) performance of our proposed IL method. Furthermore, we deploy the trained IL policy on a physical robotic fish to perform pose regulation in a swimming tank without/with external disturbances. The experimental results verify the effectiveness and robustness of our proposed methods in real world. Therefore, we believe this article is a step forward in the field of biomimetic underwater robot learning.

A large challenge in the field of soft amphibious robotics is achieving high maneuverability and multi-terrain adaptability through multi-modal locomotion in hybrid terrestrial-aquatic environments. To address this issue, drawing inspiration from fruit-fly larvae and Spanish dancer sea slugs, a novel tethered soft amphibious robot with multi-modal locomotion is proposed in this paper, performing forward, backward, turning, and self-overturn motions both on land and in water. It leverages plastic sheet-reinforced thin pneumatic actuators, which are constructed from thermoplastic membranes and embedded with a non-stretchable plastic sheet, enabling bi-directional bending with large angles. The robot achieves a forward jumping velocity of 1.77BL/s and a forward swimming velocity of 0.69BL/s, both faster than previously reported soft amphibious robots; connecting two actuator units in parallel, it achieves agile turning with a velocity of 111.8$^\circ$/s. Our proposed robot demonstrates exceptional multi-terrain adaptability, facile terrestrial-aquatic transition capabilities, and underwater buoyancy adjustment ability. Especially when accidentally overturned, it can recover itself without external assistance, a capability rarely achieved by other soft robots.

The fish-like robot is one of the typical underwater robots, which has the advantage of high maneuverability with low noise due to its bioinspired structure and biomimetic locomotion. However, it is challenging to efficiently design motion controllers for such robots to achieve satisfactory performance on specific control tasks in the real underwater environment, since the complex fluid-structure interaction exists during their swimming and exact dynamic models are absent. In this article, we propose a learning framework, incorporating a simulation system and a training methodology, to autonomously and fast train in simulation to create control policies that are capable of directly applying to a type of physical fish-like robots to perform motion control tasks. First, we construct a simulation system combining a data-driven environment and a computational fluid dynamics (CFD)-based environment, thus well balancing the simulation accuracy and the calculation speed. Second, we design a training methodology to train deep reinforcement learning (DRL)-based policies for the robot in our constructed simulation system to perform a specific control task. Then, we use two typical motion control tasks to verify our proposed framework. One is the path-following control task, which is a one-objective problem with dense rewards, while the other is the pose control task which is a two-objective problem with sparse rewards. For each task, the DRL-based control policy trained by our learning framework is directly deployed on the physical fish-like robot to perform the task in the real world. Experimental results show that the policies trained in simulation still work well in the real world, and perform even better in terms of control accuracy and stability compared with the traditional control methods, thus demonstrating the effectiveness of our learning framework.

Controlling biomimetic underwater robots in unknown flow fields remains a challenge due to the strong nonlinearity of the fluid. This article investigates the attitude holding task of a robotic fish swimming in reality. Such a typical sensing-based control task requires the fish to keep a desired angle of attack in an unknown and even varied incoming flow. To this end, we propose a learning-based approach by using a deep neural network directly maps the raw data of sensors equipped on the robot to the continuous control signals in an end-to-end manner. First, based on experimental data of the physical robot, we construct a data-driven simulation environment including three modules of dynamic, sensor, and control. The dynamic and sensor modules are established to model the dynamics of the fish and to generate its sensors’ data, based on which a deep reinforcement learning (DRL) algorithm in the control module is trained to get a control policy. Then, we directly deploy the trained policy to a physical robotic fish for attitude holding task. Experimental results demonstrate the robustness and effectiveness of the DRL policy and, thus, verify the success of our approach to achieving sim-to-real transfer.

Value decomposition recently injects vigorous vitality into multi-agent actor-critic methods. However, existing decomposed actor-critic methods cannot guarantee the convergence of global optimum. In this paper, we present a novel multi-agent actor-critic method, FOP, which can factorize the optimal joint policy induced by maximum-entropy multi-agent reinforcement learning (MARL) into individual policies. Theoretically, we prove that factorized individual policies of FOP converge to the global optimum. Empirically, in the well-known matrix game and differential game, we verify that FOP can converge to the global optimum for both discrete and continuous action spaces. We also evaluate FOP on a set of StarCraft II micromanagement tasks, and demonstrate that FOP substantially outperforms state-of-the-art decomposed value-based and actor-critic methods.

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.

For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, its limitations on flight altitude prevent the UAVs from observing the global map. Thus motion planning in the local map becomes a Partially Observable Markov Decision Process (POMDP) problem. This paper proposes a motion planning algorithm for heterogeneous unmanned systems under partial observation from UAV without reconstruction of global maps. Our algorithm consists of two parts designed for perception and decision-making, respectively. For the perception part, we propose the Grid Map Generation Network (GMGN), which is used to perceive scenes from UAV's perspective and classify the pathways and obstacles. For the decision-making part, we propose the Motion Command Generation Network (MCGN). Due to the addition of the memory mechanism, MCGN has planning and reasoning abilities under partial observation from UAVs. We evaluate our proposed algorithm by comparing it with baseline algorithms. The results show that our method effectively plans the motion of heterogeneous unmanned systems and achieves a relatively high success rate.

Underwater communication is particularly challenging for small submarine robots that have limited power and size constraints. Inspired by weakly electric fish, a novel electric current communication (termed electrocommunication) system has been developed for small underwater robots in our previous studies. However, collision problems sometimes occur during multiple robots communication because the electrocommunication network shares a common channel. In this paper, a new CSMA/CA-based electrocommunication system is presented to solve this collision problem. An efficient circuit for communication channel state (busy or idle) detection is proposed. After that, a compact Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) algorithm is introduced and finally implemented on the system to judiciously avoid collision during multiple robot communication. The effectiveness of the proposed CSMA/CA-based communication system for underwater robot groups is well verified by both simulations and experiments with three robotic fish models.

Artificial lateral line has been drawing an increasing attention recently for its potential applications in robotics. Experiments are usually conducted with a bioinspired robot in a controlled environment, where the sensing platform is held stationary or slowly driven with a simple linear motion. In this paper, we conduct a more practical and challenging study where the robot uses artificial lateral line to evaluate its linear velocity while freely swimming. We use onboard artificial lateral line to measure the pressure profiles over the surface of a robotic fish and employ onboard IMU (inertial measurement unit) to record the motion kinematics of the robot while freely swimming at various speeds. We find that 1) pressure changes are greatest on the head of the robot; 2) pressures increase along with the swimming speed and the oscillation amplitude of angular velocity of the robot. Therefore, we propose a nonlinear prediction model which incorporates distributed pressure and angular velocity to estimate the speed of the robot. Online speed evaluation experiment demonstrates the effectiveness and the accuracy of the proposed model.

In this paper, a dynamic model is proposed for a carangiform-like robotic fish swimming both forwards and backwards. The robotic fish, which consists of a streamlined head, a flexible body and a caudal fin, is able to propel itself by generating a traveling propulsive wave traversing its body. We first modify the classical body wave function suggested by Lighthill to fit our robotic fish. Then we naturally define the point on fish's body which never undulates during swimming straight as “Fixed-point” and prove its existence and uniqueness. Using the property of the Fixed-point, we propose a model for our robotic fish and further investigate how the swimming speed is affected by the position of the unique Fixed-point. It is found that the robotic fish achieves its maximum speed of swimming forwards and backwards when the Fixed-point located on the head and the tail, respectively. Finally, we apply the proposed model combining with a CPG-based locomotion controller to the real robotic fish. Both simulations and experiments show that the proposed model is capable to predict the speed of the robotic fish.