Cooperative Multi-Agent Reinforcement Learning (MARL) has become a critical tool for addressing complex real-world problems. However, off-policy MARL methods, which rely on joint Q-functions, face significant scalability challenges due to the exponentially growing joint action space. In this work, we highlight a critical yet often overlooked issue: erroneous Q-target estimation, primarily caused by extrapolation error. Our analysis reveals that this error becomes increasingly severe as the number of agents grows, leading to unique challenges in MARL due to its expansive joint action space and the decentralized execution paradigm. To address these challenges, we propose a suite of techniques tailored for off-policy MARL, including annealed multi-step bootstrapping, averaged Q-targets, and restricted action representation. Experimental results demonstrate that these methods effectively mitigate erroneous estimations, yielding substantial performance improvements in challenging benchmarks such as SMAC, SMACv2, and Google Research Football.

Recent advances in recommender systems have shown that user-system interaction essentially formulates long-term optimization problems, and online reinforcement learning can be adopted to improve recommendation performance. The general solution framework incorporates a value function that estimates the user's expected cumulative rewards in the future and guides the training of the recommendation policy. To avoid local maxima, the policy may explore potential high-quality actions during inference to increase the chance of finding better future rewards. To accommodate the stepwise recommendation process, one widely adopted approach to learning the value function is learning from the difference between the values of two consecutive states of a user. However, we argue that this paradigm involves a challenge of Mixing Random Factors: there exist two random factors from the stochastic policy and the uncertain user environment, but they are not separately modeled in the standard temporal difference (TD) learning, which may result in a suboptimal estimation of the long-term rewards and less effective action exploration. As a solution, we show that these two factors can be separately approximated by decomposing the original temporal difference loss. The disentangled learning framework can achieve a more accurate estimation with faster learning and improved robustness against action exploration. As an empirical verification of our proposed method, we conduct offline experiments with simulated online environments built on the basis of public datasets.

An increasing number of underwater robots inspired by Carangidae are developed, which is characterized by high efficiency and flexibility. However, estimating the swimming direction of these robotic fish is challenging due to the constant swinging of the head during movement, which complicates precise control. In this study, we installed two low-cost inertial measurement unit (IMU) sensors separately on the head and tail parts of a double-joint robotic fish and presented a method for accurately and timely estimating the swimming direction. Firstly, we effectively compensated for the yaw angle drift of the IMU sensors through a fused Kalman Filter. Furthermore, we propose the Anti-Shake Estimation (ASE) algorithm to calculate the real-time swimming direction using filtered yaw angles at a high updating rate of 100Hz. Finally, we applied the method to swimming direction feedback control for evaluation and comparison. The results show that our ASE method performs better than other existing methods in straight-line swimming experiments. The experiment of S-curve swimming also demonstrates the effectiveness of our method in complex missions.

In recommender systems, reinforcement learning solutions have shown promising results in optimizing the interaction sequence between users and the system over the long-term performance. For practical reasons, the policy's actions are typically designed as recommending a list of items to handle users' frequent and continuous browsing requests more efficiently. In this list-wise recommendation scenario, the user state is updated upon every request in the corresponding MDP formulation. However, this request-level formulation is essentially inconsistent with the user's item-level behavior. In this study, we demonstrate that an item-level optimization approach can better utilize item characteristics and optimize the policy's performance even under the request-level MDP. We support this claim by comparing the performance of standard request-level methods with the proposed item-level actor-critic framework in both simulation and online experiments. Furthermore, we show that a reward-based future decomposition strategy can better express the item-wise future impact and improve the recommendation accuracy in the long term. To achieve a more thorough understanding of the decomposition strategy, we propose a model-based re-weighting framework with adversarial learning that further boost the performance and investigate its correlation with the reward-based strategy.

Recently, reputation-based indirect reciprocity has been widely applied to the study on fairness behavior. Previous works mainly investigate indirect reciprocity by considering compulsory participation. While in reality, individuals may choose voluntary participation according to the opponent's reputation. It is still unclear how such reputation-based voluntary participation influences the evolution of fairness. To address this question, we introduce indirect reciprocity with voluntary participation into the dictator game (DG). We respectively consider good dictators or recipients can voluntarily participate in games when the opponents are assessed as bad. We theoretically calculate the fairness level under all social norms of third-order information. Our findings reveal that several social norms induce the high fairness level in both scenarios. However, more social norms lead to a high fairness level for voluntary participation of recipients, compared with the one of good dictators. The results also hold when the probability of voluntary participation is not low. Our results demonstrate that recipients’ voluntary participation is more effective in promoting the emergence of fairness behavior.

In this article, the pose regulation control problem of a robotic fish is investigated by formulating it as a Markov decision process (MDP). Such a typical task that requires the robot to arrive at the desired position with the desired orientation remains a challenge, since two objectives (position and orientation) may be conflicted during optimization. To handle the challenge, we adopt the sparse reward scheme, i.e., the robot will be rewarded if and only if it completes the pose regulation task. Although deep reinforcement learning (DRL) can achieve such an MDP with sparse rewards, the absence of immediate reward hinders the robot from efficient learning. To this end, we propose a novel imitation learning (IL) method that learns DRL-based policies from demonstrations with inverse reward shaping to overcome the challenge raised by extremely sparse rewards. Moreover, we design a demonstrator to generate various trajectory demonstrations based on one simple example from a nonexpert helper, which greatly reduces the time consumption of collecting robot samples. The simulation results evaluate the effectiveness of our proposed demonstrator and the state-of-the-art (SOTA) performance of our proposed IL method. Furthermore, we deploy the trained IL policy on a physical robotic fish to perform pose regulation in a swimming tank without/with external disturbances. The experimental results verify the effectiveness and robustness of our proposed methods in real world. Therefore, we believe this article is a step forward in the field of biomimetic underwater robot learning.

A large challenge in the field of soft amphibious robotics is achieving high maneuverability and multi-terrain adaptability through multi-modal locomotion in hybrid terrestrial-aquatic environments. To address this issue, drawing inspiration from fruit-fly larvae and Spanish dancer sea slugs, a novel tethered soft amphibious robot with multi-modal locomotion is proposed in this paper, performing forward, backward, turning, and self-overturn motions both on land and in water. It leverages plastic sheet-reinforced thin pneumatic actuators, which are constructed from thermoplastic membranes and embedded with a non-stretchable plastic sheet, enabling bi-directional bending with large angles. The robot achieves a forward jumping velocity of 1.77BL/s and a forward swimming velocity of 0.69BL/s, both faster than previously reported soft amphibious robots; connecting two actuator units in parallel, it achieves agile turning with a velocity of 111.8$^\circ$/s. Our proposed robot demonstrates exceptional multi-terrain adaptability, facile terrestrial-aquatic transition capabilities, and underwater buoyancy adjustment ability. Especially when accidentally overturned, it can recover itself without external assistance, a capability rarely achieved by other soft robots.

Multi-agent policy gradient methods in centralized training with decentralized execution recently witnessed many progresses. During centralized training, multi-agent credit assignment is crucial, which can substantially promote learning performance. However, explicit multi-agent credit assignment in multi-agent policy gradient methods still receives less attention. In this paper, we investigate multi-agent credit assignment induced by reward shaping and provide a theoretical understanding in terms of its credit assignment and policy bias. Based on this, we propose an exponentially weighted advantage estimator, which is analogous to GAE, to enable multi-agent credit assignment while allowing the tradeoff with policy bias. Empirical results show that our approach can successfully perform effective multi-agent credit assignment, and thus substantially outperforms other advantage estimators.

The fish-like robot is one of the typical underwater robots, which has the advantage of high maneuverability with low noise due to its bioinspired structure and biomimetic locomotion. However, it is challenging to efficiently design motion controllers for such robots to achieve satisfactory performance on specific control tasks in the real underwater environment, since the complex fluid-structure interaction exists during their swimming and exact dynamic models are absent. In this article, we propose a learning framework, incorporating a simulation system and a training methodology, to autonomously and fast train in simulation to create control policies that are capable of directly applying to a type of physical fish-like robots to perform motion control tasks. First, we construct a simulation system combining a data-driven environment and a computational fluid dynamics (CFD)-based environment, thus well balancing the simulation accuracy and the calculation speed. Second, we design a training methodology to train deep reinforcement learning (DRL)-based policies for the robot in our constructed simulation system to perform a specific control task. Then, we use two typical motion control tasks to verify our proposed framework. One is the path-following control task, which is a one-objective problem with dense rewards, while the other is the pose control task which is a two-objective problem with sparse rewards. For each task, the DRL-based control policy trained by our learning framework is directly deployed on the physical fish-like robot to perform the task in the real world. Experimental results show that the policies trained in simulation still work well in the real world, and perform even better in terms of control accuracy and stability compared with the traditional control methods, thus demonstrating the effectiveness of our learning framework.

Controlling biomimetic underwater robots in unknown flow fields remains a challenge due to the strong nonlinearity of the fluid. This article investigates the attitude holding task of a robotic fish swimming in reality. Such a typical sensing-based control task requires the fish to keep a desired angle of attack in an unknown and even varied incoming flow. To this end, we propose a learning-based approach by using a deep neural network directly maps the raw data of sensors equipped on the robot to the continuous control signals in an end-to-end manner. First, based on experimental data of the physical robot, we construct a data-driven simulation environment including three modules of dynamic, sensor, and control. The dynamic and sensor modules are established to model the dynamics of the fish and to generate its sensors’ data, based on which a deep reinforcement learning (DRL) algorithm in the control module is trained to get a control policy. Then, we directly deploy the trained policy to a physical robotic fish for attitude holding task. Experimental results demonstrate the robustness and effectiveness of the DRL policy and, thus, verify the success of our approach to achieving sim-to-real transfer.

Value decomposition recently injects vigorous vitality into multi-agent actor-critic methods. However, existing decomposed actor-critic methods cannot guarantee the convergence of global optimum. In this paper, we present a novel multi-agent actor-critic method, FOP, which can factorize the optimal joint policy induced by maximum-entropy multi-agent reinforcement learning (MARL) into individual policies. Theoretically, we prove that factorized individual policies of FOP converge to the global optimum. Empirically, in the well-known matrix game and differential game, we verify that FOP can converge to the global optimum for both discrete and continuous action spaces. We also evaluate FOP on a set of StarCraft II micromanagement tasks, and demonstrate that FOP substantially outperforms state-of-the-art decomposed value-based and actor-critic methods.

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.

Value function decomposition (VFD) methods under the popular paradigm of centralized training and decentralized execution (CTDE) have promoted multi-agent reinforcement learning progress. However, existing VFD methods proceed from a group's value function decomposition to only solve cooperative tasks. With the individual value function decomposition, we propose MFVFD, a novel multi-agent Q-learning approach for solving cooperative and non-cooperative tasks based on mean-field theory. Our analysis on the Hawk-Dove and Nonmonotonic Cooperation matrix games evaluate MFVFD's convergent solution. Empirical studies on the challenging mixed cooperative-competitive tasks where hundreds of agents coexist demonstrate that MFVFD significantly outperforms existing baselines.

For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, its limitations on flight altitude prevent the UAVs from observing the global map. Thus motion planning in the local map becomes a Partially Observable Markov Decision Process (POMDP) problem. This paper proposes a motion planning algorithm for heterogeneous unmanned systems under partial observation from UAV without reconstruction of global maps. Our algorithm consists of two parts designed for perception and decision-making, respectively. For the perception part, we propose the Grid Map Generation Network (GMGN), which is used to perceive scenes from UAV's perspective and classify the pathways and obstacles. For the decision-making part, we propose the Motion Command Generation Network (MCGN). Due to the addition of the memory mechanism, MCGN has planning and reasoning abilities under partial observation from UAVs. We evaluate our proposed algorithm by comparing it with baseline algorithms. The results show that our method effectively plans the motion of heterogeneous unmanned systems and achieves a relatively high success rate.

Underwater communication is extremely challenging for small underwater robots which typically have stringent power and size constraints. In our previous work, we developed an artificial electrocommunication system which could be an alternative for the communication of small underwater robots. This paper further presents a new electrocommunication system that utilizes Binary Frequency Shift Keying (2FSK) modulation and deep-learning-based demodulation for underwater robots. We first derive an underwater electrocommunication model that covers both the near-field area and a large transition area outside of the near-field area. 2FSK modulation is adopted to improve the anti-interference ability of the electric signal. A deep learning algorithm is used to demodulate the electric signal by the receiver. Simulations and experiments show that with the same testing condition, the new communication system outperforms the previous system in both the communication distance and the data transmitting rate. In specific, the newly developed communication system achieves stable communication within the distance of 10 m at a data transfer rate of 5 Kbps with a power consumption of less than 0.1 W. The substantial increase in communication distance further improves the possibility of electrocommunication in underwater robotics.

Novel robotic grippers have captured increasing interests recently because of their abilities to adapt to varieties of circumstances and their powerful functionalities. Differing from traditional gripper with mechanical components-made fingers, novel robotic grippers are typically made of novel structures and materials, using a novel manufacturing process. In this paper, a novel robotic gripper with external frame and internal thermoplastic elastomer belt-made net is proposed. The gripper grasps objects using the friction between the net and objects. It has the ability of adaptive gripping through flexible contact surface. Stress simulation has been used to explore the regularity between the normal stress on the net and the deformation of the net. Experiments are conducted on a variety of objects to measure the force needed to reliably grip and hold the object. Test results show that the gripper can successfully grip objects with varying shape, dimensions, and textures. It is promising that the gripper can be used for grasping fragile objects in the industry or out in the field, and also grasping the marine organisms without hurting them.

A lateral line system is a flow-responsive organ system, with which fish can effectively sense the surrounding flow field, thus serving functions in flow-aided fish behaviors. Inspired by such a biological characteristic, artificial lateral line systems (ALLSs) have been developed for promoting technological innovations of underwater robots. In this article, we focus on investigating state estimation of a freely swimming robotic fish in multiple motions, including rectilinear motion, turning motion, gliding motion, and spiral motion. The state refers to motion parameters, including linear velocity, angular velocity, motion radius, etc., and trajectory of the robotic fish. Specifically, for each motion, a pressure variation (PV) model that links motion parameters to PVs surrounding the robotic fish is first built; then, a linear regression analysis method is used for determining the model parameters. Based on the acquired PV model, motion parameters can be estimated by solving the PV model inversely using the PVs measured by the ALLS. Finally, a trajectory estimation method is proposed for estimating trajectory of the robotic fish based on the ALLS-estimated motion parameters. The experimental results show that the robotic fish is able to estimate its trajectory in the aforementioned multiple motions with the aid of ALLS, with small estimation errors.

In this paper, we present the dynamic modeling and model predictive tracking control for a fin-actuated robot with barycentre regulating mechanism in multiple motions. Specifically, a dynamic model for the robot is established firstly. Based on the dynamic model, a model predictive tracking control algorithm is proposed. And simulations of of tracking rectangle trajectory, sine-like trajectory, ascending trajectory, and spiral trajectory are conducted to validate the algorithm. The simulation results demonstrate that the proposed algorithm is able to implement trajectory tracking of the robot with small position error and orientation error. This paper contributes to trajectory tracking for an underwater robot with controllable barycentre in multiple motions, which has been rarely explored.

Lateral line system (LLS) is a sensory organ system which serves functions in varieties of flow-relative fish behaviors. Inspired by excellent performances of LLS in fish behaviors, multiple artificial lateral line systems (ALLSs) have been designed and applied to promote underwater robot technology. In this article, we focus on using ALLS for longitudinal separation sensing between two adjacent swimming robotic fish whose tails oscillate, and meanwhile the two fish are towed with a precisely controlled speed which equals to the rectilinear speed of freely-swimming robotic fish with the same oscillating parameters. Flow visualizations based on dye injection technique, hydrogen bubble technique, and computational fluid dynamics simulation are conducted to study the distribution characteristics of the vortices caused by the robotic fish. In addition, towing tank experiments are conducted for investigating the qualitative relationships between the longitudinal separations and the ALLS-measured HPVs of the two robotic fish. This work provide a guidance for the application of ALLS in relative states sensing of underwater robot group composed of two or more individuals, which has been rarely explored.

In this article, a theoretical model describing the superficial hydrodynamic pressure variations (HPVs) of two adjacent robotic fish with leader-follower formation is established based on Bernoulli equation. Model parameters are identified by measured HPVs data and yawing rate data in experiments, based on multiple linear regression method. The HPVs data are measured by pressure sensor arrays based artificial lateral line system, and the yawing rate data are measured by attitude and heading reference system of the robotic fish. Errors between the measured superficial HPVs and the HPVs estimated by the established model are analyzed in detail. And it demonstrates that the established model has perfect performance in describing the superficial HPVs, with small errors. This article contributes to modeling of flow variations caused by free motions of underwater vehicle, which is always a challenge. The established model has the potentials of providing guidance for flow-aided control of underwater vehicle.

In this paper, suffering from both complex uncertainties and underactuations, accurate trajectory tracking control problem of an asymmetric underactuated surface vehicle (AUSV) is first addressed by guiding yaw dynamics which are free of persistent excitation (PE). Using nested coordinate transformations, the AUSV is formulated in a cascade structure consisting of translation and rotation subsystems with complex uncertainties. Finite-time uncertainty observers (FUOs) are devised to exactly estimate transformed uncertainties, and enable separation principle in controller and observer syntheses. By virtue of creating yaw-guided dynamics, rotation tracking is shaped to stabilize yaw and sway tracking discrepancies, simultaneously, in collaboration with yaw controller. Nominal dynamics of translation tracking errors are globally asymptotically stabilized by surge-control synthesis using cascade analysis and Lyapunov approach, and thereby contributing to global asymptotic stability of the entire translation-rotation tracking system. Eventually, an FUO-based yaw-guided tracking control (FUO-YTC) scheme of an AUSV with complex uncertainties is established. Simulation studies demonstrate remarkable performance.

Autonomous gait optimization is an essential survival ability for mobile robots. However, it remains a challenging task for underwater robots. This paper addresses this problem for the locomotion of a bio-inspired robotic fish and aims at identifying fast swimming gait autonomously by the robot. Our approach for learning locomotion controllers mainly uses three components: 1) a biological concept of central pattern generator to obtain specific gaits; 2) an onboard sensory processing center to discover the environment and to evaluate the swimming gait; and 3) an evolutionary algorithm referred to as particle swarm optimization. A key aspect of our approach is the swimming gait of the robot is optimized autonomously, equivalent to that the robot is able to navigate and evaluate its swimming gait in the environment by the onboard sensors, and simultaneously run a built-in evolutionary algorithm to optimize its locomotion all by itself. Forward speed optimization experiments conducted on the robotic fish demonstrate the effectiveness of the developed autonomous optimization system. The latest results show that our robotic fish attained a maximum swimming speed of 1.011 BL/s (40.42 cm/s) through autonomous gait optimization, faster than any of the robot's previously recorded speeds.

Underwater communication is particularly challenging for small submarine robots that have limited power and size constraints. Inspired by weakly electric fish, a novel electric current communication (termed electrocommunication) system has been developed for small underwater robots in our previous studies. However, collision problems sometimes occur during multiple robots communication because the electrocommunication network shares a common channel. In this paper, a new CSMA/CA-based electrocommunication system is presented to solve this collision problem. An efficient circuit for communication channel state (busy or idle) detection is proposed. After that, a compact Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) algorithm is introduced and finally implemented on the system to judiciously avoid collision during multiple robot communication. The effectiveness of the proposed CSMA/CA-based communication system for underwater robot groups is well verified by both simulations and experiments with three robotic fish models.

Due to the harsh characteristic of underwater environment, it is difficult for the underwater vehicles to communicate with others. Exploring the various underwater communication schemes is essential for the future practical applications. In this paper, some commonly underwater communication schemes are introduced which emphasize on their technical index, operation environment, advantages and the improvement needed firstly. Then some efforts have been taken on the performance parameters which mainly focus on the underwater wireless communication methods. Finally, a large segment of this paper is devoted to introduce the research status and the expectation of the underwater electric current communication. In a word, the electric current communication has broad application prospect in underwater communication domain.

Artificial lateral line has been drawing an increasing attention recently for its potential applications in robotics. Experiments are usually conducted with a bioinspired robot in a controlled environment, where the sensing platform is held stationary or slowly driven with a simple linear motion. In this paper, we conduct a more practical and challenging study where the robot uses artificial lateral line to evaluate its linear velocity while freely swimming. We use onboard artificial lateral line to measure the pressure profiles over the surface of a robotic fish and employ onboard IMU (inertial measurement unit) to record the motion kinematics of the robot while freely swimming at various speeds. We find that 1) pressure changes are greatest on the head of the robot; 2) pressures increase along with the swimming speed and the oscillation amplitude of angular velocity of the robot. Therefore, we propose a nonlinear prediction model which incorporates distributed pressure and angular velocity to estimate the speed of the robot. Online speed evaluation experiment demonstrates the effectiveness and the accuracy of the proposed model.

Communication is challenging for underwater robots. This paper presents the first research into developing an underwater electric current communication system and integrating the system into a small robotic fish. It is notable for its potential for a group of underwater robots communicating within a short distance range in conditions where optical and acoustic methods would meet difficulty. The working principle of the electric current communication is explained by a simplified electric dipole model. After that, systematic design of the electric current communication system is proposed for underwater robots. Communication experiments with the robotic fish demonstrate the effectiveness of the developed electric current communication system. The experimental results show that a remote control system can communicate underwater with our robotic fish over a distance of three meters by use of electric current communication.

Fish possesses a unique sensory organ called the lateral line. The lateral line provides fish with flow-related information. It is accepted that fish can use the lateral line to sense states of its neighbours in schooling behaviors. In this study, we investigate how a focal robotic fish senses the states of its swimming neighbour by using its onboard artificial lateral line system for the first time. Dye flow visualization is used to characterize the large-scale structures of the wake behind a swimming robotic fish. In the experiment, the Reverse Karman Vortex Street generated by the anterior robotic fish was sensed by the artificial lateral line of the focal robot. The results show that the robot's artificial lateral line can detect the beating frequency of its neighbouring robot and the distance between the robots. It is promising that artificial lateral line sensing could become one of the most popular close interaction methods for a group of underwater robots in the near future.

In this paper, a dynamic model is proposed for a carangiform-like robotic fish swimming both forwards and backwards. The robotic fish, which consists of a streamlined head, a flexible body and a caudal fin, is able to propel itself by generating a traveling propulsive wave traversing its body. We first modify the classical body wave function suggested by Lighthill to fit our robotic fish. Then we naturally define the point on fish's body which never undulates during swimming straight as “Fixed-point” and prove its existence and uniqueness. Using the property of the Fixed-point, we propose a model for our robotic fish and further investigate how the swimming speed is affected by the position of the unique Fixed-point. It is found that the robotic fish achieves its maximum speed of swimming forwards and backwards when the Fixed-point located on the head and the tail, respectively. Finally, we apply the proposed model combining with a CPG-based locomotion controller to the real robotic fish. Both simulations and experiments show that the proposed model is capable to predict the speed of the robotic fish.

In recent years, many bio-inspired computation algorithms have been proposed to solve constraint problems. Biogeography-Based Optimization (BBO) is one of these newly proposed optimization algorithms. As a new way to solve complicated optimization problems, BBO has a quick convergence. In this paper, we proposed an improved BBO for solving protein structure prediction problems. Comparative experiments with standard BBO and differential evolution algorithm (DE) are also conducted, and the results demonstrate this improved BBO approach performs better in solving these complicated protein prediction problems.

This paper focuses on the dynamic modeling of a self-propelled, multimodal ostraciiform robotic fish, whose three active joints (two pectoral fins and one caudal fin) are actuated by a Central Pattern Generator (CPG) controller. Compared with other dynamic modes for robotic fish, we introduce angle of attack (AoA) theory on the fish modeling, which can be used to further explore the relationship between swimming efficiency and AoA of robotic fish. First, by using the quasi-steady wing theory, AoA of the oscillatory fins are explicitly derived. Then, with the simplification of the robot as a multi-rigid-body mechanism, AoA-based fluid forces acting on the oscillatory fins of the robot are further approximately calculated in a three-dimensional context. Next, by importing the driving signals (generated by CPG control law) into a Lagrangian function, the differential-algebraic equations are employed to establish a hydrodynamic model for steady swimming of the ostraciiform robotic fish for the first time. Finally, comparative results between simulations and experiments for forward and turning gaits of the robot are systematically conducted to show the effectiveness of the built AoA-based dynamic model.

This paper considers the consensus problem for second-order multi-agent systems with inherent nonlinear dynamics under directed topologies. A variable transformation method is used to convert the consensus problem to a partial stability problem. Both fixed and switching topologies are considered. Under the assumption that the inherent nonlinear term satisfies the Lipshitz condition, sufficient conditions on the feedback gains and the Lipschitz constant to ensure consensus are given based on a Lyapunov function method.

This paper presents a novel approach for a group of vision-based autonomous robots to localize in dynamic environments. We propose a hybrid system method for localization consisted of on-line and off-line subsystems inspired by human cognition. For the on-line subsystem, we use the landmark based Markov localization method to estimate the position. When the robot does not update the probability of current position through landmarks for a certain period, we use the off-line experience subsystem to help. In addition to the hybrid system for individual localization, we propose a method of dynamic reference object for collaborative localization. By using this method, an autonomous robot can estimate and correct its position perception more accurately and effectively, taking the odometry error and other negative influence into consideration. Satisfactory experimental results are obtained in the RoboCup Four-Legged League environment

In this paper we study the property of controllability for a class of interconnected systems with a leader in switching networks. We obtain necessary and sufficient conditions for the interconnected systems using neighbor rules, to be controllable by one of them acting as a leader. Here, we take the control laws to be an attractive force, and we assume the topology of the control interconnections is variant, that is, each swarm individual (agent) updates is current state based on the current information from neighboring individuals and the leader. Results show that the interconnected systems can be completely controllable even though every subsystem cannot be controllable by selecting one of the as a leader and neighbor interaction rules. Moreover, the swarm can begin from the given initial configuration and reach a desired final configuration while satisfying certain conditions. The model and results of this paper present a novel method to investigate a class of swarms via switching systems and provide further insight into the effect of the interaction pattern on self-organized motion in a swarm system. Numerical simulations are also worked out to illustrate the analytical results.

In this paper, we consider a group of mobile agents moving in the space with point mass dynamics. We investigate the dynamic properties of the group for the case where the topology of the neighboring relations between agents varies with time. Under the assumption that the neighboring graph is always connected, we show that stable flocking motion can be achieved by using a set of switching control laws. The control laws are a combination of attractive/repulsive and alignment forces. Using the control laws, all agent velocities become asymptotically the same, collisions can be avoided between the agents, and the final tight formation minimizes all agent potentials. Moreover, we show that the velocity of the center of mass is invariant and is equal to the final common velocity. Subsequently, we study the motion of the group when the velocity damping is taken into account. In this case, we can appropriately modify the control laws to generate the same stable flocking motion. Finally, we provide some numerical simulations to further illustrate our theoretical results.

Reachability and controllability of positive discrete-time switched linear systems are firstly studied in this paper. First, a theoretic equivalent condition for reachability is addressed. A sufficient condition is also presented, but a counterexample shows that it is not necessary. A verifiable equivalent condition is given for a class of systems. Next, we show that reachability and null controllability is equivalent to complete controllability. A equivalent condition for null controllability is addressed and a verifiable one is given for a class of systems.

This paper presents a new method for controlling a group of nonholonomic mobile robots to achieve predetermined formations without using global knowledge. Based on the dynamic leader-follower model, a reactive tracking controller is proposed to make each following robot maintain a desired pose to its leader, and the stability property of this controller is discussed using Lyapunov theory. By employing such controllers, the N-robot formation control problem can be decomposed into decentralized tracking problems between N-l followers and designated leaders. Additionally, graph theory is introduced to formalize general formation patterns in a simple but effective way and two types of switching between these formations are also proposed. Numerical simulations and physical robots experiments show the effectiveness of our approach.

This paper presents a hierarchical framework for controlling a group of biomimetic fish robots to achieve cooperative tasks. Based on our previous successful work on the design and development of a robotic fish prototype, we attempt further to investigate the cooperation in groups of these fish. Employing top-down design approach, we propose a hierarchical architecture consisting of five levels: task level, role (or mode) level, behavior level, action level and controller level, to formalize the processes from task decomposition, role assignments and control performance. Two typical cases are developed to demonstrate the feasibility of the architecture and corresponding experimental results show that high efficiency and much greater capabilities are exhibited when the fish try to cooperate.

The problem of stabilization of networked control systems (NCSs) with time-varying transmission period is studied. By viewing the time-varying transmission period as time-varying parameter uncertainties, sufficient conditions expressed in linear matrix inequalities are presented under which there exists constant state feedback controller proving the stabilization of NCSs. A numerical example is given to illustrate the utility of our results.

In the paper, commuting and stable feedback design for switched linear systems is investigated. This problem is formulated as to build up state feedback controller for each subsystem such that the closed-loop systems are not only asymptotically stable but also commuting each other. A new concept, common admissible eigenvector set (CAES), is introduced to establish necessary/sufficient conditions for such feedback controllers. For second-order systems, a equivalent condition is established. Moreover, a parametrization of the CAES is also obtained. The motivation comes from stabilization of switched linear systems which consist of a family of LTI systems and a switching law specifying the switching between them, where if all the subsystems are stable and commuting each other, then the total system is stable under arbitrary switching.

We study the stabilization of discrete-time switched linear systems and discrete-time uncertain switched systems when an arbitrary switching rule is imposed on the observer-based static output feedback controllers. Several linear matrix inequality (LMI) based conditions are established to this problem with switched Lyapunov function method for both cases.

Switched systems are a special class of hybrid systems. Different from the existing mathematical models for switched systems, in this paper it is assumed that there are two groups of subsystems and the switching between the two groups is alternate. Based on the new model of switched linear systems under constrained switching, this paper studies the reachability of such systems. A necessary and sufficient geometric criterion for reachability of such systems is established. Finally, an example is given to illustrate the results.

This paper studies the reachability and controllability of a class of switched linear systems with state jumps at the switching instants. Necessary and sufficient geometric criteria are presented. A structural method is given to realize the reachability and controllability.

This paper studies periodic stabilization of a class of switched linear systems. The concepts of periodically asymptotical stabilizability (PAS) and periodically exponential stabilizability (PES) are first introduced. First, for the continuous-time case, a necessary and sufficient condition for PAS is established. Then, it is proved that PES is equivalent to PAS, and controllability is equivalent to PES with arbitrary decaying rate, respectively. The corresponding algorithms are given as well. Finally, a numerical examples are given to illustrate our results.

In this paper, the controllability of a class of planar piecewise linear systems is investigated. An explicit necessary and sufficient condition for null controllability of such systems is established. Moreover, it is proved that the null controllability can be realized infinite steps.