The underwater hexapod robot, driven by eight thrusters and six C-shaped legs, can perform complex locomotion tasks such as climbing rough terrain. Unlike conventional point-contact legs, the C-shaped leg rolls on the terrain. The rolling fashion brings significantly complex loop-closure kinematic constraints and complicates the finding of feasible gaits. In addition, when C-shaped legs roll on rough terrain, their contact condition will change in real-time, leading to time-varying contact force, which may result in the leg slipping or even the robot falling. To address the two issues, we propose gait planning and thruster force allocating methods for rough terrain climbing. First, we propose a sampling-based gait planner that extends random trees in task space and finds feasible gaits to fulfill the loop-closure kinematic constraints, which avoids designing the complex sampling and steering procedures in an implicitly-defined manifold. Second, by designing a gait interval-based cost function, we propose an optimal sampling-based planner to find smooth climbing gaits. Third, by establishing a simplified single rigid body (SRB) model, we formulate an optimization problem to allocate thruster forces to guarantee that contact forces cannot lead to the leg slipping. Finally, the effectiveness and practicality of the proposed methods are validated via extensive Gazebo simulations as well as hardware experiments. Note to Practitioners—The motivation for this paper stems from the need to develop a practical rough terrain climbing algorithm for a thruster-assisted underwater hexapod robot that can walk the underwater structure with any dip angles to perform some meticulous small-range operations such as hull cleaning, fracture detection, and damage restoration. However, existing climbing algorithms mainly focus on finding legs’ torques or footsteps for point-contact legged robots. They may fail to be directly used in the underwater robot simultaneously driven by thrusters and C-shaped rolling-contact legs. Then, we propose an optimal gait planning method to find C-shaped legs’ smooth desired rotation angles and an optimal thruster force allocation method to regulate each support leg’s contact force to avoid slipping. Finally, the gait planning method can also be applied to other rolling-contact legged robots, and the thruster force allocation method can help other thruster-assisted legged robots perform complex locomotion tasks.

The stability criterion is critical for the design of legged robots' motion planning and control algorithms. If these algorithms cannot theoretically ensure legged robots' stability, we need many trials to identify suitable parameters for stable locomotion. However, most existing stability criteria are tailored to robots driven solely by legs and cannot be applied to thruster-assisted legged robots. Here, we propose a stability criterion for a thruster-assisted underwater hexapod robot by finding maximum and minimum allowable thruster forces and comparing them with the current thrusts to check its stability. On this basis, we propose a method to increase the robot's stability margin by adjusting the value of thrusts. This process is called stability enhancement. The criterion uses the optimization method to transform multiple variables such as attitude, velocity, acceleration of the robot body, and the angle and angular velocity of leg joints into one kind of variable (thrust) to judge the stability directly. In addition, the stability enhancement method is straightforward to implement because it only needs to adjust the thrusts. These provide insights into how multiclass forces such as inertia force, fluid force, thrust, gravity, and buoyancy affect the robot's stability.

In this work, pursuit-evasion (PE) games with marine surface vessels (MSVs) as pursuers are solved while considering velocity constraints and unknown dynamics simultaneously. Differentiable performance index functions are designed for PE games based on minimum and maximum approximation functions. Then, we can obtain the desired pursuit velocities for MSVs satisfying velocity constraints and evasion strategies by applying game theory. NN are established to approximate unknown dynamics, which is suitable to design neural network (NN)-based control to ensure that all velocities of MSVs converge to their desired ones. Through rigorous Lyapunov analyses, it can be guaranteed that all convergence and weight errors are uniformly ultimately boundedUUB. Simulation results and comparison with known dynamics are provided and analyzed, which show that the proposed NN-based PE game is effective for MSVs with velocity constraints and unknown dynamics.

In this article, we investigate the problem of capturing a noncooperative target with deception behavior using reinforcement learning (RL) under incomplete information. The pursuer copes not only with its maneuverability constraint but also with the target’s deception behavior, in which the target deliberately conceals its private preference information. The target capture game involving deception behavior is formulated as a nonlinear differential game framework where the information structure is incomplete and asymmetric. The solution to this differential game is proposed based on an RL policy that incorporates critic, actor, and virtual actor neural networks (NNs), when taking into consideration the maneuverability constraint and information structure of the pursuer. Moreover, the states of the constrained adversarial system and the weight errors are proven to be ultimately uniformly bounded (UUB). To counter the deception of the target, we adopt unscented Kalman filter (UKF) to obtain the target intention on energy preference, and integrate it into the pursuer strategy. The feasibility of the proposed strategy and its superiority are verified through comparisons with recent works.

In this study, we present a coordinated protocol for a multiagent system (MAS) in competitive and cooperative manners for moving target defense with dynamic task-switching. The protocol comprises three components. First, we design a distance-based competitive distributed decision algorithm within an improvedk-Winner-Take-All (k-WTA) framework. This algorithm generates dynamic binary task-driven signals for each agent, enabling near-optimal online grouping of MAS with arbitrary proportions. Second, we introduce a cooperative strategy that employs a shared decision-making mechanism and utilizes feedback linearization without global position information. This strategy generates motion planning signals to coordinate the agents’ actions, achieving overall cooperative behaviors such as tracking, capturing, and intercepting. Finally, we incorporate an adaptive sliding mode technique based on second-order nonlinear dynamics to enhance robustness against disturbances, ensuring uniformly ultimately boundedness (UUB) of the closed-loop system. In addition, simulations and experiments with wheeled mobile robots (WMRs) validate the effectiveness of our method.

Path planning in unpredictable dynamic environments remains a challenging problem due to the unpredictable appearance, disappearance, and movement of dynamic obstacles during navigation. To address this problem, we propose a reverse tree guided rapid exploration random tree (RTRRT) algorithm that can efficiently perform navigation tasks in dynamic environments. The method first constructs a reverse tree rooted as goal state to search for an initial path. If a collision occurs on the path, The RT-RRT constructs a forward tree rooted as the current robot state in the same configuration space, until it connects with the reverse tree to find a new path. Furthermore, The RT-RRT improves the tree construction method and designs a path optimization strategy to reduce the path cost. The method is validated in different scenarios and has excellent navigation capabilities in unpredictable dynamic environments. In the same scenarios, the RT-RRT algorithm improves the success rate by 16.7%, reduces the path length by 20.54% and reduces the travel time by 10X compared to the RRTXalgorithm with the same number of samples.

In this article, we present a novel and robust registration method for autonomous underwater vehicles (AUV) using the forward-looking sonar (FLS). Due to the sparsity and repeatability of the underwater environment, and the loss of elevation angle, the FLS is difficult to consistently provide reliable information. The feature extraction and matching are the main difficulties encountered in registrations. This motivates us to propose a method to deal with various underwater environments and provide reliable information to improve the robustness and accuracy of the registration. First, the sonar images are preprocessed to highlight the region of interest and transformed. Next, a sonar image sequence is used for preliminary registration. Last, a combinatorial method including a feature-based region selection and a region-based registration is proposed. The validation is based on datasets and real-world experiments. Comparison studies verify the effectiveness of the proposed method.

This article studies the distributed robust optimal control for discrete-time linear multiagent systems (MASs) with parametric uncertainties, where digraphs that only contain a directed spanning tree are allowed. Using the linear quadratic regulator approach, an optimal control protocol is presented. The presented controller is fully distributed, since the global information of graphs is unneeded for the design and implementation of the presented controller. The global performance index of MASs can be minimized by using the presented control protocol, and the optimal solution is independent with the information of parametric uncertainties. Finally, some simulated examples are provided to show the effectiveness of the proposed approaches.

This article presents an improved guidance law for underactuated marine vessels that compensates cross-track error caused by external disturbances through its sideslip. The proposed guidance law demonstrates improved path-following performance regardless of disturbances, such as waves, winds, and ocean currents. This article also presents an adaptive neural-network (NN) control law for the partially known vessel dynamics with state constraints. For satisfying the state constraints, this control scheme adopts an integral barrier Lyapunov function (iBLF)-based backstepping control technique. It is shown that the closed-loop system remains bounded, and state constraints are always satisfied. Finally, the efficacy of the improved guidance law and iBLF-based adaptive control strategy was verified in simulation and experiments using an autonomous surface vessel.

This article is concerned with the robust cooperative optimal control of nonlinear multiagent systems (MASs) with external disturbances and modeling uncertainties. Using the super-twisting algorithm, a continuous sliding-mode control protocol is presented for high-order nonlinear MASs with multiple inputs. The sliding-mode dynamics is modeled by the Takagi-Sugeno fuzzy approach, and the nominal control protocol that guarantees the robust optimization of the cost function is designed. Directed topologies are allowed using the presented protocol, and many assumptions about topologies are removed. Finally, three numerical examples are reported to demonstrate the effectiveness and improved performance of the presented protocol.

This article investigates the task assignment problem in which multiple dispersed robots need to visit a set of target locations while trying to minimize the robots’ total travel distance. Each robot initially has the position information of all the targets and of those robots that are within its limited communication range, and each target demands a robot with some specified capability to visit it. We propose a decentralized auction algorithm which first employs an information consensus procedure to merge the local information carried by each communication-connected (CC) robot subnetwork. Then, we apply a marginal-cost-based strategy to construct conflict-free target assignments for the CC robots. When the communication network of the robots is not connected, we demonstrate that the robots’ total travel distance might in fact increase when their communication range grows, and more importantly, such a somewhat counterintuitive fact holds for a range of algorithms. Furthermore, the proposed algorithm guarantees that the total travel distance of the robots is at most twice of the optimal when the communication network is initially connected. Finally, Monte Carlo simulation results demonstrate the satisfying performance of the proposed algorithm.

This article develops a self-triggered adaptive neural network (NN) tracking controller for a class of continuous-time nonlinear systems, that is, input constrained and with unknown drift and input dynamics. Since the drift and input dynamics are both unknown, an NN is built within a self-triggered update paradigm to approximate the unknown tracking control. The error derivative used in the weight update algorithm is derived using a robust exact differentiator technique. To address input constraints, an auxiliary compensator is designed for the unimplemented control effort. Through rigorous Lyapunov analyses, we can guarantee that all the tracking and weight errors are uniformly ultimately bounded. Finally, to show the effectiveness of the proposed control performance, simulation results of a two-link robot are provided and analyzed.

This article studies the task assignment problem for a fleet of dispersed vehicles to efficiently visit a set of target locations where some target locations might be unreachable for one or several vehicles. The objectives are to visit as many target locations as possible by using the minimum number of vehicles while minimizing the vehicles' total travel time. We first propose a target merging strategy to deal with the optimization problem, which is in general NP-hard, and show that for the special case of a single vehicle, it requires linear time to calculate the maximum number of targets to be visited. Second, we design a longest path-based algorithm and analyze the cases in which the objective to visit the maximum number of targets by using the minimum number of vehicles can be obtained through the proposed algorithm within linear running time. Once the targets to be visited and the corresponding employed vehicles are determined, the marginal-cost-based target inserting principle to be discussed guarantees that the chosen targets will be visited within a computable finite maximal travel time, which is at most twice of the optimal when the cost matrix is symmetric. Integrating the longest path-based algorithm with two target inserting principles used to minimize the vehicles' total travel time, we design two two-phase task assignment algorithms. Furthermore, we propose a one-phase algorithm to optimize the multiple objectives simultaneously by improving a co-evolutionary multipopulation genetic algorithm. Numerical simulations show that the proposed task assignment algorithms can lead to satisfying solutions against popular genetic algorithms.

This article investigates the task planning problem where one vehicle needs to visit a set of target locations while respecting the precedence constraints that specify the sequence orders to visit the targets. The objective is to minimize the vehicle's total travel distance to visit all the targets while satisfying all the precedence constraints. We show that the optimization problem is NP-hard, and consequently, to measure the proximity of a suboptimal solution from the optimal, a lower bound on the optimal solution is constructed based on the graph theory. Then, inspired by the existing topological sorting techniques, a new topological sorting strategy is proposed; in addition, facilitated by the sorting, we propose several heuristic algorithms to solve the task planning problem. The numerical experiments show that the designed algorithms can quickly lead to satisfying solutions and have better performance in comparison with popular genetic algorithms.

This paper studies the precedence-constrained task assignment problem for a team of heterogeneous vehicles to deliver packages to a set of dispersed customers subject to precedence constraints that specify which customers need to be visited before which other customers. A truck and a micro drone with complementary capabilities are employed where the truck is restricted to travel in a street network and the micro drone, restricted by its loading capacity and operation range, can fly from the truck to perform the last-mile package deliveries. The objective is to minimize the time to serve all the customers respecting every precedence constraint. The problem is shown to be NP-hard, and a lower bound on the optimal time to serve all the customers is constructed by using tools from graph theory. Then, integrating with a topological sorting technique, several heuristic task assignment algorithms are proposed to solve the task assignment problem. Numerical simulations show the superior performances of the proposed algorithms compared with popular genetic algorithms.

In this paper, event-triggered reinforcement learning-based adaptive tracking control is developed for the continuous-time nonlinear system with unknown dynamics and external disturbances. The critic and action neural networks are designed to approximate an unknown long-term performance index and controller, respectively. The dead-zone event-triggered condition is developed to reduce communication and computational costs. Rigorous theoretical analysis is provided to show that the closed-loop system can be stabilized. The weight errors and the filtered tracking error are all uniformly ultimately bounded. Finally, to demonstrate the developed controller, the simulation results are provided using an autonomous underwater vehicle model.

In this article, a synchronous reinforcement-learning-based algorithm is developed for input-constrained partially unknown systems. The proposed control also alleviates the need for an initial stabilizing control. A first-order robust exact differentiator is employed to approximate unknown drift dynamics. Critic, actor, and disturbance neural networks (NNs) are established to approximate the value function, the control policy, and the disturbance policy, respectively. The Hamilton-Jacobi-Isaacs equation is solved by applying the value function approximation technique. The stability of the closed-loop system can be ensured. The state and weight errors of the three NNs are all uniformly ultimately bounded. Finally, the simulation results are provided to verify the effectiveness of the proposed method.

In this paper, an integral reinforcement learning-based adaptive neural network (NN) tracking control is developed for the continuous-time (CT) nonlinear system with unknown control directions. The long-term performance index in the CT domain is prescribed. Critic and action NNs are designed to approximate the unavailable long-term performance index and the unknown dynamics, respectively. The reinforcement signal is explicitly embedded in the updated law of the action NN and then the estimated long-term performance index can be minimized. Rigorous theoretical analysis is provided to show that the closed-loop system is stabilized and all closed-loop signals are semiglobally uniformly ultimately bounded. Finally, to demonstrate the control performance, simulation results are provided to verify the tacking control performance of an autonomous underwater vehicle model.

This paper investigates the sliding-mode control (SMC) problem of Takagi-Sugeno (T-S) fuzzy multiagent systems (MASs). A cooperative fuzzy-based dynamical sliding-mode (SM) controller is designed and the overall closed-loop T-S fuzzy MAS is constructed. A new model transformation method for T-S fuzzy MASs is presented to transform the fuzzy weighting matrix into a set of fuzzy weighting scalars. By applying the method of linear matrix inequality, a general stability analysis approach for T-S fuzzy MASs is proposed. Moreover, the energy-cost constraint problem is studied by using the linear quadratic regulator method. Finally, numerical examples are provided to illustrate the effectiveness of the proposed theoretical approaches and the improved performance compared to existing results.

This paper investigates the dynamic coverage control problem for a group of agents with unknown density function. A cost function, depending on a certain metric and the density function, is defined to describe the performance of coverage network. Since the optimal deployment of agents is closely depending on the density function, we employ the Bayesian prediction approaches to estimate the density function. Moreover, a novel coverage-control-customized algorithm is proposed to acquire the Bayesian parameters. The merits of this Bayesian-based spatial estimation algorithm are the consideration of measurement noise and the capability of dealing time-varying density function. However, the estimated density function from Bayesian framework follows normal distribution, which leads the cost function to a stochastic process. To deal with this type of cost function, a discrete control scheme is proposed to steer the agents approaching to a near-optimal deployment. The mean-square stability of the proposed coverage system is further analyzed. Finally, numerical simulations are provided to verify the effectiveness of the proposed approaches.

This paper proposes a novel adaptive control methodology based on the admittance model for multiple manipulators transporting a rigid object cooperatively along a predefined desired trajectory. First, an admittance model is creatively applied to generate reference trajectory online for each manipulator according to the desired path of the rigid object, which is the reference input of the controller. Then, an innovative integral barrier Lyapunov function is utilized to tackle the constraints due to the physical and environmental limits. Adaptive neural networks (NNs) are also employed to approximate the uncertainties of the manipulator dynamics. Different from the conventional NN approximation method, which is usually semiglobally uniformly ultimately bounded, a switching function is presented to guarantee the global stability of the closed loop. Finally, the simulation studies are conducted on planar two-link robot manipulators to validate the efficacy of the proposed approach.

Source localization is a fundamental problem in underwater wireless sensor networks. From the observability analysis, we know that the sensor placement can significantly affect the localization performance. This letter is concerned with the optimal sensor placement for underwater source localization, and a parameter is introduced into the measurement model to represent the distance-dependent noise. The evaluation criterion used to solve the optimal placement is built by the Cramer-Rao lower bound theory. Subsequently, we mainly discuss the case when the distances between the sensors and the source are fixed, and the optimal sensor placement is affected by the relative magnitude of the distances.

This paper investigates the issue on how to utilize neighbor information in the distributed receding horizon control (RHC)-based consensus problem for first-order multiagent systems. The distributed RHC-based consensus problem is first formulated in a general framework in terms of using neighbor information. Based on the framework, a sufficient condition on utilizing neighbor information to ensure consensus is developed for the finite horizon case. For the infinite horizon case, a necessary and sufficient condition is proposed, and the best way of using neighbor information to achieve fastest convergence rate is also presented. It is shown that: 1) the way of utilizing neighbor information plays an important role in reaching consensus; 2) the parameter that ensures consensus is related with the network topology; and 3) the best convergence rate in consensus can be attained if the neighbor information is appropriately utilized. Simulation studies verify the proposed theoretical results.

Autonomous underwater vehicles (AUVs) have been widely employed in ocean survey, monitoring, and search and rescue tasks for both civil and military applications. It is beneficial to use multiple AUVs that perform environmental sampling and sensing tasks for the purposes of efficiency and cost effectiveness. In this paper, an adaptive path planning algorithm is proposed for multiple AUVs to estimate the scalar field over a region of interest. In the proposed method, a measurable model composed of multiple basis functions is defined to represent the scalar field. A selective basis function Kalman filter is developed to achieve model estimation through the information collected by multiple AUVs. In addition, a path planning method, the multidimensional rapidly exploring random trees star algorithm, which uses mutual information, is proposed for the multi-AUV system. Employing the path planning algorithm, the sampling positions of the AUVs are determined to improve the quality of future samples by maximizing the mutual information between the scalar field model and observations. Extensive simulation results are provided to demonstrate the effectiveness of the proposed algorithm. Additionally, an indoor experiment using four robotic fishes is carried out to validate the algorithms presented.

The relative sensor-target formation configuration can significantly affect the potential performance of any particular localization algorithm. An evaluation measure based on the one-step prediction error covariance in extended Kalman filter process is derived in this letter. We determine the optimal sensor placement for the range-only localization considering the prior target location information. The conclusions are different from that in the static estimation problem.