• We propose a smart semantic enhancement collaborative reasoning framework called CORE-CLIP, which is well driven by the openness and robustness of pre-training model CLIP. This framework effectively utilizes the semantic priors of visual-language pre-trained models to construct a text guided visual cross-modal alignment strategy to achieve fine-grained understanding of human-object interaction. The CORE-CLIP can achieve a good tradeoff between the model size and inference speed. • Two novel modules combining text features are elaborated for fine-grained interaction recognition and interaction classification. Specifically, the TDA module uses a dual fusion method to aggregate interactive text information and visual context in the structure to achieve accurate recognition of interaction categories, while the SEC module focuses on three special types of text information to explicitly and efficiently classify objects, actions, and interactions, respectively. • Extensive experiments demonstrate excellent performance on the two benchmarks and self-constructed validating set TableTennis. In particular, the proposed CORE-CLIP improves 1.36 mAP on V-COCO and 1.01 mAP on HICODET compared to GEN-VLKT. Zero-shot experimental results outperform the existing state-of-the-art methods such as GEN-VLKT and HOICLIP. The inference speed of CORE-CLIP surprisingly attains 38.3 FPS while its total parameters are only 52.9M. Ablation studies validate the contribution of each module for efficient and accurate interaction understanding. Human-object interaction (HOI) detection has attracted more and more attention due to its wide potential applications. Recently, Contrastive Language-Image Pre-training (CLIP) achieves promising results in 2D/3D zero-shot and few-shot learning. To overcome current heavy reliance on the large collection of annotated HOI data and often failure to recognize such unseen HOI relationships in training datasets in existing methods, we propose a smart collaborative reasoning framework named CORE-CLIP based on semantic enhancement driven by the openness and robustness of CLIP. Specifically, the text guided dual fusion attention module (TDA) captures precise interaction patterns by progressively integrating CLIP text embeddings, CLIP visual features, and global contextual features. The semantically enhanced explicit interaction classification module (SEC) is initialized by leveraging semantic information of objects, actions, and interactions generated through CLIP text embeddings, to ensure alignment between visual features and linguistic semantics. Extensive experiments demonstrate that CORE-CLIP has state-of-the-art performance on two benchmark datasets. In particular, compared to GEN-VLKT, the proposed CORE-CLIP improves 1.36 mAP on V-COCO and 1.01 mAP on HICO-DET. Zero-shot experimental results outperform state-of-the-art methods GEN-VLKT by a signifigant margin of 34.4 % in UO type setting and HOICLIP by 50.9 % in UO type setting, respectively. The inference speed of CORE-CLIP surprisingly attains 38.3 FPS while its total parameters are only 52.9M.

Enabling quadruped robots to navigate complex, unstructured 3-D environments using onboard sensors remains a significant challenge, particularly in the absence of maps. This difficulty stems from coupling long-horizon decision-making with terrain-aware control under partial observability and limited onboard perception. To address this, we propose a depth-based distillation learning framework featuring a hierarchical architecture that decouples high-level navigation from low-level locomotion. The navigation policy predicts velocity commands from raw depth and goal observations, while the locomotion controller executes motor actions based on proprioception and compact depth features distilled from privileged geometric scans of terrain. To improve representation quality and training efficiency, we incorporate contrastive forward prediction and inverse dynamics modeling into the reinforcement learning loop. A GPU-parallel Warp-based depth renderer is integrated into Isaac Gym to accelerate visual simulation and support large-scale training. To ensure robustness in cluttered environments, we employ an adaptive curriculum learning strategy that progressively increases terrain complexity during training. Our system enables robust, map-free navigation across both structured and unstructured terrain, and demonstrates successful zero-shot deployment on a real quadruped robot.

This paper presents a prescribed time learning-based near optimal enclosing controller to ensure that unmanned aerial vehicles (UAVs) encircle around the specified target with prescribed performance constraints and minimum cost efforts. First, a basic enclosing controller is established to achieve the enclosing error stabilization and stable circumnavigation around a given target. Second, a new prescribed time behavior envelope that eliminates the availability on initial error is proposed. To render the satisfaction of performance constraints and optimal enclosing actions, a transformed enclosing error is obtained by enforcing state conversion on original error. Then, aiming at stabilizing the enclosing error to a prescribed accuracy within a prescribed time, a prescribed time learning-based near optimal enclosing controller under a critic-only adaptive dynamic programming (ADP) is explored, approximating the solution of a novel Hamilton-Jacobi-Bellman (HJB) equation via pursuing the minimum cost associated with transformed errors. Especially, a novel prescribed time learning rule driven by weight errors is elaborated by revisiting real-time and historical information, such that the convergence of weights is only determined by a user-defined time constant. The prominent merit is that the optimal enclosing with performance guarantees can be achieved by a prescribed time ADP. Lyapunov stability demonstrates that involved error variables are ultimately limited and resultant controller satisfies optimality. Finally, simulations verify the values and superiority of the proposed methodology.

Robotic tool usage still falls short in fine-grained task execution, specifically, learning how to select tools and determine configurations, achieving task compatibility, and enhancing subsequent manipulation remains challenging. Thus, this paper proposes a Graph Attention Grasp (GATGrasp) concept, which is dedicated to establishing a task-aware affordance grasping framework. It integrates a Visual-Language Model (VLM) for tool localization and segmentation across diverse scenes, while employing the GraspNet component to generate 6D candidates. The approach is then devoted to leveraging multimodal semantic and geometric features, learning the spatial relationships between objects and task attributes to address affordance grasp detection. Furthermore, we have also constructed a semantic knowledge graph to encode the mappings of affordance grasping, enabling the designed Graph Attention Network (GAT) to generalize tool usage reasoning for novel tasks under the guidance of semantic information. Evaluations on the TaskGrasp dataset demonstrate that GATGrasp outperforms the established baselines and state-of-the-art methods, while experiments conducted on a real robot further verify the effectiveness of our method in performing taskaware affordance grasps on novel tools.

With respect to the accurate tracking control for unmanned surface vehicles (USVs), this paper proposes a novel hierarchical control structure consisting of a velocity planner and a tracking controller. With full consideration of control accuracy and tracking safety, a model predictive control planning method based on dynamic artificial potential field method (DAPF-MPC) is proposed, aiming to generate the current optimal reference tracking velocity and achieve online obstacle avoidance guidance. Simultaneously, a fixed-time generalized super-twisting controller based on an extended state observer (ESO-FiTGST) is proposed to restrict the convergence time and tackle with challenge resulting from the uncertain disturbances and model parameters. Noticeably, the design process of the proposed control law and its rigorous stability analysis, especially for the computable fixed-time convergence property, are detailed. Ultimately, the simulation and experimental results demonstrate the superiority and feasibility of the proposed method, providing a reliable and effective reference solution for tracking control tasks of USVs in aquatic scenarios with complex obstacles.

Category-level 6D object pose estimation has gained increasing attention in applications of robotic manipulation, augmented reality, and scene understanding, due to its ability to generalize to unseen instances within the same category. However, existing methods struggle with handling the intra-class shape variations as they either adopt mean shape as priors, or build the associations among different instances without explicit category-shared information. To address this problem, a novel category-level object pose estimation method based on diffusion model is proposed, which utilize the generative ability of diffusion model to refine a sparse categorical representation. In contrast to existing dense correspondence-based methods, our method employs a set of keypoints provided by learnable queries to represent object shape, enabling better categorical representation of different instances by focusing on the representative object components. The keypoints are then refined through a forward diffusion process and a reverse denoising process conditioned on category information. This allows the flexible adaption to various instances, especially for those that deviate from the mean shape within the same category. On this basis, a geometric-semantic feature fusion module is presented to enhance keypoint feature representation. By integrating the geometric information from point cloud with the high-level semantics from RGB image using a two-branch attention mechanism, the keypoint feature is enriched and deeply combined, which facilitates the subsequent pose estimation. Extensive experiments on the REAL275 dataset, the CAMERA25 dataset, and real-world complex scenarios demonstrated the effectiveness of proposed method.

A novel collaborative position and orientation control scheme (CPOCS) for dual-arm robots is proposed, which is capable of controlling the end-effectors' positions with high precision while preserving their orientations unchanged to some practical tasks (e.g., box handling). To solve the proposed CPOCS in real time while considering key factors such as unknown bounded noise and strict time response constraints in practical engineering environments, this article proposes a novel precisely predefined-time convergent barrier recurrent neural network (PCB-RNN) based on a newly developed piecewise barrier evolution formula. Unlike existing RNNs, the proposed PCB-RNN, owing to its piecewise barrier evolution formula, can achieve precisely predefined-time convergence (PPTC) when addressing the proposed CPOCS under unknown bounded noise conditions. Comprehensive theoretical analysis rigorously proves the PPTC ability of the PCB-RNN under both noise-free and unknown bounded noise conditions. Furthermore, extensive simulation and physical experiments on dual-arm robots validate the effectiveness of the proposed CPOCS and demonstrate the advanced PPTC capability of the proposed PCB-RNN under unknown bounded noises.

Time-variant-gain zeroing neural networks (TVG-ZNNs) are among the most powerful solvers for time-variant linear matrix-vector inequalities (TVLMVIs). Although TVG-ZNNs with complex nonlinear activation functions achieve effective convergence within finite or predefined time, they incur high computational costs and face challenges in precisely predefining their actual convergence time. In contrast, TVG-ZNNs with linear activation functions offer lower computational costs but struggle to achieve convergence within a finite or predefined time. In addition, the gain values of most existing TVG-ZNNs tend to increase over time, resulting in a significant rise in computational costs. To address these contradictory issues, we propose a novel hybrid-gain ZNN without a nonlinear activation function (HG-ZNN-WNAF) to solve TVLMVIs in both noisy and noise-free environments. Specifically, a new hybrid gain is cleverly designed to construct the HG-ZNN-WNAF activated by a linear activation function, while ensuring that the gain value does not keep increasing over time. Unlike the state-of-the-art TVG-ZNNs with or without nonlinear activation functions, our proposed HG-ZNN-WNAF achieves precisely predefined-time convergence due to the hybrid gain, meaning its actual convergence time can be accurately predefined. Additionally, the piecewise design of the hybrid gain, along with the use of the simple linear activation function, effectively reduces the model's computational cost. Rigorous theoretical analysis demonstrates the precisely predefined-time convergence ability of the HG-ZNN-WNAF in both noisy and noise-free environments. Simulation and physical experiments validate the theoretical analysis and demonstrate that the HG-ZNN-WNAF achieves state-of-the-art performance in terms of convergence speed, robustness, and computational cost.

Target search and tracking (SAT) is a fundamental problem for various robotic applications such as search and rescue and environmental exploration. This article proposes an informative trajectory planning approach, namely, reusable belief tree search with sigma point-based mutual information reward approximation (ReSPIRe), for SAT in unknown cluttered environments under considerably inaccurate prior target information and a limited sensing field of view (FOV). We first develop a novel sigma point (SP)-based approximation approach to fast and accurately estimate mutual information (MI) reward under non-Gaussian belief distributions, utilizing informative sampling in state and observation spaces to mitigate the computational intractability of integral calculation. To tackle the significant uncertainty associated with inadequate prior target information, we propose the hierarchical particle structure in ReSPIRe, which not only extracts critical particles for global route guidance, but also adjusts the particle number adaptively for planning efficiency. Building upon the hierarchical structure, we develop the reusable belief tree search (RBTS) approach to build a policy tree for online trajectory planning under uncertainty, which reuses rollout evaluation to improve planning efficiency. Extensive simulations and real-world experiments demonstrate that ReSPIRe outperforms representative benchmark methods with smaller MI approximation error, higher search efficiency, and more stable tracking performance, while maintaining outstanding computational efficiency.

Existing approaches of hand reconstruction predominantly adhere to a multi-stage framework, encompassing detection, left-right classification, and pose estimation. This paradigm induces redundant computation and cumulative errors. In this work, we propose HandOS, an end-to-end framework for 3D hand reconstruction. Our central motivation lies in leveraging a frozen detector as the foundation while incorporating auxiliary modules for 2D and 3D keypoint estimation. In this manner, we integrate the pose estimation capacity into the detection framework, while at the same time obviating the necessity of using the left-right category as a prerequisite. Specifically, we propose an interactive 2D-3D decoder, where 2D joint semantics is derived from detection cues while 3D representation is lifted from those of 2D joints. Furthermore, hierarchical attention is designed to enable the concurrent modeling of 2D joints, 3D vertices, and camera translation. Consequently, we achieve an end-to-end integration of hand detection, 2D pose estimation, and 3D mesh reconstruction within a one-stage framework, so that the above multi-stage drawbacks are overcome. Meanwhile, the HandOS reaches state-of-the-art performances on public benchmarks, e.g., 5.0 PA-MPJPE on FreiHand and 64.6% [email protected] on HInt-Ego4D.

Multimodal task specification is essential for enhanced robotic performance, where Cross-modality Alignment enables the robot to holistically understand complex task instructions. Directly annotating multimodal instructions for model training proves impractical, due to the sparsity of paired multimodal data. In this study, we demonstrate that by leveraging unimodal instructions abundant in real data, we can effectively teach robots to learn multimodal task specifications. First, we endow the robot with strong Crossmodality Alignment capabilities, by pretraining a robotic multimodal encoder using extensive out-of-domain data. Then, we employ two Collapse and Corrupt operations to further bridge the remaining modality gap in the learned multimodal representation. This approach projects different modalities of identical task goal as interchangeable representations, thus enabling accurate robotic operations within a well-aligned multimodal latent space. Evaluation across more than 130 tasks and 4000 evaluations on both simulated LIBERO benchmark and real robot platforms showcases the superior capabilities of our proposed framework, demonstrating significant potential in overcoming data constraints in robotic learning. Website: zh1hao.wang/Robo_MUTUAL

The semi-supervised medical image segmentation with a few annotated data can provide significant help in robot-assisted surgery. This step plays a pivotal role in identification of pathological regions, more appropriate planning of surgical procedures, and so on. In this work, we develop an evidence-based tri-branch cross-pseudo supervision model, which integrates evidence-based uncertainty estimation and multi-branch cross supervision to bolster the effectiveness of semi-supervised learning. The overall framework consists of a vanilla network and an evidential dual-branch network. Two evidential branches EPB and ERB are proposed to complement each other and improve the quality of pseudo-labels. The EPB places more focus on classification accuracy at the pixel level and the ERB emphasizes the similarity and overall integrity of the segmented regions. Then, a novel cross-pseudo supervision strategy among the three branches is designed, to guarantee that valuable and diverse unlabeled knowledge is explored and transferred for segmentation improvement. The effectiveness of the proposed method was verified on the ACDC dataset, achieving outstanding performance compared with other state-of-the-art methods. In addition, we conducted ablation study to validate the effectiveness of the evidential branches (EPB and ERB) and tri-branch cross-supervision strategy, respectively.

Tibetan Jiu chess, recognized as a national intangible cultural heritage, is a complex game comprising two distinct phases: the layout phase and the battle phase. Improving the performance of deep reinforcement learning (DRL) models for Tibetan Jiu chess is challenging, especially given the constraints of hardware resources. To address this, we propose a two-stage model called JFA, which incorporates hierarchical neural networks and knowledge-guided techniques. The model includes sub-models: strategic layout model (SLM) for the layout phase and hierarchical battle model (HBM) for the battle phase. Both sub-models use similar network structures and employ parallel Monte Carlo tree search (MCTS) methods for independent self-play training. HBM is structured as a hierarchical neural network, with the upper network selecting movement and jump capturing actions and the lower network handling square capturing actions. Human knowledge-based auxiliary agents are introduced to assist SLM and HBM, simulating the entire game and providing reward signals based on square capturing or victory outcomes. Additionally, within the HBM, we propose two human knowledge-based pruning methods that prune parallel MCTS and capture actions in the lower network. In the experiments against a layout model using the AlphaZero method, SLM achieves a 74% win rate, with the decision-making time being reduced to approximately 1/147 of the time required by the AlphaZero model. SLM also won the first place at the 2024 China National Computer Game Tournament. HBM achieves a 70% win rate when playing against other Tibetan Jiu chess models. When used together, SLM and HBM in JFA achieve an 81% win rate, comparable to the level of a human amateur 4-dan player. These results demonstrate that JFA effectively enhances artificial intelligence (AI) performance in Tibetan Jiu chess.

This paper focuses on the design of event-triggered controllers for the synchronization of delayed Takagi–Sugeno (T–S) fuzzy neural networks (NNs) under deception attacks. The traditional event-triggered mechanism (ETM) determines the next trigger based on the current sample, resulting in network congestion. Furthermore, such methods suffer from the issues of deception attacks and unmeasurable system states. To enhance the system stability, we adaptively detect the occurrence of events over a period of time. In addition, deception attacks are recharacterized to describe general scenarios. Specifically, the following enhancements are implemented: First, we use a Bernoulli process to model the occurrence of deception attacks, which can describe a variety of attack scenarios as a type of general Markov process. Second, we introduce a sum-based dynamic discrete event-triggered mechanism (SDDETM), which uses a combination of past sampled measurements and internal dynamic variables to determine subsequent triggering events. Finally, we incorporate a dynamic output feedback controller (DOFC) to ensure the system stability. The concurrent design of the DOFC and SDDETM parameters is achieved through the application of the cone complement linearization (CCL) algorithm. We further perform two simulation examples to validate the effectiveness of the algorithm.

Robust moving object pose estimation is crucial in fine manipulation tasks, such as surgical instrument tracking. This paper presents a distributed-camera system with robotic adjustments to maintain consistent tracking of moving objects, thus avoiding tracking failures. An integrated framework for camera adjustment and pose estimation is developed for this distributed-camera system. In each detection cycle, the camera exhibiting the largest deviation with the object is adjusted by a visual servoing technique. After adjustment, the camera extrinsics are re-calibrated in the following detection cycles. For the unadjusted cameras, an online extrinsic optimization method based on multi-frame detection results is proposed to refine the camera extrinsics. Based on the refined camera extrinsics and detection results from multiple cameras, the pose of moving objects relative to the principal camera can be robustly estimated. We test the performance of this system in both simulation environments and real-world scenarios. The results indicate that our system achieves higher pose estimation accuracy and exhibits strong resistance to limited field-of-view (FoV) compared to conventional equivalent fixed multi-camera systems.

The development of an accurate and robust positioning system for oral and maxillofacial surgery (OMS) is a challenging task, primarily due to the oral space limitations and line-of-sight occlusions. This paper presents a novel lightweight integrated positioning system for OMS, which can provide practical guidance utilizing only a micro camera installed on the end of the surgical instrument. An efficient region-based pose tracking method for texture-less teeth is proposed, which can use search lines around the object contour and simple local region partitioning strategy to improve pose accuracy. Besides, to deal with the possible partial occlusions of target during surgery, an occlusion-aware weight function is presented and utilized seamlessly in the pose optimization pipeline. This function calculates the pixel-wise occlusion probability using object contour and distance constraint, helping to improve the tracking robustness. Pivot calibration evaluation reveals that the tracking accuracy of the proposed camera-based handpiece is higher than the marker-based handpieces. Comparative experiments demonstrate that proposed pose tracking method has higher accuracy than existing state-of-the-art methods and ablation study confirms the effectiveness of the occlusion handling strategy. The overall positioning experiment indicates that the proposed system has satisfactory static poses stability and positioning accuracy. Furthermore, the main advantage of our system is that it is lighter and more integrated than other systems, which can reduce the system complexity, decrease the risk of line-of-sight occlusion, and lower the surgery cost.Note to Practitioners—This paper is motivated by the problem of restricted oral space constraints and partial occlusions during positioning for OMS. Compared with traditional OMS navigation systems, the designed system is more lightweight and more integrated without other external cameras and additional fiducial markers. Our system can provide practical guidance utilizing only a micro camera installed on the end of the surgical instrument. In addition, an efficient region-based pose tracking method for texture-less teeth is proposed to increase pose accuracy. Since the target can partially be occluded during the procedure, we present a novel occlusion-aware strategy to improve the tracking performance of partial occlusions. Our proposed system achieves a decent balance between positioning accuracy and hardware cost, and can easily be integrated into various dental surgical tools, thus it has tremendous potential for commercialization.

For field tasks, it is quite challenged to operate in a complex environment for the underwater robots, especially for those with multiple control surfaces due to different response and gain characteristics. To this end, this paper develops a highly integrated robotic dolphin followed by a robust motion control system. For better maneuverability and fault-tolerant capabilities, a newly-designed robotic dolphin is presented, owning a wide array of sensors and multiple control surfaces, in which passive flukes are particularly applied. On this basis, a robust motion control system is proposed, including a depth controller based on velocity-related allocation strategies and a heading controller based on clearance compensation. In detail, considering the degradation of motion performance caused by passive flukes, a sliding mode controller for gain uncertainty and an allocation-related parameter tuning strategy for inputs response characteristics are designed. Extensive simulations and aquatic experiments are conducted, and the obtained results demonstrate the satisfied maneuverability of the designed prototype and the effectiveness of the proposed methods. This study can lay a foundation for further development of robotic dolphins with a robust motion system to execute complex tasks in the field.Note to Practitioners—This paper is inspired by the issue of robust motion control system for a newly-designed practical robotic dolphin that possesses a passive tail and redundant control surfaces. The traditional methods are usually susceptible to uncertainties in the passive tail gain, exhibiting degraded control performance. Moreover, control oscillations and slow convergence speed often occur caused by neglecting the characteristics of different control surfaces, including response patterns and clearance. This paper suggests a robust depth controller based on velocity-related allocation strategies and a robust heading controller based on clearance compensation. Specifically, an allocation-related parameter tuning strategy is given by considering inputs response characteristics, including response speed, saturations, and hydrodynamic force variation patterns. To guarantee fine regulations of heading control, a nonlinear disturbance observer (NDOB)-based clearance compensation is proposed. Extensive aquatic experiments on the newly-designed robotic dolphin verified the effectiveness of the proposed methods. It is envisioned that all these presented results can provide valuable engineering practice insights for industry practitioners.

This paper presents a two-step method to automatically and accurately segment the dental crown components from CT images. Firstly, a multi-scale attention based U-Net model is proposed for pulp segmentation, which is embedded with global and local attention modules. The constructed attention modules can automatically aggregate pixel-wise contextual information and focus on catching the real dental pulp region. Secondly, two efficient level set models are proposed: one is the shape constraint-based level set model for enamel and dentin segmentation, the other is the region mutual exclusion-based level set model for neighboring teeth segmentation. The proposed shape constraint term can better handle topology changes of teeth and the region mutual exclusion term can more effectively avoid intersecting segmentation. Besides, a starting slice initialization method is introduced to achieve automatic segmentation, and an accurate contour propagation strategy is developed for slice-by-slice segmentation. We set up a series of comparative experiments for evaluation. Experimental results verify that the proposed method obtains promising performance for each crown component segmentation, and outperforms state-of-the-art tooth segmentation methods in terms of accuracy. This suggests that the proposed method can be used to accurately segment the crown components for precise tooth preparation treatment.Note to Practitioners—The motivation of this work is to reduce the burden on dentists during tooth preparation treatment, which requires accurate segmentation of crown components (i.e., enamel, dentin, and pulp) from dental CT images. Existing methods only focused on the segmentation of teeth or alveolar bone. Therefore, we present a novel automatic segmentation model for the dental crown components with high accuracy. A key strength of this study is the combination of a data-driven method (deep learning) and model-driven methods (level-set), which can provide good accuracy under limited training samples. This ability is highly desirable for practitioners by saving labor-intensive, costly labeling efforts. Furthermore, our proposed method will provide tools to help reduce subjectivity and human errors, as well as streamline and expedite the clinical workflow. This will significantly facilitate tooth preparation automation.

Gait planning is one of the most critical technologies for quadruped robots. However, far too little attention has been paid to the tight coupling mechanism of gait planning with terrain understanding and energy efficiency. To date, it is still challenging to plan optimal gait strategies that are highly adapted to terrain features with stable and efficient transitions. Accordingly, this paper proposes an adaptive gait control framework for quadruped robots that combines terrain recognition, Cost of Transport (CoT), and the Froude (Fr) number. More specifically, an optimal gait selection strategy for quadruped robots is designed based on different terrain texture features and the CoT characteristics of different gaits. To address the gait transition process induced thereby, an adaptive method for gait parameters based on the Fr number is further proposed, which can make the process more stable. Besides, model predictive control (MPC) and whole-body control (WBC) are employed as the motion controllers for the quadruped robot. Furthermore, simulation and experimental results indicate that the proposed method possesses superior terrain adaptability, energy efficiency, and motion stability during gait transitions, which is beneficial for the quadruped robots to maintain stable motion and reduce energy consumption when performing tasks in changeable terrains.Note to Practitioners—This paper is motivated by the problem of adaptive gait planning for quadruped robots that walks through different terrains. We propose a method that ensures optimal gait selection by robots facing diverse terrains and maintains the stability of gait transition. The proposed control framework, upon testing in a simulated environment, can be directly deployed on real-world robot without further adjustments and allows the robot to traverse various terrains with minimal sim-to-real issues. Hopefully, our proposed method can provide valuable guidance and support for facilitating the enhancement of capabilities in performing prolonged endurance tasks in unstructured environments for quadruped robots.

The development of bionic underwater robots has brought new vitality to ocean exploration. Motion control is crucial for the stability of underwater robots due to significant differences in flow field characteristics at various swimming speeds. This study focuses on vertical-plane motion and proposes a model predictive control method to achieve integrated control of depth position and pitch attitude for bionic robotic fish. First, based on a robotic tuna system, high-maneuverability vertical-plane motion configuration elements are analyzed and summarized, laying the foundation for motion stability and controllability. Second, through hydrodynamic sampling in aquatic environments, a system model covering the range of swimming speeds is established. Regarding the control method, the proposed motion planning approach converts the desired motion sequence into an equivalent “pitch-depth” trajectory curve. A nonlinear model predictive controller (NMPC) is then designed to track the trajectory curve, ultimately achieving the desired vertical-plane motion. Experimental results validate that the proposed method not only ensures control accuracy under both low and high-speed conditions, but also enables the execution of complex motion sequence control. This study provides a fresh perspective on the motion instability analysis of robotic fish at high swimming speed and a novel control framework for regulating continuous posture sequences in the vertical plane.Note to Practitioners—The motivation of this paper is to address the challenges associated with stable motion and control of robotic fish in the vertical plane, given the variability of flow field characteristics at different swimming speeds. Existing methods for controlling pitch attitude and depth in bionic underwater robots are typically designed for stable flow conditions encountered during low-speed swimming. However, the instability and agility of high-speed robotic fish movements have not been adequately considered. Additionally, the coupling between pitch attitude and depth poses challenges for joint control of their combined states. This paper proposes a configuration analysis and control methodology to achieve desired vertical-plane locomotion for robotic fish. Specifically, using a robotic tuna as the research subject, a configuration analysis method for high-maneuverability motion in the vertical plane is presented, providing a foundation for ensuring motion stability and controllability. To accurately evaluate the motion of robotic fish under varying flow speeds, a system model for vertical plane motion is constructed based on hydrodynamic data collected from aquatic environments. A motion planning approach is proposed to convert desired vertical plane motion sequences into controllable “pitch-depth” trajectory curves, and a nonlinear model predictive controller is designed to track these trajectories. Configuration simulations and control experiments validate the effectiveness of the proposed method. Hopefully, our proposed methods can provide valuable insights and support for high-maneuverability motion control and continuous posture sequences tracking of bionic underwater robots in the vertical plane.

Information-based autonomous robot exploration methods, aiming to maximize the exploration rewards, e.g., mutual information (MI), get more prevalent in field robotics applications. However, most MI-based exploration methods assume known poses or use inaccurate pose uncertainty approximation, which may lead to deviation or even failure when exploring prior unknown environments. In this paper, we explicitly consider full-state (pose & map) uncertainty for balancing exploration and localizability, i.e., avoiding the robot guiding itself to complex scenes with high exploration rewards but hard to localize. We first propose a Rao-Blackwellized particle filter-based localization and mapping framework (RBPF-CLAM) for a dense environmental map with continuous occupancy distribution. Then we develop a new closed-form particle weighting method to improve the localization accuracy and robustness. We further use these weighted particles to approximate the unknown pose uncertainty and combine it with our previous confidence-rich mutual information (CRMI) metric to evaluate the expected information utility of the robot’s new control actions. This new information metric is calleduncertainCRMI (UCRMI). Dataset experiments show our RBPF-CLAM improves about 44.7% average root mean square error than the state-of-the-art RBPF localization method, and real-world experimental results show that our UCRMI reduces the pose uncertainty about 32.85% more than CRMI and 25.36% time cost than UGPVR in the exploration of unknown and unstructured scenes given sparse measurements, which shows better performance than other state-of-the-art information metrics.Note to Practitioners—This work was motivated by the problem of ‘planning for state estimation’ for a range-sensing robot, i.e., the robot can choose a better future place to facilitate its localization more accurately and explore new areas rationally to gather more information. Existing methods mainly assume the robot’s poses during the exploration can be estimated by an independent localization approach or simply propagated via a predefined probabilistic distribution. However, localization failure would lead to higher planning deviation for the planner that does not consider the pose uncertainty, and manually set parametric distribution is more prone to overestimate the pose uncertainty. This paper proposes an RBPF-based localization and mapping scheme and an improved particle weight update method in a confidence-rich map, then uses the weighted particles to approximate trajectory entropy and combines it with CRMI to evaluate the expected information gain of a candidate action/node. Our newly defined information function ‘UCRMI’ can prevent the robot from exploring too aggressively without considering its localizability in prior unknown and unstructured environments. These scenes may lack robust features to conduct feature-based SLAM or lack accurate external localization information such as GPS. This method can be applied in underwater, planetary, and subterranean robot exploration tasks, even using low-resolution sensors. Future work mainly involves adapting UCRMI to applications in large-scale scenes using small autonomous platforms.

Directly learning the swimming behaviors of real fish can significantly enhance the swimming performance of bionic robotic fish. This paper presents a novel transfer learning method based on a dynamic trajectory control approach for the robotic fish to learn swimming skills from real fish. First, we develop a fish motion capture system and a crucial motion extraction approach to realize precise decomposition of fish motions and collect abundant meaningful features from a snakehead fish as pre-training data. Next, we construct a two-stage transfer learning method based on Deep Deterministic Policy Gradient (DDPG), including an offline and an online stage. Specifically, in the offline stage, the obtained pre-training data is processed for experience learning within a DDPG-based network, whereas in the online stage, a dynamic trajectory tracking method is utilized to refine the robotic fish’s motions in real time based on the learned strategies. Experimental results on a self-developed four-joint robotic fish show that the proposed method effectively extracts and transfers biological motion features into the motion control of the robotic fish. Compared to the conventional CPG method, the proposed approach exhibits stronger acceleration capabilities and more efficient swimming, resulting in enhanced maneuverability of the robotic fish. Overall, this approach provides a technical foundation for bionic robotics to learn from nature.

The high-quality datasets and generalized network model combined with robust evaluation strategies serve as a significant benchmark for developing new policies for industrial bin-picking. In this paper, we propose the concept of region-aware grasping, a sim2real cutting-edge system to generate and evaluate 6D poses for robots to pick up novel workpieces from stacked environments. It consists of Region-Aware-Dataset, a large-scale synthetic point cloud grasp dataset; and Semantic-Graspnet, a 6D-wise affordance policy that predicts full 6D grasp pose for stacked workpieces. The introduced Semantic-Graspnet transforms the 6D pose prediction problem into semantic categorization via point cloud encoding and decoding. Meanwhile, we propose a robust evaluation strategy based on pose evaluation and mechanical grasping evaluation, which enhances the robot’s grasping success rate and sorting efficiency. In real industrial tasks, the robot achieves a grasp completion rate of 91.3% in cluttered scenes and 89.2% in densely stacked scenes, demonstrating state-of-the-art results in robotic picking-and-placing applications.

Autonomous grasping is a critical topic of robotic embodied intelligence. However, it remains challenging for robots to grasp an intended target, particularly in cluttered and densely stacked environments. This paper presents a novel solution by proposing a Robot Intention Grasp Network (RIGNet) with a multi-task siamese schema, which is based on an improved Region Proposal Network (RPN) and a Region of Interest (RoI) learning approach. More specifically, the RPN robustly outputs the RoIs to describe the candidates in stacked scenes, while the multi-task siamese network consisting of category comprehension and grasp perception modules serves to detect the intended object and generate the optimal grasp configuration. To improve the reasoning precision for grasp posture, a dynamic-oriented anchor matching strategy is further proposed to adapt to the grasp perception module. The proposed RIGNet is comparable to the state-of-the-art algorithms on single-object tasks and has better performance on stacked multi-object grasp detection. This study can provide new insights for autonomous robot reasoning in real time, facilitating an understanding of both the rationale and methodology for grasping an intended target in highly stacked environments. Note to Practitioners—This work is motivated by robotic embodied intelligence technique, which plays an irreplaceable role in robot autonomous grasping manipulation. Using this technology, the robot is capable of performing grasp tasks for human. The previous grasp detection methods focus on optimizing grasp posture for successful robot grasping manipulation as maximum as possible, but ignore the comprehension of candidate object’s attributes. Hence, this paper proposes a novel robot intention grasp network (RIGNet) for robot perception of how to grasp and comprehend why grasping. The improved RPN module robustly predict the RoIs to locate candidates in a bunch of overlap and occlusion objects, and then the multi-task siamese module finely detect the object category corresponding to the grasp configuration. What is more important, the proposed technology does not lose efficiency in single-object grasping, and even outperforms the state-of-the-art algorithms in multi-object stacked grasping. The novel RIGNet schema is suitable for human intends a real robot to execute desired grasp tasks.

In this paper, the dual spacecraft pursuit-evasion game problem under incomplete information is investigated, and a strategy-solving method for the incomplete information pursuit-evasion game based on particle swarm optimization and unscented particle filter (PSO-UPF) estimation is proposed. The completeness of the information available about the target’s cost function, which is determined by the weighting information, has a significant impact on the success of the pursuing strategy. For the cost function is unknown in incomplete information scenarios, a research framework of the pursuit-evasion game based on following observation and one-sided pursuit two stages is established. Besides, to describe the more accurate motion of the spacecraft, a Schweighart-Sedwick (SS) dynamic model is introduced that considers the effect ofJ2perturbation. Firstly, an equilibrium strategy for the SS model-based pursuit-evasion problem is derived under complete information. Next, for the incomplete information scenarios, an estimation method based on PSO-UPF of weight matrix information is established, which allows the cost function to be determined by the estimation method in the observation stage. Then, the pursuit strategy is re-designed in the one-sided pursuit stage based on the estimated cost function. Finally, the performance of the proposed method is validated by simulation. The results demonstrate that the approach can achieve good performance by efficiently estimating the weight information in the opponent’s cost function.

Achieving precise depth control of autonomous underwater vehicles (AUVs) poses a considerable challenge owing to packet losses and ocean current disturbances. Consequently, there is an urgent necessity to investigate methods for anti-disturbance depth control of AUVs in scenarios that involve packet losses. In contrast to traditional disturbance estimation techniques that depend on complete state information, which rely on complete states, we propose an intermittent information-based disturbance observer, which leverages most recent data packet to compensate for any current data losses. The novel feature of this new anti-disturbance control lies in its consideration of the issue of information intermittency and its dynamic selection of gains with or without packet losses. Through integration of estimated disturbances, a PI recursive controller has been developed to effectively track desired depth of AUVs. In addition, a stability criterion has been established using the Lyapunov stability theory and stochastic analysis techniques, with calculation of the possibility that the Lyapunov function is bounded through Chebyshev’s inequality. Finally, comprehensive simulations have been performed to demonstrate the tracking effectiveness of the proposed algorithm, and its real-world applicability and reliability have also been confirmed through experimental trials.

Visual place recognition is crucial to accurate localization in large-scale environments. Existing methods combine convolutional neural network and deep metric learning to improve performance, however, it is still challenging to promote the adaptability of features under various environmental conditions. To address the problem, this paper proposes a hardness-aware metric learning method with cluster-guided attention. By leveraging the affinity between each local feature and the corresponding scene cluster center, the model is attracted to focus on the local features proximate to their clusters, while suppressing outlier local features that deviate from their clusters. In this way, the scene-related reliable local features are concentrated to construct the global feature of the whole image with adaptability to environmental conditions. Meanwhile, a hardness-aware metric loss is designed to train the proposed network, which determines the hardness of negative samples based on their similarity to the query images and the training iterations. Subsequently, the hardness is used to reweigh each term of the loss function to promote network optimization. In addition, a condition normalization layer is also introduced to regularize the feature distributions under different environmental conditions to a canonical space, improving the feature robustness to condition variations. Our method achieves the top-10 recalls of 97.2%, 99.0%, and 94.9% on Pitts250k-test, TokyoTM-val, and Tokyo 24/7 datasets, respectively. Extensive experiments demonstrate that the proposed method learns robust global features with the adaptability to various environmental conditions.

In this article, we propose a tightly coupled visual-inertial-acoustic sensor fusion method to improve the autonomous localization accuracy of underwater vehicles. To address the performance degradation encountered by existing visual or visual-inertial simultaneous localization and mapping systems when applied in underwater environments, we integrate the Doppler velocity log (DVL), an acoustic velocity sensor, to provide additional motion information. To fully leverage the complementary characteristics among visual, inertial, and acoustic sensors, we perform multimodal information fusion in both frontend tracking and backend mapping processes. Specifically, in the frontend tracking process, we first predict the vehicle's pose using the angular velocity measurements from the gyroscope and linear velocity measurements from the DVL. Thereafter, measurements performed by the three sensors between adjacent camera frames are utilized to construct visual reprojection error, inertial error, and DVL displacement error, which are jointly minimized to obtain a more accurate pose estimation at the current frame. In the backend mapping process, we utilize gyroscope and DVL measurements to construct relative pose change residuals between keyframes, which are minimized together with visual and inertial residuals to further refine the poses of the keyframes within the local map. Experimental results on both simulated and real-world underwater datasets demonstrate that the proposed fusion method improves the localization accuracy by more than 30% compared to the current state-of-the-art ORB-SLAM3 stereo-inertial method, validating the potential of the proposed method in practical underwater applications.

Zeroing neural network (ZNN), as a special type of recurrent neural network (RNN), is very competitive in solving time-varying linear matrix-vector equations. Recently, various ZNNs with predefined-time convergence (PTC) capabilities have been reported. Such ZNNs with PTC capabilities can achieve the predefined convergence time via explicitly presetting multiple parameters related to the upper bounds of their convergence time. However, obtaining suitable and robust values for these parameters through reasonable adjustments is a challenging task in many engineering applications. To address this problem, we propose a novel arbitrarily predefined-time convergent RNN (APTC-RNN) with a novel nonlinear piecewise activation-function (NPAF). Unlike most existing ZNNs with PTC capabilities, the proposed APTC-RNN, due to its NPAF, can achieve arbitrarily PTC (APTC) without adjusting any upper bound parameters. Furthermore, due to the piecewise computation form of the NPAF, the proposed APTC-RNN can provide a lower computational cost compared to most existing RNNs. The stability and APTC capability of the proposed APTC-RNN are proven by rigorous theoretical analysis and mathematical derivation. Numerical simulations show that APTC-RNN has faster and more accurate PTC capability than three state-of-the-art RNNs, while having less computational time. Finally, the practicality of the APTC-RNN is verified by applying it to the UR3 robotic arm and multiagent systems.

Digital twins and visual monitoring of conveyor systems require accurate digital models of dynamic bulk material flows, but existing methods struggle to achieve both speed and precision. This study develops a rapid online method to reconstruct dynamic bulk material flows on conveyor belts. First, a standardized online reconstruction scheme using visual detection of material flow contour lines is presented. Then, a feature detection algorithm is proposed to extract more refined points from laser line skeleton to accelerate the reconstruction process. An iterative-filtering interpolation algorithm that generates smooth interframe point clouds is introduced to improve mesh quality. Experimental results demonstrate that our method outperforms traditional corner detection-based reconstruction techniques in feature point detection, accuracy, mesh quality, and runtime performance. This research provides a practical solution for material handling digitalization, promoting the advancement of conveyor system digital twins and potentially improving operational efficiency and predictive maintenance in bulk material handling industries.

This article presents an innovative visual method for measuring material mass online by quantified conveyor belt deformation with deep learning, which offers a noncontact and safe alternative to traditional pressure- and radioactivity-based weighing techniques. The correlation between the belt deformation and the carried material mass is further investigated through finite element simulations. Then, a visual weighing method by belt deformation is proposed, comprising a calibration algorithm to construct a measurement model using a gated recurrent unit-based network, and an online measurement algorithm to calculate material mass with the trained network. Finally, a case study is presented to analyze the effect of different dimension configurations and networks. The results validate that the proposed method attains a notable accuracy and is suitable for high-velocity conveyor environments. The demonstrated benefits signify an advancement in visual perception of materials, enabling a new approach for intelligent operation and monitoring in material handling field.

Tracking accuracy is critical for practical applications. However, due to the presence of various disturbances and intermittent information caused by packet losses or network attacks, ensuring tracking precision and speed becomes extremely challenging. Consequently, this article proposes a novel control architecture for output tracking with intermittent information, which includes an intermittent information-based disturbance observer (IIBDO) and a flexible antidisturbance control strategy based on conditional disturbance utilization (CDU). The novel IIBDO addresses the issue of information intermittency by compensating for lost information using available signals and their trends. Building on this, a disturbance diagnosis condition (DDC) is introduced to assess whether disturbances are beneficial to system performance. Through integration of DDC and disturbance estimation, the CDU-based flexible antidisturbance control is designed, enabling the system to utilize disturbances rather than merely rejecting them. Sufficient conditions are derived to ensure both tracking and antidisturbance performance, and the potential stabilizing effects of disturbances on the system are also analyzed using Lyapunov stability theory. Finally, comprehensive simulations and experiments confirm the effectiveness of the IIBDO, and the improvement in tracking performance brought about by CDU is also verified.

In the field of multisensor fusion for simultaneous localization and mapping (SLAM), monocular cameras and IMUs are widely used to build simple and effective visual-inertial systems. However, limited research has explored the integration of motor-encoder devices to enhance SLAM performance. By incorporating such devices, it is possible to significantly improve active capability and field of view (FOV) with minimal additional cost and structural complexity. This article proposes a novel visual-inertial-encoder tightly coupled odometry (VIEO) based on a video detection and ranging (ViDAR) device. A ViDAR calibration method is introduced to ensure accurate initialization for VIEO. In addition, a platform motion decoupled active SLAM method based on deep reinforcement learning (DRL) is proposed. Experimental data demonstrate that the proposed ViDAR and the VIEO algorithm significantly increase cross-frame co-visibility relationships compared to its corresponding visual-inertial odometry (VIO) algorithm, improving state estimation accuracy. Additionally, the DRL-based active SLAM algorithm, with the ability to decouple from platform motion, can increase the diversity weight of the feature points and further enhance the VIEO algorithm's performance. The proposed methodology sheds fresh insights into both the updated platform design and decoupled approach of active SLAM systems in complex environments.

Pedestrian trajectory prediction is crucial for intelligent surveillance, social robot navigation, and autonomous driving systems, attracting substantial research attention in recent years. Despite significant advances, accurate trajectory prediction remains challenging due to the inherent uncertainty in pedestrian intentions and the multimodal nature of human movement patterns. There remain two limitations in existing methods. First, they focus solely on predicting final goals while overlooking crucial intermediate intentions that guide pedestrian movement. Second, they utilize a static latent distribution model across all future timesteps, which fails to capture the dynamic and evolving nature of trajectory uncertainties as pedestrians move. To address these challenges, we propose a novel timewise intentions and time-varying distribution network, TITDNet, which can estimate pedestrian intentions over time while dynamically modeling trajectory uncertainties at each future timestep. Specifically, TITDNet includes two key components: an intention generator that estimates dynamic pedestrian intentions, and a variational autoencoder that captures the time-varying multimodal nature of future trajectories. A trajectory decoder then integrates historical movement patterns, predicted intentions, and learned distributions to generate accurate future trajectories. Extensive experiments on ETH, UCY, and SDD benchmark datasets demonstrate that our approach significantly outperforms the state-of-the-art methods.

In this study, the task assignment problem for heterogeneous underwater vehicle collaborative system, which involves autonomous underwater vehicles (AUVs) and underwater gliders (UGs), is studied for high-efficiency data collection. UGs and AUVs show different motion modes. The advantages of different motion modes can be mutually complemented to achieve the preference-matched task assignment results, which show greatly promising prospect to enhance the data collection efficiency. Most of existing underwater task assignment algorithms focus on the single-type vehicles, which can not be applied to the heterogeneous system. To address this issue, a hierarchical learning algorithm is proposed. Firstly, based on the evaluated emergency degree of tasks, the preliminary-task-assignment hierarchy is proposed to assign the emergency tasks to AUVs and assign the non-emergency tasks to UGs, thereby achieving the preference-matched task assignment. Therefore, the collaborative efficiency of heterogeneous system can be enhanced. Then, in the UG-task-assignment hierarchy, the adaptive serial cluster mechanism is proposed to extract the high utility task-connectivity regions for UGs, thereby fully leveraging the UG advantages in region data collection. Furthermore, in the AUV-task-assignment hierarchy, the extended self-organizing mapping neural network is constructed to eliminate the disorganization of neuronal loops. As a result, the crossed paths of AUVs can be excluded to reduce the energy consumption. Finally, the superior performance is verified by numerical results.

A novel binary channel fuzzy self-adjusted neural network (BCF-SANN) is proposed and researched for solving time-changing quadratic programming (QP) problems in this article. Unlike the fixed parameters of the typical zeroing neural network, the main parameters of the proposed BCF-SANN are time-changing, and its errors are adaptively quickly convergent. The biggest advantage of the novel neural network is that it combines a fuzzy self-adjusted controller, which takes the errors and derivatives of errors as fuzzy inputs and neural networks, further improving the convergence and robustness of the neural networks. To design the novel neural network, a time-changing QP problem is first established; then, using Lagrange's law, the time-changing QP problem is transformed into a time-changing matrix equation; and finally, based on the time-changing parameter neural dynamics method, a novel BCF-SANN is proposed. The detailed design process is given in this article, and the convergence and robustness of the proposed BCF-SANN are proved by theoretical analysis. Through comparative experiments, it is demonstrated that the proposed BCF-SANN has a faster convergence rate and stronger robustness than the traditional zeroing neural network and 1-D fuzzy recurrent neural network (RNN).

Aiming at the situation where the measurement matrix B has a flexible block decomposition, this article designs two novel distributed continuous- and discrete-time projection neurodynamic approaches to solve the basis pursuit (BP) problem for sparse recovery. These approaches only require information from each flexible block of the measurement matrix B, rather than from each row, column, or the entire matrix. First, with the aid of the primal-dual dynamical approach, projection operator, and second-order multiagent consensus condition, a novel distributed projection neurodynamic approach in continuous time (DPNA-CT-B) is proposed, and its optimality and global asymptotic stability are rigorously proved. Moreover, based on the forward and backward Euler methods and variable substitution methods, a corresponding distributed projection neurodynamic approach in discrete time (DPNA-DT-B) is designed. Finally, through sparse signal and image reconstruction experiments, the effectiveness and superiority of the proposed neurodynamic approaches are verified.

This paper investigates a joint discrete-channel and continuous-power allocation problem for multi-user underwater acoustic communication networks. The unknown underwater acoustic Channel State Information (CSI) and the distributed optimization requirement make the proposed hybrid discrete-continuous optimization problem full of challenges. Firstly, an adversarial multi-player bandit game model is formulated, which enables each user to independently optimize its own strategy, thereby achieving the distributed decision. In the strategic game, the Multi-armed Bandit (MAB) learning theory is exploited to achieve the best response strategy of independent user without prior CSI. Secondly, an evolutive finite discrete strategy pool learning structure is proposed to achieve an efficient search for the hybrid discrete-continuous space. The constant evolvement of strategy pool endows the proposed MAB-based algorithm with the ability to search the whole continuous power space, thereby avoiding missing the superior strategy caused by the discretization of continuous space. Thirdly, a selection probability setting rule is proposed, which promotes the exploration-exploitation balance for the dynamic strategy pool, thereby improving the learning efficiency. Finally, simulation results demonstrate the superiority of the proposed algorithm.

In nature, fish have evolved sophisticated muscular systems that enable them to dynamically regulate their body movements for efficient and agile swimming, which has inspired the development of compact and fast flexibility regulation mechanisms in robotic fish. While existing robotic fish have primarily relied on passive flexible mechanisms and tunable stiffness mechanisms, these approaches often lack the dynamic adjustment capabilities that are characteristic of living fish. This article proposes a novel biomimetic flexible fishtail capable of dynamically controlling its deformation through artificial muscles made from macrofiber composite. In detail, the fishtail is equipped with a servo motor as the sole driving joint, while the artificial muscles regulate the deformation to indirectly adjust stiffness. A dynamic model considering both flexibility and hydrodynamics is established, and a partial differential equation observer is particularly developed to estimate the tail's full states. Subsequently, a deformation control framework incorporating a deep reinforcement learning strategy is constructed and successfully deployed on an embedded platform via lightweight design. Simulation and experimental results validate the accuracy and effectiveness of the dynamic model, observer, and control strategy. Especially, the proposed fishtail demonstrates the ability to enhance propulsion in fishlike swimming modes across various frequencies, ranging from 15% to 203%. When assembled into an untethered robotic prototype, deformation control allows the prototype's swimming speed to vary, achieving up to 42% slower or 37% faster speeds compared to passive compliance. Its rapid adjustability and adaptability to different frequencies represent significant advancements not widely reported in previous studies. The obtained results will offer some significant insights for flexible robotic systems to enhance their agility and interactivity.

This article addresses the problem of designing an attack-resilient adaptive event-triggered (AET) controller for complex dynamical networks (CDNs) under DoS attacks and actuator saturation, with a focus on robust practical stability (RPS). First, considering the impact of DoS attacks on closed-loop systems, an AET controller against DoS attacks is designed. Unlike other event-triggered controllers, the complete timeline is divided, and the AET controller is built with two switching modes based on the intervals of dormant and active periods of DoS attacks in which the system is located. Second, to reconcile AET controller with actuator saturation, a switched system modeling approach is established that explicitly incorporates saturation constraints into the coupled network dynamics. Third, a switched Lyapunov-Krasovskii functional (LKF) is proposed, with which sufficient conditions are provided to ensured the RPS, and a joint design strategy is developed for the desired triggered matrix and feedback gain using linear matrix inequalities (LMIs). Moreover, the results are generalized to the case of actuator faults, indicating that the system is able to achieve RPS with actuator faults. Finally, the proposed method is verified through an example.

In general, the motion control problem of redundant robot manipulators (RRMs) can be transformed into a constrained time-varying quadratic programming (TVQP) problem. Recently, various recurrent neural networks (RNNs) with predefined time convergence (PTC) abilities have been proposed to solve this constrained TVQP problem in real-time. However, there is still a lack of a unified paradigm to guide researchers and engineers design such RNNs more effectively based on specific requirements. To bridge this gap, we propose a unified paradigm derived from a novel segmentation evolution formula incorporating a special$\mathfrak{B}$–Classfunction. This paradigm enables the construction of various RNNs, collectively referred to as unified arbitrarily predefined-time convergent RNNs (U-APTC-RNNs). Compared with most existing RNNs, the constructed U-APTC-RNN has two significant advantages: 1) it has the arbitrarily PTC (APTC) ability, meaning its actual convergence time can be arbitrarily and precisely predefined without setting other model parameters and 2) using a novel piecewise computation strategy, redundant nonlinear calculations are effectively minimized, leading to a notable reduction in computational costs. The stability and APTC ability of the constructed U-APTC-RNN are demonstrated through detailed theoretical analysis. Numerical simulation experiments confirm the APTC capabilities of various U-APTC-RNNs constructed using the proposed unified paradigm. Comparative experiments show that U-APTC-RNN has more competitive convergence performance and lower computational cost than other state-of-the-art RNNs with PTC abilities. Finally, simulation and physical motion control experiments on the Jaco and UR5 robotic arms demonstrate the superiority and practicality of the proposed U-APTC-RNN.

This article deals with a joint multiple resources (relay, channel, and power) allocation problem for underwater acoustic (UWA) cooperative communication in time-varying Internet of Underwater Things scenarios. The strong coupling of multiple resources and the unknown time-varying characteristic of UWA communication scenes make the joint optimization problem full of challenges. To address this issue, the adversarial multiarmed bandit online learning model without any prior channel information and statistic assumptions is employed. Furthermore, a double closed-loop learning structure with multiple intelligent experts assistance is proposed. Multiple experts embedded in inner loop can intelligently learn the derived inferential information to provide more efficient advice for the player in outer loop, thereby enriching learning information and enhancing learning ability. In addition, the expert diversity learning mechanism is proposed to fully reflect the characteristics of seeking advantages and avoiding disadvantages in the double closed-loop learning structure. As a result, the learning speed and performance of the proposed algorithms are significantly improved. The superiorities of the proposed algorithms are demonstrated through numerical results.

The game of Tibetan Go faces the scarcity of expert knowledge and research literature. Therefore, we study the zero learning model of Tibetan Go under limited computing power resources and propose a novel scale-invariant U-Net style two-headed output lightweight network TibetanGoTinyNet. The lightweight convolutional neural networks and capsule structure are applied to the encoder and decoder of TibetanGoTinyNet to reduce computational burden and achieve better feature extraction results. Several autonomous self-attention mechanisms are integrated into TibetanGoTinyNet to capture the Tibetan Go board’s spatial and global information and select important channels. The training data are generated entirely from self-play games. TibetanGoTinyNet achieves 62%–78% winning rate against other four U-Net style models including Res-UNet, Res-UNet Attention, Ghost-UNet, and Ghost Capsule-UNet. It also achieves 75% winning rate in the ablation experiments on the attention mechanism with embedded positional information. The model saves about 33% of the training time with 45%–50% winning rate for different Monte-Carlo tree search (MCTS) simulation counts when migrated from 9 × 9 to 11 × 11 boards. Code for our model is available at https://github.com/paulzyy/TibetanGoTinyNet .

This paper presents a novel lightweight navigation system for oral and maxillofacial surgery (OMS). An external curved checkerboard with self-identifying markers is set as the reference object around the surgical scene. A customized oral clip with a micro camera is designed for oral localization by tracking the external checkerboard. Similarly, the dental handpiece is also equipped with a micro camera, which can be localized like the clip. The spatial model of the markers is provided by binocular stereoscopic reconstruction. A front surface mirror is taken up for the registration between the oral cavity and the camera on the clip. The pivot calibration of the dental handpiece is accomplished by our proposed calibration method. We set up an experimental group and a control group for evaluation. The surgical tools of the experimental group were approximately 30% lighter, 35% less bulky, and 90% cheaper than those of the control group. Our system yielded the comprehensive navigation accuracy of 0.92 mm whereas the accuracy of the control group was 0.87 mm. Results revealed that our system can achieve similar accuracy compared with a prevailing system at a lighter weight, a more compact volume, and a lower cost. Note to Practitioners—The motivation of this work is to reduce the burden on patients and surgeons during OMS. Current commercial navigation systems are still limited by the burden of extra cumbersome fiducial markers and high hardware costs. Their high accuracy benefits from the large size of fiducial markers. To take full advantage of the camera’s localization effect, we propose the concept of “marker-camera inverse projection”, i.e., reversing the roles of the camera and the markers. In this way, cameras on surgical tools detect more points with a more uniform distribution. Our proposed system achieves a decent balance between navigation accuracy and hardware cost, which facilitates the development of surgical tools to be lighter and more economical, and involves tremendous potential for commercialization.

Artificial lateral line (ALL) sensors hold the potential to enhance the perception abilities of robotic fish by capturing surface pressure gradients and identifying near-field object, such as dipole source. However, the robotic fish’s free-swimming motion introduces periodic low-frequency noise into the ALL data, while dipole sources with time-varying positions generate pressure signals with complex time-frequency characteristics. This paper proposes a complete solution to these challenges that would enable freely swimming robotic fish to locate dipole source. Firstly, an ALL system consisting of pressure sensors is integrated into the robotic fish, further constructing a real-time data acquisition and processing system. Secondly, to effectively estimate and remove the swimming-induced noise from the ALL data, a noise estimation model is developed based on the bionic motion mode and unsteady Bernoulli equation. Subsequently, short-time Fourier transform is applied to the high-quality data after noise elimination, followed by developing a convolution regression neural network for feature extraction and dipole source localization. Finally, extensive simulations and experiments are conducted to validate the effectiveness of the proposed methods and perform the positive impact of the noise estimation model. Remarkably, within the range of perception, the average accuracy of dipole source location can reach 13.6 mm, providing a promising reference for improving the perception abilities of underwater robots.Note to Practitioners—This paper is motivated by the problem of blind zones in near-field perception of underwater robots. The existing perception methods as visual sensing are limited by the dark and cloudy underwater environment, and are powerless in near-field localization. In addition, the artificial lateral line, as a potential near-field sensor, is challenging to be applied in self-propelled robots due to the swimming noise. This paper proposes an integrated near-field sensory system that includes an ALL sensor, a swimming noise elimination method and a dipole source localization method. Specifically, a fish-inspired ALL sensor is designed by high-accuracy pressure sensors and integrated into the robotic fish. To enhance localization performance, a swimming noise elimination model is constructed based on unsteady Bernoulli equation. Furthermore, a convolution regression network is developed for accurate localization of near-field objects. A series of simulations and experiments demonstrate the effectiveness and superiority of the proposed near-field sensory system. Hopefully, our proposed methods can provide valuable guidance and support for near-field object localization to improve the intelligent operation ability of underwater bionic robots, such as cooperative control, underwater navigation, environment exploration, and so forth.

The application potential of robotic fish embedded with intelligent visual navigation algorithms in underwater autonomous operation is on full display recently. However, the existing visual navigation methods are limited by the underwater visual conditions and the motion characteristics of robotic fish. To this end, this paper proposes a novel autonomous navigation framework integrated with visual stabilization control. In practice, a stereo vision-based navigation network is proposed to generate the guidance law. On this basis, a biomimetic robotic hammerhead shark with a controllable cephalofoil is developed, and a nonlinear model predictive controller for cephalofoil stabilization relying on the dynamic model is elaborately designed. Extensive simulations and underwater experiments are conducted to validate the effectiveness and superiority of the proposed methods, which significantly enhance exploration efficiency and reduce image jitter by 26.02% compared to the traditional methods. The obtained results provide a new idea for underwater robots to autonomously explore the ocean.Note to Practitioners—This paper is motivated by the problem of vision-based underwater autonomous navigation for a biomimetic robotic fish that possesses underwater visual stability and good maneuverability. The existing visual navigation networks usually generate unexpected navigation instructions when dealing with complex or ambiguous underwater scenes. Additionally, image jitter caused by the rhythmic motion of robotic fish can lead to navigation failure. This paper suggests an integrated navigation framework that includes a biomimetic platform design, a visual stabilization controller, and an intelligent underwater navigation network. Specifically, a novel sphyrnidae-inspired robotic shark is designed as a new platform with superior motion performance. To enhance underwater visual stability, a nonlinear model predictive control-based visual stabilization controller is proposed. Furthermore, a deep stereo attention navigation network based on a parallax attention mechanism is proposed to improve the generalization of vision-based autonomous navigation. A series of underwater search experiments on the robotic shark demonstrate the effectiveness and superiority of the proposed navigation framework. Hopefully, our proposed methods can provide valuable guidance and support for universal underwater robot navigation to accomplish practical marine tasks, such as underwater rescue, resource exploitation, biological observation, and so on.

Few-shot object detection (FSOD) aims to detect novel objects with limited annotated examples. Mainstream methods suffer from the data scarcity of novel classes with insufficient intra-class variations, which makes the trained model biased to base classes. Actually, there are massive unlabeled novel instances in the base dataset and their adequate utilization will enhance the discriminability of model to novel classes. This paper proposes a semi-supervised few-shot object detection method, which utilizes a teacher model and a pre-trained few-shot object detector to guide the learning of a student model through adaptive pseudo labeling. In particular, a class-adaptive threshold filtering (CATF) strategy is designed to deal with the class-imbalance problem of pseudo labels. And for each novel class, the threshold to select valuable pseudo labels is determined by quantile statistics of the confidence score distribution of pseudo labels. Furthermore, the pre-trained detector and the teacher model are associated with the preliminary CATF and in-depth CATF, respectively, and then the pseudo labels from the two-stream CATF are fused to provide supervisions. In this way, the knowledge of these two models is exploited, which improves the quality of pseudo labels. Under these supervisions, the student model is trained and the teacher model is correspondingly updated through parameters sharing, thus forming a positive feedback to improve the performance of both models. Besides, an attention module is integrated to the teacher and student models to enhance the feature representation of novel instances. The validations on PASCAL VOC and MS COCO show the effectiveness of the proposed method.

The broad and powerful pectoral fins of manta rays are crucial to their efficient and maneuverable swimming. However, very little is currently known about the pectoral-fin-driven 3-D locomotion of manta-inspired robots. This study is focused on the development and 3-D path-following control of an agile robotic manta. First, a novel robotic manta with 3-D mobility is constructed, of which the distinctive pectoral fins provide the only propulsion. Specifically, the unique pitching mechanism is detailed in which the time-coupled coordination movement of the pectoral fins is applied. Second, based on a 6-axis force measuring platform, the propulsion characteristics of the flexible pectoral fins are analyzed. Then, the force-data-driven 3-D dynamic model is further established. Third, a control scheme combined with a line-of-sight (LOS) guidance system and a sliding-mode fuzzy controller is conceived, addressing the 3-D path-following task. Finally, various simulated and aquatic experiments are conducted, demonstrating the superior performance of our prototype and the effectiveness of the proposed path-following scheme. This study will hopefully generate fresh insights into the updated design and control of agile bioinspired robots performing underwater tasks in dynamic environments.

In recent years, there has been a prominent focus on enhancing the quality of features derived from convolutional neural networks (CNNs) within the field of polarimetric synthetic aperture radar (PolSAR) image classification. Targeting this challenge, this article first visualizes the lack of discriminability and generalizability in CNN features through several empirical observations. Subsequently, we explain why these problems arise from a causal perspective, accomplished by means of a structural causal model (SCM) constructed according to the training and testing process of CNNs. This SCM facilitates the identification of variables that affect the quality of PolSAR image feature learning, as well as an intervention on those variables using backdoor adjustment. Building upon this groundwork, a novel causal inference-guided feature enhancement framework is constructed. It can be seamlessly integrated into any CNN-based PolSAR image classifier in a plug-and-play manner, enabling the enhanced classifier to filter out interference information and prevent model overfitting. These two aspects bring better feature discriminability and generalizability, respectively, leading to improved classification performance. Experimental results on four widely-used PolSAR image datasets demonstrate the effectiveness of our proposed framework. We integrate it into several mainstream methods in the field and show that the accuracy of the enhanced classifier is improved compared to the original model.

Remote sensing image (RSI) classification is a popular research topic that aims to assign semantic labels to images acquired from aerial or maritime platforms. Existing deep feature learning methods for this task can be divided into two paradigms: generative and discriminative. The former methods are good at capturing every local detail of images, while the later approaches focus on the most salient area. The significant differences between the two types of methods, both in terms of their underlying mechanisms and practical implementation, motivate us to integrate information acquired by both paradigms by exploiting their complementary strengths. However, this idea faces a challenge that local information in the extracted features, especially those from generative methods, may not be reliable for RSI classification. The reason for this challenge is that, due to the characteristics of the ground observation perspective, some RSIs, while semantically different, exhibit a significant degree of similarity in local details. This phenomenon leads to insufficient discriminability of local features to separate multiple RSI categories, which implies that the classification results overly focused on local information may be unreliable. To address this issue, in this article, we propose a novel framework that integrates generative and discriminative feature learning methods with evidential learning for RSI classification. Our framework uses the Dirichlet distribution to model the predicted probabilities to be integrated, thereby collecting evidence about their reliability. This enables us to integrate multiple features at an evidence level and make reliable decisions, overcoming the unreliabilities of generative-discriminative joint feature learning induced by RSI characteristics. We evaluate the proposed framework on several satellite and shipborne RSI classification datasets. The experimental results show that our method outperforms the state-of-the-art baselines in terms of accuracy and robustness.

The high precision measurement of binocular camera depends on the calibration accuracy. Mainstream methods focus on the calibration of intrinsic parameters and the pose relationship of two cameras in sequence. However, the transmission of error will decrease the calibration quality. It is still challenging especially in the case of an unknown imaging model due to the intervention of baffle and multimedia refraction. In this article, a general calibration method for a binocular vision system with an unknown imaging model is proposed. Based on the one-to-one mapping between 4-D image coordinates and 3-D Cartesian coordinates of spatial points, the intersection line of two calibration planes is extracted. All calibration planes are organized in the form of a three-plane group, and in each group, the poses are solved by the intersection lines. These poses are further optimized to acquire accurate 3-D coordinates of points in calibration planes. Combined with the correspondences of image pixels and points in a calibration plane, the line-plane intersection point between the projection line and this plane is calculated. Finally, projection lines are fitted based on corresponding line-plane intersection points. By modeling the relationship between image pixels and object-end projection lines, the uncertainty of the imaging model is solved. The proposed method is universal and stable for different imaging models including refraction, and the results of simulation and experiment show its effectiveness.

For few-shot object detection, this work proposes a binary similarity detector (BSDet), which realizes a novel similarity-based multiple binary classification and enhances the feature margin between positive and hard negative samples. First, we revisit the classification paradigm, concluding that multiple binary classification paradigm is more suitable than multi-class classification paradigm for the few-shot task. Hence, we propose a binary similarity head (BSH) by posing the classification task as multiple binary similarity measurements rather than a multi-class prediction. Second, focusing on the hard negative samples, we propose a feature enhancement module (FEM). During training phase, the FEM can push the features of positive and hard negative samples far away from each other, and thus effectively suppresses false positives. Abundant experiments and visualizations indicate that our method achieves state-of-the-art performances on few-shot object detection tasks.

This article deals with the task assignment problem for multiple autonomous underwater vehicles to efficiently collect underwater data from sensors. We formulate a unified framework to consistently address the heterogeneous task assignment problem (nonemergency and emergency cases) without strictly distinguishing the mixed cases. First, a unified problem, which bridges the gap between different constraints and optimization objectives of different cases, is constructed. Then, the proposed reinforced self-organizing mapping algorithm is reinforced in three aspects: the regional learning rate, the self-configuring neuron (SCN) strategy, and the workload balance mechanism. Specifically, the proposed regional learning rate comprehensively considers the individual worth of tasks and the topology to generate the regional learning rate of dynamic task regions, which consists of dynamic remaining tasks and the reconstructed topology. Based on this idea, the constructed unified problem can be solved consistently. Furthermore, the proposed SCN strategy optimizes the neuron population both in quality and quantity, and guides the update of neurons with enriched historical information to improve the mapping ability. This strategy greatly improves learning efficiency and applicability in a wide range of scenarios. Meanwhile, the proposed workload balance mechanism takes into consideration of both the work capability and consumed energy to extend the continuous working capability. The numerical results validate the effectiveness and adaptability of the proposed unified task assignment framework.

Model-based robot controllers require customized control-oriented models, involving expert knowledge and trial and error. Remarkably, the Koopman operator enables the control-oriented model identification through the input–output mapping set, breaking through the barriers of the customization services. However, in recent years, research on Koopman-based robot control has mostly focused on lifting function construction, deviating from the original intention of improving the controller performance. Thus, we propose a robot controller autogeneration framework using the Bayesian-based Koopman operator, significantly releasing labor and eliminating the design obstacle. First, we introduce the Koopman-based system identification method and offer the basic lifting function design criteria. Then, a Bayesian-based optimization strategy with resource allocation is designed, which allows for the simultaneous optimization of the lifting function and the controller. Next, taking model-predictive control (MPC) as an example, a mission-oriented controller autogeneration framework is developed. Simulation and experimental results indicate that, under various robots and data sources, the proposed framework can effectively generate the robot controllers and perform with a far greater level of mission accuracy than the unoptimized Koopman-based MPC. Meanwhile, the proposed technique exhibits an obvious compensation effect against disturbances, demonstrating its practicability in robot control.

Fish locomotion which adopts body and/or caudal fin swimming mode consists of different motions, such as Cruising-straight, Cruising-turn, and various fast turns, among others. Currently, there is no single mathematical model that could illustrate all these motions. Thus, for scientists and engineers, it is quite cumbersome and complicated to model and control different motions with multiple principles. In this article, we proposed a general kinematic model to illustrate the kinematics of all aforementioned swimming motions. The model is synthesized by a nonlinear oscillator and a traveling wave equation. By changing four parameters extracted from the model, the kinematic model can demonstrate all the aforementioned swimming motions with different amplitudes and frequencies. To verify the model, we built a multijoint robotic fish and developed its dynamic model and control method to perform all the maneuvers under the guidance of the general kinematic model. Through this systematic methodology, one can easily study the principles of different swimming motions and design the multimotions controller for a robotic fish through only one governing kinematic model.

As a key technology for autonomous underwater operations, precise tracking control in tight space environments is a great challenge. With the aid of high maneuverability of the underwater bionic robot, this article proposes an integrated tracking control framework for a robotic dolphin to move through narrow areas, including top-level planning, middle-level tracking, and bottom-level control allocation. First, a nonlinear model predictive control-based planning method is presented with full consideration of tracking accuracy and obstacle avoidance safety. Second, in order to improve the anti-interference ability, we derive a nonlinear path tracking control law by combining the backstepping technique with a nonlinear disturbance observer. More importantly, through hydrodynamic analysis of the bionic multimodal motions under flippers and flukes, a fuzzy-based nonlinear control allocation system is particularly adopted to convert calculated control forces into bionic motion parameters. Finally, extensive simulations and aquatic experiments are conducted, and the obtained results validate the effectiveness of proposed methods, providing a new idea to further ocean exploration.

This study deals with a joint channel selection and power allocation problem for time-varying underwater acoustic communication system. Without any prior channel information, designing a highly adaptable resource allocation algorithm to cope with the fast time-varying environment is a very challenging issue. To address this issue, a hierarchical learning approach, which is combined with adversarial multiarmed bandit theory and outdated pilot-based feedback information, is proposed. The proposed learning approach can online optimize joint resource allocate strategy without any prior channel state information. Specifically, a hierarchical self-reflection learning structure is proposed to offer different learning manners and spaces for the actual played information and outdated feedback information, thereby balancing the exploitation and exploration to cope with the time-varying environment effectively. Further, an integration learning structure is proposed to alleviate the solving difficulty and policy explosion of joint multiple substrategies problem. The user can rapidly achieve a few superior strategies in low-dimension space, then efficiently search the expected optimal strategy in high-dimension space, as a result, the learning efficiency is significantly improved. The proposed algorithms show strong tolerance for delay and noncomplete information due to the elaborate learning structures. The superiority of the proposed algorithms is demonstrated through numerical results.

Sparse optimization problems have been successfully applied to a wide range of research areas, and useful insights and elegant methods for proving the stability and convergence of neurodynamic algorithms have been yielded in previous work. This article develops several neurodynamic algorithms for sparse signal recovery by solving the$\ell _{1}$regularization problem. First, in the framework of the locally competitive algorithm (LCA), modified LCA (MLCA) with finite-time convergence and MLCA with fixed-time convergence are designed. Then, the sliding-mode control (SMC) technique is introduced and modified, i.e., modified SMC (MSMC), which is combined with LCA to design MSMC-LCA with finite-time convergence and MSMC-LCA with fixed-time convergence. It is shown that the solutions of the proposed neurodynamic algorithms exist and are unique under the observation matrix satisfying restricted isometry property (RIP) condition, while finite-time or fixed-time convergence to the optimal points is shown via Lyapunov-based analysis. In addition, combining the notions of finite-time stability (FTS) and fixed-time stability (FxTS), upper bounds on the convergence time of the proposed neurodynamic algorithms are given, and the convergence results obtained for the MLCA and MSMC-LCA with fixed-time convergence are shown to be independent of the initial conditions. Finally, simulation experiments of signal recovery and image recovery are carried out to demonstrate the superior performance of the proposed neurodynamic algorithms.

This article investigates a fully distributed inertial neurodynamic approach for sparse recovery. The approach is based on proximal operators and inertia items. It aims to solve the$L_{1}$-norm minimization problem with consensus and linear observation constraints over directed communication networks. The proposed neurodynamic approach has the advantages of only requiring the communication network to be directed and weight-balanced, does not involve a central processing node and global parameters, which means that no single node can access the entire network and observe it at any time, so it is fully distributed. To effectively deal with the nonsmooth objective function,$L_{1}$-norm, the proximal operator method is used here. For efficiently handling linear observation and consensus constraints, a primal-dual method is applied to the inertial dynamic system. With the aid of maximal monotone operator theory and Baillon-Haddad lemmas, it reveals that the trajectories of our approach can converge to consensus solution at the optimal solution, provided that the distributed parameters satisfy technical conditions. In addition, we aim to demonstrate the weak convergence of the trajectories in our proposed neurodynamic approach toward the zeros of the optimal operator in Hilbert space, using Opial’s lemma. Finally, comparative experiments on sparse signal and image recovery confirm the efficiency and effectiveness of our proposed neurodynamic approach.

In this paper, a novel Field-Programmable-Gate-Array (FPGA) implementation framework based on Lagrange programming neural network (LPNN), projection neural network (PNN) and proximal projection neural network (PPNN) is proposed which can be used to solve smooth and nonsmooth optimization problems. First, Count Unit (CU) and Calculate Unit (CaU) are designed for smooth problems with equality constraints, and these units are used to simulate the iteration actions of neural network (NN) and form a feedback loop with other basic digital circuit operations. Then, the optimal solutions of optimization problems are mapped by the output waveforms. Second, the digital circuit structures of Path Select Unit (PSU), projection operator and proximal operator are further designed to process the box constraints and nonsmooth terms, respectively. Finally, the effectiveness and feasibility of the circuit are verified by three numerical examples on the Quartus II 13.0 sp1 platform with the Cyclone IV E series chip EP4CE10F17C8.

High-precision pose estimation based on visual markers has been a thriving research topic in the field of computer vision. However, the suitability of traditional flat markers on curved objects is limited due to the diverse shapes of curved surfaces, which hinders the development of high-precision pose estimation for curved objects. Therefore, this paper proposes a novel visual marker called CylinderTag, which is designed for developable curved surfaces such as cylindrical surfaces. CylinderTag is a cyclic marker that can be firmly attached to objects with a cylindrical shape. Leveraging the manifold assumption, the cross-ratio in projective invariance is utilized for encoding in the direction of zero curvature on the surface. Additionally, to facilitate the usage of CylinderTag, we propose a heuristic search-based marker generator and a high-performance recognizer as well. Moreover, an all-encompassing evaluation of CylinderTag properties is conducted by means of extensive experimentation, covering detection rate, detection speed, dictionary size, localization jitter, and pose estimation accuracy. CylinderTag showcases superior detection performance from varying view angles in comparison to traditional visual markers, accompanied by higher localization accuracy. Furthermore, CylinderTag boasts real-time detection capability and an extensive marker dictionary, offering enhanced versatility and practicality in a wide range of applications. Experimental results demonstrate that the CylinderTag is a highly promising visual marker for use on cylindrical-like surfaces, thus offering important guidance for future research on high-precision visual localization of cylinder-shaped objects.

This paper proposes a tightly-coupled visual-Doppler-Velocity-Log (visual-DVL) fusion method for underwater robot localization through integrating the velocity measurements from a DVL into a visual odometry (VO). Considering that employing the DVL measurements in dead-reckoning systems easily leads to error accumulation and suboptimal results in previous works, we directly integrate them into the visual tracking process. Specifically, the velocity measurements are utilized to improve the initial estimation of camera pose during visual tracking, aiming to provide a better initial value for pose optimization. Thereafter, these velocity measurements are also directly employed to constrain the position change of the camera between two adjacent frames by constructing a novel DVL error term, which is optimized jointly with the visual constrains to obtain a more accurate camera pose. Various experiments are carried out in the datasets collected from several scenarios of the underwater simulation environment HoloOcean, and the results illustrate that the proposed fusion method can effectively improve the localization accuracy for underwater robots by about 20% compared to pure visual odometry. The proposed method provides valuable guidance for the accurate localization of underwater robots.

In Internet of Underwater Things, data collection is assisted by autonomous underwater vehicle (AUV) to enhance the reliable transmission. AUV acts as a mobile collector and transmits the collected data to the station via relay nodes. However, the highly mobile nature of AUV needs an adaptive and efficient relay selection scheme for achieving good capacity performance. In this article, we propose a new contextual multiarmed bandit with evolving relay set (CMAB-ERS) learning framework, which successfully addresses crucial issues, including dynamic environment conditions and evolving relay set. To deal with the evolving relay set, CMAB-ERS incorporates collaborative effects into inference as well as learning processes, the new relays will acquire prior knowledge by having experienced nodes sharing observations, reducing the learning time significantly. To overcome the uncertainty of environmental information, we exploit the contextual environment factors to assist relay reward estimation and execute time-sensitive parameter update after every transmit–receive cycle, aiming for minimizing potential loss due to the time-varying channel. Correspondingly, the collaboration-aware online contextual bandit learning (COCBL) algorithm is designed that enables AUV to switch optimal relay adaptively and promises high-capacity transmission. Further, we rigorously prove the convergence of the COCBL algorithm by considering the evolving relay set and give its upper bound on the cumulative regret. Finally, extensive simulation results elucidate the effectiveness of the proposed COCBL.

For complex functions to emerge in artificial systems, it is important to understand the intrinsic mechanisms of biological swarm behaviors in nature. In this paper, we present a comprehensive survey of pursuit–evasion, which is a critical problem in biological groups. First, we review the problem of pursuit–evasion from three different perspectives: game theory, control theory and artificial intelligence, and bio-inspired perspectives. Then we provide an overview of the research on pursuit–evasion problems in biological systems and artificial systems. We summarize predator pursuit behavior and prey evasion behavior as predator–prey behavior. Next, we analyze the application of pursuit–evasion in artificial systems from three perspectives, i.e., strong pursuer group vs. weak evader group, weak pursuer group vs. strong evader group, and equal-ability group. Finally, relevant prospects for future pursuit–evasion challenges are discussed. This survey provides new insights into the design of multi-agent and multi-robot systems to complete complex hunting tasks in uncertain dynamic scenarios.

An n-order marker field is a special binary matrix whose n×n subregions are all distinct from each other in four orientations. It is commonly used to guide the composing process of position-sensing markers, which can be detected and identified in a camera image with very limited scope or severe visibility problems. Despite the advantages, position-sensing markers are rare and overlooked because generating marker fields is difficult. In this article, we broaden the definition of marker field, making it more powerful and flexible. Then, we propose bWFC (binary wave function collapse) and its high-speed version, fast-bWFC, to solve the generation problem. The methods are packaged into an open-sourced toolkit named HydraMarker, with which, users not only can generate marker fields on laptops within a short period of time, but also can highly customize them: preset values; fields and subregions in any shape; multifold local uniqueness. Comparative results indicate that the proposed method has superior efficiency, quality, and capability. It makes marker field generation accessible to common marker designers, opening up more possibilities for fiducial markers.

The grasp detection is crucial to high-quality robotic grasping. Typically, the mainstream encoder-decoder regression solution is attractive due to its high accuracy and efficiency, however, it is still challenging to solve the checkerboard artifacts from the uneven overlap of convolution results in decoder, and features from the encoder also need to be further refined. In this paper, a novel pixel-wise grasp detection network is proposed, which is composed of an encoder, a multi-dimensional attention bottleneck, and a decoder based on twin deconvolution. The proposed decoder introduces a twin branch upon the original transposed convolution branch. Through the overlap degree matrix provided by the twin branch, the original branch is re-weighted and then the checkerboard artifacts of the original branch are eliminated. Besides, to deeply explore the intrinsic relationship of features and strengthen feature discrimination, residual multi-head self-attention, cross-amplitude attention, and channel attention are integrated together. As a result, adaptive feature refinement is achieved. The effectiveness of the proposed method is verified by experiments.

This article proposes a novel barrier-based adaptive line-of-sight (ALOS) three-dimensional (3-D) path-following system for an underactuated multijoint robotic fish. The framework of the developed path-following system is established based on a detailed dynamic model, including a barrier-based ALOS guidance strategy, three integrated inner-loop controllers, and a nonlinear disturbance observer (NDOB)-based sideslip angle compensation, which is employed to preserve a reliable tracking under a frequently varying sideslip angle of the robotic fish. First, a barrier-based convergence strategy is proposed to deal with probable along-track error disruption and suppress the error within a manageable range. Meanwhile, an improved adaptive guidance scheme is adopted with an appropriate look-ahead distance. Afterward, a novel NDOB-based sideslip angle compensation is put forward to identify the varying sideslip angle independent of speed estimation. Subsequently, inner-loop controllers are intended for regulation about the controlled references, including a super-twisting sliding-mode control (STSMC)-based speed controller, a global fast terminal sliding-mode control (GFTSMC)-based heading controller, and a GFTSMC-based depth controller. Finally, simulations and experiments with quantitative comparison in 3-D linear and helical path following are presented to verify the effectiveness and robustness of the proposed system. This path-following system provides a solid foundation for future marine autonomous cruising of the underwater multijoint robot.

Dynamic object perception is an important yet challenging direction in the field of robot navigation. Without any prior knowledge about motion and objects, a novel dynamic rigid bodies mining and motion estimation method based on monocular camera is proposed in this article. Different from the existing works based on sampling that associate feature points to motion hypotheses according to the reprojection errors, our work endeavors to find the intrinsic relevance among motion hypotheses. To represent this relevance, the concept of the probabilistic field on the Lie group Sim(3) manifold is introduced, which is established using random sampling. It provides a computable way for the regions on the manifold where rigid bodies possibly appear. The probability of a motion hypothesis falling on a region is expressed by its confidence. The regions with large confidences in the probabilistic field are selected as potential rigid bodies, whose corresponding feature points are further sampled for pose calculation. As a result, the randomness of sampling is reduced and the inliers for possible rigid bodies are enhanced, which guarantees the accuracy of motion estimation. On this basis, the tracking of rigid bodies is achieved. The proposed method distinguishes the feature points of dynamic objects with 3-D motion from those in the static background, thus enabling simultaneous localization and mapping (SLAM) to be initialized in dynamic environments. The experimental results on the KITTI, Hopkins 155, and MTPV62 datasets demonstrate the effectiveness. Comparison experiments indicate that our method outperforms the other methods in sensitivity of dynamic objects perception.

In this article, a multi-underwater unmanned vehicle (UUV) maneuvering decision-making algorithm is proposed for a counter-game with a dynamic target scenario. The game is modeled with interval-valued intuitionistic fuzzy rules, and an optimal maneuvering strategy is realized using a fractional-order recurrent neural network (RNN). First, underwater environments with weak connectivity, underwater noise, and dynamic uncertainties are analyzed and incorporated into the interval-valued intuitionistic fuzzy set. Then, the maneuvering decision-making model and the expected return of the multi-UUV countermeasure are designed based on the interval-valued intuitionistic fuzzy rules. Subsequently, to optimize the counter-game maneuvering strategy, a fractional-order RNN is formulated based on the Karush-Kuhn-Tucker optimality conditions. In addition, the existence and uniqueness of the optimal maneuvering solutions as well as the stability of the equilibrium point are discussed. Finally, simulation and experimental results are compared to determine the effectiveness of the proposed algorithm. The influence of the fractional order on the convergence rate and optimization error of the proposed algorithm is also minutely examined.

Within the last few years, great efforts have been made to study few-shot learning. Although general object detection is advancing at a rapid pace, few-shot detection remains a very challenging problem. In this work, we propose a novel decoupled metric network (DMNet) for single-stage few-shot object detection. We design a decoupled representation transformation (DRT) and an image-level distance metric learning (IDML) to solve the few-shot detection problem. The DRT can eliminate the adverse effect of handcrafted prior knowledge by predicting objectness and anchor shape. Meanwhile, to alleviate the problem of representation disagreement between classification and location (i.e., translational invariance versus translational variance), the DRT adopts a decoupled manner to generate adaptive representations so that the model is easier to learn from only a few training data. As for a few-shot classification in the detection task, we design an IDML tailored to enhance the generalization ability. This module can perform metric learning for the whole visual feature, so it can be more efficient than traditional DML due to the merit of parallel inference for multiobjects. Based on the DRT and IDML, our DMNet efficiently realizes a novel paradigm for few-shot detection, called single-stage metric detection. Experiments are conducted on the PASCAL VOC dataset and the MS COCO dataset. As a result, our method achieves state-of-the-art performance in few-shot object detection. The codes are available at https://github.com/yrqs/DMNet.

This article proposes an improved general zeroing neural network model to suppress noise and to enhance the real-time performance of solving TVQP problems. The proposed model allows nonconvex activation functions and has noise suppression characteristics, i.e., the NCNSZNN model. Theoretical analyses show that the developed NCNSZNN model converges globally to an accurate solution to the TVQP problem and is robust in the case of MN. Illustrative examples and comparisons are supplied to verify the validity and superiority of the proposed model for online solving TVQP constrained by EAI with MN.

Robust vision restoration of underwater images remains a challenge. Owing to the lack of well-matched underwater and in-air images, unsupervised methods based on the cyclic generative adversarial framework have been widely investigated in recent years. However, when using an end-to-end unsupervised approach with only unpaired image data, mode collapse could occur, and the color correction of the restored images is usually poor. In this paper, we propose a data- and physics-driven unsupervised architecture to perform underwater image restoration from unpaired underwater and in-air images. For effective color correction and quality enhancement, an underwater image degeneration model must be explicitly constructed based on the optically unambiguous physics law. Thus, we employ the Jaffe-McGlamery degeneration theory to design a generator and use neural networks to model the process of underwater visual degeneration. Furthermore, we impose physical constraints on the scene depth and degeneration factors for backscattering estimation to avoid the vanishing gradient problem during the training of the hybrid physical-neural model. Experimental results show that the proposed method can be used to perform high-quality restoration of unconstrained underwater images without supervision. On multiple benchmarks, the proposed method outperforms several state-of-the-art supervised and unsupervised approaches. We demonstrate that our method yields encouraging results in real-world applications.

Relay selection solutions for underwater acoustic cooperative networks suffer significant performance degradation as they fail to adapt to incomplete information, noisy interference and overwhelming dynamics. To address this challenge, a hierarchical adversarial multi-armed bandit learning framework by proposing an online reward estimation layer is designed to improve adaptive relay decision control. In online reward estimation layer, adaptive Kalman filter estimator is developed to properly handle noisy observation to support accurate reward. Meanwhile, an online predict mechanism is projected for all relays to enrich learning information. Furthermore, based on estimate error variance, an adaptive exploration structure is developed to accelerate the balance between exploration and exploitation. All gathered information are exploited to learn relay quality for the decision-making. Accordingly, we present a Hierarchical Adversarial Bandit Learning (HABL) algorithm to fully exploit the heuristic interaction between the hierarchical framework. HABL integrates reward estimation, information prediction, adaptive exploration and decision making carefully in a holistic algorithm to maximize the learning efficiency. Thereby, the HABL-based relay selection algorithm has higher system throughput and lower communication cost. Further, we rigorously analyze the convergence of HABL algorithm and give its upper bound on the cumulative regret. Finally, extensive simulations elucidate the effectiveness of the HABL.

This article investigates the fault estimation and fault-tolerant control problem for underwater vehicles with the Takagi–Sugeno (T–S) fuzzy model. In order to deal with the disturbance of complex ocean environment, a stable fuzzy controller for underwater vehicle is proposed to realize efficient operation, which is on the basis of T–S fuzzy model with pitch angle membership function. Meanwhile, to reduce the waste of communication resources, a novel discrete event-triggered control scheme is proposed to use multiple historical sampled data to determine the next release instant. The discrete event-triggered fault-tolerant controller compensates for the influence of system faults by using state estimators and fault estimators. It is noted that the canonical Bessel–Legendre inequality and delay-dependent canonical orthogonal Legendre polynomials play an important role in dealing with the asymptotical stability of the T–S fuzzy delayed model with an$H_{\infty }$performance. Finally, a simulation example is carried out to show the validity of the presented theorem.

Underwater robotic operation usually requires visual perception (e.g., object detection and tracking), but underwater scenes have poor visual quality and represent a special domain which can affect the accuracy of visual perception. In addition, detection continuity and stability are important for robotic perception, but the commonly used static accuracy based evaluation (i.e., average precision) is insufficient to reflect detector performance across time. In response to these two problems, we present a design for a novel robotic visual perception framework. First, we generally investigate the relationship between a quality-diverse data domain and visual restoration in detection performance. As a result, although domain quality has an ignorable effect on within-domain detection accuracy, visual restoration is beneficial to detection in real sea scenarios by reducing the domain shift. Moreover, non-reference assessments are proposed for detection continuity and stability based on object tracklets. Further, online tracklet refinement is developed to improve the temporal performance of detectors. Finally, combined with visual restoration, an accurate and stable underwater robotic visual perception framework is established. Small-overlap suppression is proposed to extend video object detection (VID) methods to a single-object tracking task, leading to the flexibility to switch between detection and tracking. Extensive experiments were conducted on the ImageNet VID dataset and real-world robotic tasks to verify the correctness of our analysis and the superiority of our proposed approaches. The codes are available at https://github.com/yrqs/VisPerception .

To tackle the problem of aquatic environment pollution, a vision-based autonomous underwater garbage cleaning robot has been developed in our laboratory. We propose a garbage detection method based on a modified YOLOv4, allowing high-speed and high-precision object detection. Specifically, the YOLOv4 algorithm is chosen as a basic neural network framework to perform object detection. With the purpose of further improvement on the detection accuracy, YOLOv4 is transformed into a four-scale detection method. To improve the detection speed, model pruning is applied to the new model. By virtue of the improved detection methods, the robot can collect garbage autonomously. The detection speed is up to 66.67 frames/s with a mean average precision (mAP) of 95.099%, and experimental results demonstrate that both the detection speed and the accuracy of the improved YOLOv4 are excellent.

This article explores the asymptotic stabilization criteria of the uncertain nonlinear time-delay system subject to actuator saturation. A switched integral-based event-triggered scheme (IETS) is established to reduce the redundant data transmission over the networks. The switched IETS condition uses the integration of system states over a time period in the past. A fixed waiting time is included to avoid the Zeno behavior. In order to estimate a larger domain of attraction, a delay-dependent polytopic representation method is presented to deal with the effects of actuator saturation in the proposed model. A new series of less conservative linear matrix inequalities (LMIs) is proposed on the basis of delay-dependent Lyapunov-Krasovskii functional (LKF) to ensure the stability of nonlinear time-delay system subject to actuator saturation using the proposed IETS. Numerical examples are used to confirm the effectiveness and advantages of the proposed IETS approach.

A heuristic fast marching (FM*)-based comprehensive path planning system involving task allocation, initial planning, and replanning is presented for the robotic floating garbage cleaning mission. There are three primary contributions in this paper. First, to tackle the invalidation of the Euclidean distance metric in the obstacle environment, the task allocation is modeled as a travelling salesman problem (TSP) employing the FM*-based distance metric in order to obtain an optimal travel sequence. Second, to meet the maneuverability constraint from the surface robot and avoid the collision, a Gaussian filter is employed to adjust the curvature radius of the generated path. Third, for an efficient replanning, a neural network-based replanning point generator with the input of garbage movement vector is provided to strike a compromise for the distance cost and the computational burden. Moreover, a case study and a virtual obstacle experiment in the laboratory water tank demonstrate the feasibility of the proposed comprehensive path planning system. This work lays a firm foundation for the development of intelligent equipment for aquatic environment protection.

This article proposes the mechanical design and dynamic model of an innovative manta-inspired robot system for both efficient fast swimming and high spatial maneuverability. Inspired by some biological studies, a pair of unique pectoral fins with six separate degrees of freedoms (DOFs) are developed. The novel design is characterized by an improved crank-rocker mechanism and a distinctive horizontal DOF. The former not only endows the robot with high swimming speed, but also guarantees efficient flapping patterns which are close to manta rays. The latter is employed to coordinate with the flapping movement, allowing remarkable pitch adjustment. Further, the basic motion strategy is presented by detailed analyses to the pectoral fins. Besides, based on the Morrison equation and infinitesimal method, a complete dynamic model for robotic manta with flexible pectoral fins is established, whose parameters are determined through experimental data. Moreover, the linear swimming and pitching experiments are conducted, demonstrating the prominent movement performance of the presented design and the effectiveness of the dynamic model. The obtained results shed light on updated design and control of next-generation agile underwater vehicles and robots capable of multimodal motions in dynamic and complex aquatic environments.

In this article, the distributed finite-time and fixed-time optimization problems are investigated by adopting the zero-gradient-sum (ZGS) framework in multiagent systems. Specifically, when the local convex functions are nonquadratic, a basic optimization protocol is proposed to obtain a finite-time convergence, such that the networked system can cooperatively seek the optimal solution of the global objective, the sum of local objective, in a limited time. By utilizing the property of quadratic functions, a reduced algorithm can remove the dependence of initial conditions in the estimation of the upper bound of settling time and achieve a fixed-time result. Besides, the problem with time-varying topologies is studied by introducing a modified algorithm with an artificial potential function to preserve the network connectivity. Finally, the validity of the protocols is demonstrated via some example simulations.

The manipulating robots receive much attention by offering better services, where object grasping is still challenging especially under background interferences. In this article, a novel two-stream grasping convolutional neural network (CNN) with simultaneous detection and segmentation is proposed. The proposed method is cascaded by an improved simultaneous detection and segmentation network BlitzNet and a two-stream grasping CNN TsGNet. The improved BlitzNet introduces the channel-based attention mechanism, and achieves an improvement of detection accuracy and segmentation accuracy with the combination of the learning of multitask loss weightings and background suppression. Based on the obtained bounding box and the segmentation mask of the target object, the target object is separated from the background, and the corresponding depth map and grayscale map are sent to TsGNet. By adopting depthwise separable convolution and designed global deconvolution network, TsGNet achieves the best grasp detection with only a small amount of network parameters. This best grasp in the pixel coordinate system is converted to a desired 6-D pose for the robot, which drives the manipulator to execute grasping. The proposed method combines a grasping CNN with simultaneous detection and segmentation to achieve the best grasp with a good adaptability to background. With the Cornell grasping dataset, the image-wise accuracy and object-wise accuracy of the proposed TsGNet are 93.13% and 92.99%, respectively. The effectiveness of the proposed method is verified by the experiments.

Soft robots have been applied widely to various constrained scenarios due to the advantages over traditional rigid manipulators such as softness, deformability and adaptability to constrained surroundings. To make full use of this merit, this paper proposes a method that integrates collision detection, localization and force estimation for a cable-driven soft manipulator without any prior geometrical knowledge of its surroundings. First of all, a collision detection algorithm is presented based upon Cosserat-rod statics by a threshold method through using the cable tension and the shape information, which are obtained by the load cells and the Vicon system, respectively. Secondly, a collision localization and force estimation method is proposed through optimizing the discrepancy between the actual and the theoretical shapes. Finally, experiments are carried out to validate these algorithms. The experimental results demonstrate that the site, the magnitude as well as the direction can be estimated.

This paper proposes a multi-objective visionbased navigation network for biomimetic underwater robots to cope with scientific observation, target selection, and obstacle avoidance in marine missions. Structurally, a stereo block attention module is first constructed to serially extract the channel and spatial attention portion of the real-time visual feedback. Next, the parallax attention mechanism is introduced to enable the network to excavate implicit parallax information in stereo pairs, effectively eliminating the oscillation of the network output in the presence of ambiguous visual input. Further, with the assistance of other low-cost sensors, the proposed navigation network can be expanded in some largescale application scenarios, such as sparse coral observation. Finally, underwater simulations reveal that the proposed method obtains significantly improved control effect and real-time ability, compared with other related works. In particular, based on a self-developed biomimetic robotic dolphin, collision-free simulations with a cumulative distance beyond 1000 m were carried out and validated the effectiveness and the superiority of the navigation network, where both dense and sparse targets were fully tested. The robotic dolphin can not only successfully conduct accurate coral observation without collision, but also quest the observation targets as much as possible in the area where the observation targets are concentrated. The proposed network provides an intelligent and efficient navigation scheme for autonomous underwater operation of small-size underwater robots.

Underwater visual localization is an essential technique for the autonomous operation of underwater robots. However, the unique underwater image characteristics, including refraction, sparse features, and severe noise, pose an enormous challenge to it. For addressing these issues, this paper proposes an open-source fiducial-based underwater stereo visual-inertial localization method under the extended Kalman filter (EKF) framework, which is called FBUS-EKF. First, the refraction is corrected by the refractive camera model and akin triangulation. Second, the fiducial marker and a novel marker pose estimation method are applied to alleviate the adverse effect of sparse features. Third, the EKF is utilized to fuse the inertial and visual information so as to reject the serious noise. Finally, extensive experiments on a test bench demonstrate the effectiveness of the FBUS-EKF method, where the typical localization error is less than 3%, namely, the average error is lower than 3 cm within one meter. The obtained results reveal that the FBUS-EKF method has the prospect to be applied in the precise short-range operation and the localization for underwater robots, which offers a valuable insight for further autonomous underwater task.

In this paper, we propose a novel edge consistency metric for multi-modal correspondence. It is based on a novel observation on image truncated SVD (singular value decomposition) termed regression robustness, which describes the fact that, a good approximation from image truncated SVD can be inherited even if the eigen-images change due to expansion and channel-dependent offsets. Compared to state-of-the-arts, multi-modal edge consistency metric can simultaneously handle multiple images with complex modality changes, including local variation, gradient reverse, intensity order change, and texture loss. Its complexity is almost linear to pixel number. Remarkable accuracies have been achieved in experiments.

Object detection has been vigorously investigated for years but fast accurate detection for real-world scenes remains a very challenging problem. Overcoming drawbacks of single-stage detectors, we take aim at precisely detecting objects for static and temporal scenes in real time. Firstly, as a dual refinement mechanism, a novel anchor-offset detection is designed, which includes an anchor refinement, a feature location refinement, and a deformable detection head. This new detection mode is able to simultaneously perform two-step regression and capture accurate object features. Based on the anchor-offset detection, a dual refinement network (DRNet) is developed for high-performance static detection, where a multi-deformable head is further designed to leverage contextual information for describing objects. As for temporal detection in videos, temporal refinement networks (TRNet) and temporal dual refinement networks (TDRNet) are developed by propagating the refinement information across time. We also propose a soft refinement strategy to temporally match object motion with the previous refinement. Our proposed methods are evaluated on PASCAL VOC, COCO, and ImageNet VID datasets. Extensive comparisons on static and temporal detection verify the superiority of DRNet, TRNet, and TDRNet. Consequently, our developed approaches run in a fairly fast speed, and in the meantime achieve a significantly enhanced detection accuracy, i.e., 84.4% mAP on VOC 2007, 83.6% mAP on VOC 2012, 69.4% mAP on VID 2017, and 42.4% AP on COCO. Ultimately, producing encouraging results, our methods are applied to online underwater object detection and grasping with an autonomous system. Codes are publicly available at https://github.com/SeanChenxy/TDRN.

In this article, an extended state observer (ESO)-based controller with a model predictive governor is designed for 3-D trajectory tracking of underactuated underwater vehicles. The proposed control scheme takes three primary challenges including underactuated property, velocity constraint, and lumped disturbance into consideration. With respect to the model predictive governor, an underactuated kinematic tracking error model is utilized to produce reference velocities. Meanwhile, a heading angle compensation mechanism is utilized to avoid the steady tracking errors resulting from dynamics coupling of the vehicle. Besides, an ESO is designed to estimate the lumped disturbances and unmeasured velocity states. Based on the ESO, a kinetic controller is offered to accomplish the precise velocity tracking only in virtue of the position and orientation information. Note that this article details both the design process of the control scheme and rigorous theoretical analysis. Eventually, simulation and experimental results demonstrate the feasibility and superiority of the proposed method. Notably, this work lays the foundation for the underactuated trajectory tracking control in complicated and turbulent underwater environments.

This article presents the mechanical design and locomotion control of a novel tuna-inspired robot system for both fast swimming and high maneuverability. Mechanically, the developed robotic fish named CasiTuna comprises three important parts, i.e., an innovative two-motor-actuated propulsive mechanism, a buoyancy adjustment structure, and a pair of pectoral fins. Unlike most robotic fishes' multiple concatenated links-based propulsive mechanism, CasiTuna's two-motor-actuated one places both motors in the anterior body and utilizes a transmission system to achieve tuna-like lateral undulations. Meanwhile, the buoyancy adjustment mechanism in conjunction with pectoral fins endows the robot with the capability of three-dimensional maneuverability. Kinematic and dynamic analyses are further conducted to reveal the interactive hydrodynamic forces. Regarding the locomotion control method, a bio-inspired central pattern generator-based controller is adopted to achieve multimodal swimming. In particular, two kinds of turning maneuvers are implemented and discussed. Aquatic experiments, including straight swimming, circular turning, and nearly static pitching validate the effectiveness of proposed mechatronic design and locomotion control methods. Remarkably, CasiTuna achieved a peak forward speed of 0.8 m/s (corresponding to 1.52 body lengths per second) and a minimum turning radius of less than 0.3 body lengths.

In this article, a robot system for intelligent water surface cleaner named IWSCR is developed to collect floating plastic garbage. It is able to accomplish three major tasks autonomously, i.e., cruise and detection, tracking and steering, and grasping and collection. The challenges behind these tasks involve how to realize the accurate and real-time garbage detection, how to resist the disturbances while IWSCR conducts vision-based steering, and how to grasp the floating garbage reliably despite the turbulent conditions on the surface of the water. To overcome these difficulties, three key techniques are proposed for IWSCR. First, the YOLOv3 network, which is widely applied in the high speed and accuracy object detection field, is trained on the proposed floating garbage dataset to realize accurate and real-time garbage detection. Next, to improve the ability of resisting disturbances, a control law based on the sliding-mode controller is proposed for vision-based steering. Furthermore, inspired by the stability of floating bottles in fluid, a feasible grasping strategy is utilized for IWSCR. Finally, the experimental results demonstrate that IWSCR is competent to carry out the task of water surface cleaning.

This article proposes a modified rapidly exploring random tree (RRT)-based path planner and behavior-based cooperative tracking strategy for a dual robotic dolphin system to fulfill a cooperative target-tracking task. Specifically, with full consideration of both task requirements and mechatronic configuration, a robotic dolphin with a waist-caudal propulsive mechanism for thrust forces and differential bilateral flippers for maneuverability is developed. To satisfy the demand of fast path planning, a variant RRT algorithm is employed to generate feasible paths for the dual robotic dolphins with fewer waypoints, faster convergence speed, and better stability. Furthermore, a behavior-based approach in conjunction with centralized architecture is implemented to achieve high-level decision-making. Finally, simulations, analysis, as well as field experiments are carried out to verify the effectiveness of the proposed control scheme. The success of the experiments further offers insight into the mechanisms of cooperative multirobot target tracking in aquatic environments.

The quadruped robot imitates the motions of four-legged animals with a superior flexibility and adaptability to complex terrains, compared with the wheeled and tracked robots. Its leader-following ability is unique to help a human to accomplish complex tasks in a more convenient way. However, long-term following is severely obstructed due to the high-frequency vibration of the quadruped robot and the unevenness of terrains. To solve this problem, a visual approach under a novel T-D-R framework is proposed. The proposed T-D-R framework is composed of a visual tracker based on correlation filter, a person detector with deep learning, and a person re-identification (re-ID) module. The result of the tracker is verified by the detector to improve tracking performance. Especially, the re-ID module is introduced to handle distractions and occlusion caused by other persons, where the convolutional correlation filter (CCF) is employed to discriminate the leader among multiple persons through recording the appearance information in the long run. By comparing the results of the tracker and the detector as well as their similarity scores with the leader identified by the re-ID module, a stable and real-time tracking of the leader can be guaranteed. Experiments reveal that our approach is effective in handling distractions, appearance changes, and illumination variations. A long-distance experiment on a quadruped robot indicates the validity of the proposed approach.

This paper presents a three-dimensional (3-D) path planning method that combines the gliding with dolphin-like motions for the gliding robotic dolphin. A specific task that the robot uses the gliding motion for long-distance cruise and the dolphin-like motion for maneuverable obstacle avoidance is employed. The results of simulations and aquatic experiments validate the full-state dynamic model and the specific task, further offer some theoretical supports for 3-D path planning. Further, the 3-D path planning method is composed of three main components: 1) gliding path generation; 2) improved Astar (A*) algorithm; and 3) segmented Bezier curve smoothing. First, the gliding path is generated autonomously with the kinematic constraints that are obtained via the simulations of dynamic model. Furthermore, when the obstacles are detected by the sonar, an improved A* algorithm is employed to avoid the obstacles. Afterward, considering the path planned by A* is unsmoothed, a segmented Bezier curve method is presented. Simulation results demonstrate the effectiveness of the method, offering valuable insight into the utilization of hybrid underwater robots in the context of real-time task execution.

This article proposes a model predictive control (MPC)-based depth control system for the gliding motion of a gliding robotic dolphin. An injector-based buoyancy-driven mechanism is employed to achieve more precise control of net buoyancy. In the system, a novel framework of depth control is proposed on the basis of a simplified model, including a depth controller with improved MPC, a heading controller with velocity-based proportional-integral-derivative, and a sliding mode observer. Extensive simulation and experimental results demonstrate the effectiveness of the proposed control methods. In particular, a variety of slider-based experiments are also conducted to explore the performance of a movable slider in the depth control so as to better govern the gliding angle. The results obtained reveal that it is feasible to realize regular gliding angles via regulating the slider, which offers promising prospects for bio-inspired gliding robots playing a key role in ocean exploration.

Implementing underwater target tracking remains difficult for a free-swimming robotic fish owing to the intrinsically reciprocating motion in fishlike propulsion. In this article, we present a novel robotic fish platform with a camera stabilizing system and achieve real-time two-dimensional target tracking assisted by reinforcement learning (RL) in continuous environments. More specifically, we first develop an active visual tracking system based on cascade control structure to obtain the relative orientation between the robotic fish and the underwater target. Then, we propose a target tracking controller dealing with continuous state and action spaces based on deep RL (DRL). The controller takes the position of the target object as input and yields the motion parameters of the bioinspired central pattern generator governed robotic fish. The robustness and adaptability of the proposed controller as well as the influence of time-delays on the control system are explored via simulated experiments under different scenarios. Finally, both static and dynamic tracking experiments on the actual robotic fish demonstrate the effectiveness of the proposed mechatronic design and control methods, providing insights to executing aquatic vision-based tracking tasks.

Temporal object detection has attracted significant attention, but most popular detection methods cannot leverage rich temporal information in videos. Very recently, many algorithms have been developed for video detection task, yet very few approaches can achieve real-time online object detection in videos. In this paper, based on the attention mechanism and convolutional long short-term memory (ConvLSTM), we propose a temporal single-shot detector (TSSD) for real-world detection. Distinct from the previous methods, we take aim at temporally integrating pyramidal feature hierarchy using ConvLSTM, and design a novel structure, including a low-level temporal unit as well as a high-level one for multiscale feature maps. Moreover, we develop a creative temporal analysis unit, namely, attentional ConvLSTM, in which a temporal attention mechanism is specially tailored for background suppression and scale suppression, while a ConvLSTM integrates attention-aware features across time. An association loss and a multistep training are designed for temporal coherence. Besides, an online tubelet analysis (OTA) is exploited for identification. Our framework is evaluated on ImageNet VID dataset and 2DMOT15 dataset. Extensive comparisons on the detection and tracking capability validate the superiority of the proposed approach. Consequently, the developed TSSD-OTA achieves a fast speed and an overall competitive performance in terms of detection and tracking. Finally, a real-world maneuver is conducted for underwater object grasping.

This paper presents the impulsive average quasi-consensus algorithm for distributed constrained convex optimization. First, the constrained optimization problem can be transformed into an unconstrained problem using the interior point method, and then a distributed algorithm is modeled by means of impulsive differential equation. In the framework of the continuous-time gradient method and algebraic graph theory, each agent can deal with one local objective function with local constraints. At the impulsive instants, each agent can communicate with its neighboring agents over the network. Under certain conditions, the impulsive average quasi-consensus is achieved. It is shown that the state of average quasi-consensus is the optimal solution of the aforementioned unconstrained optimization problem, and the state of each agent can also reach the neighborhood of the optimal solution. Finally, two numerical examples show the effectiveness of the proposed impulsive average quasi-consensus algorithm. Moreover, the feasibility of the approach is verified by an application to one sensor network localization problem.

In this correspondence paper, an image dynamics-based visual servoing for quadrotors is proposed to realize stable hovering and tracking. Four perspective image moments are adopted as visual features to control all the independent degrees of freedom of a quadrotor. The complicated interaction matrix is simplified by projecting original image to virtual image plane. On this basis, the dynamics of the system is determined by considering the dynamics of image features and the quadrotor simultaneously. Backstepping controllers are then designed to stabilize the visual servoing system of the quadrotor. In reality, it is unrealistic to have exact prior knowledge about the trajectory parameters of an unpredictable moving target. To solve this problem, a trajectory observer based on nonlinear tracking-differentiator to estimate trajectory parameters of the target is firstly integrated into the quadrotor with image dynamics, which guarantees a satisfactory performance. The effectiveness of the proposed approach is verified by simulations.

Most existing robotic fish have a large body size driven by servo motor system, while conventional small-sized actuators hardly generate a high swimming performance. This paper reports a miniature untethered robotic fish, whose body length is 69 mm. In particular, a newly designed magnetic actuator system (MAS) is equipped, which guarantees both small-sized dimension and flexibility of the robot. More specifically, the magnetic field generated by a permanent magnet is first investigated based on Biot-Savart law. Then, a novel tail-beating rhythm called magnetically actuated pulse width modulation (MAPWM) is modeled for the new actuator system. Further, an MAPWM-based control method is presented, in which the duty ratio of MAPWAM is innovatively utilized to realize the turning maneuvers for the first time. In addition, Lagrangian method is employed to establish the dynamic model to assess the MAPWM-based control method and the turning performance of the robotic fish. To further improve the maneuverability, the effect of a shape-variable caudal fin is analyzed based on computational fluid dynamics and the built dynamic model. Finally, combined with the MAS, the MAPWM-based control method, and the optimally selected caudal fin, extensive aquatic experiments are conducted on the robotic prototype. The results indicate that the developed miniature robotic fish achieves a considerably higher level of maneuverability in terms of turning radius when compared to swimming robots with equivalent dimensions.

In this paper, adaptive quantized state estimation fusion is deeply studied. To approach the model mismatching problem induced by random quantization, some quantized Kalman filters have been presented in the previous work, such as the quantized Kalman filter with strong tracking filtering (QKF-STF), the variational Bayesian adaptive quantized Kalman filter (VB-AQKF), and a centralized fusion frame-based complex quantized filter called variational Bayesian adaptive QKF-STF (VB-AQKF-STF). Based on the previous work for the single sensor system, a distributed complex quantized filter is designed in this paper. A novel quantized Kalman filter based on multiple-method fusion scheme (QKF-MMF) is proposed. Similar to the VB-AQKF-STF, the QKF-MMF can also realize joint estimation on the state and the quantization error covariance under the distributed fusion frame. Furthermore, it extends the single sensor results to multisensor tracking systems by using centralized and distributed fusion frames. Two multisensor quantized fusion estimators are proposed for a parallel structure with main-secondary processors in the fusion center. The weighted fusion and embedded integration ways are deeply applied to design the multisensor quantized fusion methods. The proposed work can perfect the quantized estimation algorithms and provide different choices for practical engineering applications.

Femtocell is a promising technology for wireless service networks to facilitate sustainable and efficient services for users. This paper deals with a downlink joint channel assignment and power allocation problem with multiple channels, users, constraints, and uncertainties in the orthogonal frequency division of a multiple-access hierarchical femtocell network system. Specifically, a hierarchical robust Stackelberg game, which aims to achieve robust equilibrium, is first proposed for resource allocation with uncertainties. Then, a low-complexity, low-interference, high-efficiency, and high-performance algorithm is presented to handle the complex robust joint allocation problem. Considering the demand capacity of macro-base stations, an efficient fitness function in conjunction with particle swarm optimization-constriction factor, is utilized to yield the best response in the upper game to satisfy multiple constraints. Meanwhile, an iterative waterfilling algorithm is exploited to achieve the best response of femto-base stations in the lower game. Lastly, a stop protocol is established for two-tier users to accomplish an efficient robust Stackelberg equilibrium. Comparative results demonstrated that the proposed algorithm is superior to the existing game-based algorithms.

Object detection methods fall into two categories, i.e., two-stage and single-stage detectors. The former is characterized by high detection accuracy while the latter usually has a considerable inference speed. Hence, it is imperative to fuse their merits for a better accuracy vs. speed trade-off. To this end, we propose a dual refinement network (DRN) to boost the performance of the single-stage detector. Inheriting from the advantages of two-stage approaches (i.e., two-step regression and accurate features for detection), anchor refinement and feature offset refinement are conducted in a novel anchor-offset detection, where the detection head is comprised of deformable convolutions. Moreover, to leverage contextual information for describing objects, we design a multi-deformable head, in which multiple detection paths with different receptive field sizes devote themselves to detecting objects. Extensive experiments on PASCAL VOC and ImageNet VID datasets are conducted, and we achieve a state-of-the-art detection performance in terms of both accuracy and inference speed.

This paper considers the cost-driven optimal energy management strategy under a complex environment, multimicrogrids system. To provide the flexibility of load in depth, the heating, ventilating, and air conditioning (HVAC) load is investigated and explicitly formulated incorporated with indoor temperature dynamic function. The operation cost of wind generation includes the basic cost and additional cost to deal with the uncertainty of wind generation, which provides the tradeoff between optimality and possibility. Furthermore, the energy management problem of multimicrogrid is presented according to different characteristics of generation devices, storage devices, and load. A distributed neurodynamic algorithm is presented to solve the nonsmooth optimization of energy management of multimicrogrids system. By this method, the only information exchanged among microgrids is the intermediate variable when computing. The simulation results validate the effectiveness of the proposed cost-driven energy management strategy.

Based on the multiagent networks, this paper introduces a continuous-time algorithm to deal with distributed convex optimization. Using nonsmooth analysis and algebraic graph theory, the distributed network algorithm is modeled by the aid of a nonautonomous differential inclusion, and each agent exchanges information from the first-order and the second-order neighbors. For any initial point, the solution of the proposed network can reach consensus to the set of minimizers if the graph has a spanning tree. In contrast to the existing continuous-time algorithms for distributed optimization, the proposed model holds the least number of state variables and relaxes the strongly connected weighted-balanced topology to the weaker case. The modified form of the proposed continuous-time algorithm is also given, and it is proven that this algorithm is suitable for solving distributed problems if the undirected network is connected. Finally, two numerical examples and an optimal placement problem confirm the effectiveness of the proposed continuous-time algorithm.

This paper addresses the problem of path following for biomimetic underwater vehicles (BUVs) propelled by undulatory ribbon-fins. First, the general kinematics and dynamics models of underwater vehicles are presented, followed by a fuzzy logic model for dealing with a nonlinear relationship between the propulsive force/torque and the control parameters of the undulatory fins of the BUV. Then the path following problem of the BUV is formulated. A path following control paradigm integrating the line-of-sight guidance system with backstepping (BP) technique is proposed to maneuver the BUV to follow a predefined parameterized curve without time constraints. The stability of the BP controller is analyzed and guaranteed by Lyapunov stability theory. Finally, simulations and experimental results illustrate the performance of the proposed path following control paradigm.

Temporal object detection has attracted significant attention, but most popular methods can not leverage the rich temporal information in video or robotic vision. Although many different algorithms have been developed for video detection task, real-time online approaches are frequently deficient. In this paper, based on attention mechanism and convolutional long short-term memory (ConvLSTM), we propose a temporal single-shot detector (TSSD)for robotic vision. Distinct from previous methods, we aim to temporally integrate pyramidal feature hierarchy using ConvLSTM, and design a novel structure including a high-level ConvLSTM unit as well as a low-level one (HL-LSTM)for multi-scale feature maps. Moreover, we develop a creative temporal analysis unit, namely, ConvLSTM-based attention and attention-based ConvLSTM (A&CL), in which the ConvLSTM-based attention is specially tailored for background suppression and scale suppression while the attention-based ConvLSTM temporally integrates attention-aware features. Finally, our method is evaluated on ImageNet VID dataset. Extensive comparisons on detection performance confirm the superiority of the proposed approach, and the developed TSSD achieves a considerably enhanced accuracy vs. speed trade-off, i.e., 64.8% mAP vs. 27 FPS.

The conventional inspection of fragile insulators is critical to grid operation and insulator segmentation is the basis of inspection. However, the segmentation of various insulators is still difficult because of the great differences in colour and shape, as well as the cluttered background. Traditional insulator segmentation algorithms need many artificial thresholds, thereby limiting the adaptability of algorithms. A compact end‐to‐end neural network, which is trained in the framework of conditional generative adversarial networks, is proposed for the real‐time pixel‐level segmentation of insulators. The input image is mapped to a visual saliency map, and various insulators with different poses are filtered out at the same time. The proposed two‐stage training and empty samples are also used to improve the segmentation quality. Extensive experiments and comparisons are performed on many real‐world images. The experimental results demonstrate superior segmentation and real‐time performance. Meanwhile, the effectiveness of the proposed training strategies and the trade‐off between performance and speed are analysed in detail.

This paper proposes two second-order continuous-time algorithms to solve the economic power dispatch problem in smart grids. The collective aim is to minimize a sum of generation cost function subject to the power demand and individual generator constraints. First, in the framework of nonsmooth analysis and algebraic graph theory, one distributed second-order algorithm is developed and guaranteed to find an optimal solution. As a result, the power demand constraints can be kept all the time under appropriate initial condition. The second algorithm is under a centralized framework, and the optimal solution is robust in the sense that different initial power conditions do not change the convergence of the optimal solution. Finally, simulation results based on five-unit system, IEEE 30-bus system, and IEEE 300-bus system show the effectiveness and performance of the proposed continuous-time algorithms. The examples also show that the convergence rate of second-order algorithm is faster than that of first-order distributed algorithm.

In this paper, we design and build a power line inspection robot capable of hybrid operation modes. Specifically, the developed robot is able to land on the overhead ground wire (OGW) and to move as the climbing robot. When to negotiate obstacles, it can vertically take off the wire and fly over the obstacles as the unmanned aerial vehicle (UAV). A customized trumpet-shaped undercarriage is used to guarantee that the robot can land and move safely. With the aid of a swingable 2D Laser Range Finder (LRF), the robot can not only determine whether there are obstacles but also detect the position and orientation of the OGW, making it suitable for automatic inspection of power lines. The outdoor experimental results1demonstrate the effectiveness of the robot in landing and obstacle negotiation. In addition, the average power consumption of the robot is much lower than that of traditional flying robots for power line inspection.

Recently, projection neural network (PNN) was proposed for solving monotone variational inequalities (VIs) and related convex optimization problems. In this paper, considering the inertial term into first order PNNs, an inertial PNN (IPNN) is also proposed for solving VIs. Under certain conditions, the IPNN is proved to be stable, and can be applied to solve a broader class of constrained optimization problems related to VIs. Compared with existing neural networks (NNs), the presence of the inertial term allows us to overcome some drawbacks of many NNs, which are constructed based on the steepest descent method, and this model is more convenient for exploring different Karush-Kuhn-Tucker optimal solution for nonconvex optimization problems. Finally, simulation results on three numerical examples show the effectiveness and performance of the proposed NN.

Fractional-order neural networks play a vital role in modeling the information processing of neuronal interactions. It is still an open and necessary topic for fractional-order neural networks to investigate their global stability. This paper proposes some simplified linear matrix inequality (LMI) stability conditions for fractional-order linear and nonlinear systems. Then, the global stability analysis of fractional-order neural networks employs the results from the obtained LMI conditions. In the LMI form, the obtained results include the existence and uniqueness of equilibrium point and its global stability, which simplify and extend some previous work on the stability analysis of the fractional-order neural networks. Moreover, a generalized projective synchronization method between such neural systems is given, along with its corresponding LMI condition. Finally, two numerical examples are provided to illustrate the effectiveness of the established LMI conditions.

This paper presents a multirobot pairwised transportation (MRPWT) approach for factory automated material and product deliveries. We consider MRPWT from the viewpoint of robotics and incorporate practical factory application constraints in the transportation method design. The proposed MRPWT approach is a two-level hybrid planning method, consisting of an incidental delivery based single robot level planner and a simulated annealing based robot group level planner. Each robot resolves its individual transportation plan incidentally to reduce the transportation cost, whereas the group level planner utilizes predefined random actions to search the task assignment solution space and then incorporates the simulated annealing algorithm to resolve the MRPWT problem as a combinatorial optimization problem. By implementing a distributed auction mechanism, the proposed MRPWT approach can be further extended to resolve the online task allocation or reallocation problem in dynamic environments. Experiments performed on a group of mobile robots successfully demonstrate the effectiveness and the practical applicability of the proposed MRPWT approach for factory automated material and product deliveries.

The optimal formation problem of multirobot systems is solved by a recurrent neural network in this paper. The desired formation is described by the shape theory. This theory can generate a set of feasible formations that share the same relative relation among robots. An optimal formation means that finding one formation from the feasible formation set, which has the minimum distance to the initial formation of the multirobot system. Then, the formation problem is transformed into an optimization problem. In addition, the orientation, scale, and admissible range of the formation can also be considered as the constraints in the optimization problem. Furthermore, if all robots are identical, their positions in the system are exchangeable. Then, each robot does not necessarily move to one specific position in the formation. In this case, the optimal formation problem becomes a combinational optimization problem, whose optimal solution is very hard to obtain. Inspired by the penalty method, this combinational optimization problem can be approximately transformed into a convex optimization problem. Due to the involvement of the Euclidean norm in the distance, the objective function of these optimization problems are nonsmooth. To solve these nonsmooth optimization problems efficiently, a recurrent neural network approach is employed, owing to its parallel computation ability. Finally, some simulations and experiments are given to validate the effectiveness and efficiency of the proposed optimal formation approach.

In this brief, we investigate the parameter optimization issue of a central pattern generator (CPG) network governed forward and backward swimming for a fully untethered, multijoint biomimetic robotic fish. Considering that the CPG parameters are tightly linked to the propulsive performance of the robotic fish, we propose a method for determination of relatively optimized control parameters. Within the framework of evolutionary computation, we use a combination of dynamic model and particle swarm optimization (PSO) algorithm to seek the CPG characteristic parameters for an enhanced performance. The PSO-based optimization scheme is validated with extensive experiments conducted on the actual robotic fish. Noticeably, the optimized results are shown to be superior to previously reported forward and backward swimming speeds.

This paper presents a bio-inspired caterpillar-like climbing method. Natural caterpillars climb relatively slow, but their multisegmented body trunk strongly enhances climbing versatility and stability, which thus motivates us to design an imitative robot to study the caterpillar-like climbing locomotion. Based on observations from natural caterpillars, we propose a three-stage climbing method to imitate the caterpillar-like straight line climbing on a planar wall. Due to the redundancy property of caterpillars' multisegmented body trunk, we formulate the caterpillar-like climbing problem as an end-effector tracking problem using the redundant robotics terminology. A dual-mode optimal controller, which effectively resolves the end-effector tracking problem even when the robot configuration is ill-conditioned, is incorporated for realizing the caterpillar-like climbing locomotion. Both simulation and experiment results are presented to demonstrate the effectiveness of the proposed caterpillar-like climbing method.

This paper surveys the developments of the last 20 years in the field of central pattern generator (CPG) inspired locomotion control, with particular emphasis on the fast emerging robotics-related applications. Functioning as a biological neural network, CPGs can be considered as a group of coupled neurons that generate rhythmic signals without sensory feedback; however, sensory feedback is needed to shape the CPG signals. The basic idea in engineering endeavors is to replicate this intrinsic, computationally efficient, distributed control mechanism for multiple articulated joints, or multi-DOF control cases. In terms of various abstraction levels, existing CPG control models and their extensions are reviewed with a focus on the relative advantages and disadvantages of the models, including ease of design and implementation. The main issues arising from design, optimization, and implementation of the CPG-based control as well as possible alternatives are further discussed, with an attempt to shed more light on locomotion control-oriented theories and applications. The design challenges and trends associated with the further advancement of this area are also summarized.

This paper is devoted to the active turn control of a free-swimming multilink dolphin-like robot, with emphasis on yaw and pitch controls. With full consideration of both mechanical configuration and propulsive principle of the robot consisting of a yaw joint and multiple pitch joints, a viable approach to perform yaw maneuvers via laterally directed biases is formed, providing an advantage in qualitative and quantitative assessment. Meanwhile, based on the feedback of the pitch angle measured by an onboard gyroscope, a closed-loop control strategy in dorsoventral motions is proposed to achieve agile and swift pitch maneuvers. More remarkably, two hybrid acrobatic stunts, i.e., frontflip and backflip, are first implemented on the physical robot. The latest results obtained demonstrate the effectiveness of the proposed methods. It is also confirmed that the dolphin robot achieves better performance for pitch maneuvers than it does for yaw maneuvers, agreeing well with the biological observations.

This paper presents the behavior design and multimodal locomotion control of a biomimetic amphibious robot based on a bio-inspired CPG (central pattern generator). A set of four key parameters are introduced serving as external stimuli to shape the CPG rhythmic activities where necessary speed and orientation modulation as well as 3-D locomotion can be obtained. In terms of the built parameter set, a library of movement primitives based on finite state machine is established to facilitate rapid and smooth gait transitions. To enhance adaptive behaviors, well-integrated sensory feedback by means of two liquid-level detectors enables the gait transition between ground and water autonomously. Simulations and experiments are also conducted to demonstrate the feasibility of a behavior based control architecture governed by CPGs.

In this paper, we present the formulation of a dynamic model of a free-swimming multi-joint robotic fish with a pair of wing-like pectoral fins, in which the whole robot is regarded as a moving multilink rigid body in fluids. Considering that the thrust of fish mainly results from the force of trailing vortex, added lateral pressure, and leading-edge suction force, the dynamic equations of the swimming fish have been derived by summing up the longitudinal force, lateral force, and yaw moment on each propulsive component in the framework of Lagrangian mechanics. Furthermore, using the bio-inspired Central Pattern Generators (CPGs) as the swimming data generator, the overall dynamic propulsive characteristics of the swimming robot are estimated in a mathematical environment (i.e., Mathematica). As a case study, the created dynamic model offers a good guide to seeking pragmatic backward swimming patterns for a carangiform robotic fish, which exemplifies the validity of the CPG-coupled dynamic model.

Presented in this paper is our effort to create a multifunctional swimming robot, i.e., robotic fish, inspired by the well-integrated, configurable multiple control surfaces existing in real fish. By virtue of the hybrid propulsion capability in the tail plus the caudal fin and the maneuverability in accessory fins, a novel, synthesized propulsion scheme composed of multiple artificial control surfaces is proposed, involving the tail plus the caudal fin, pectoral fins, pelvic fin, and dorsal fin. Multimodal locomotion is then accomplished by manipulation of control surfaces, separately or cooperatively, allowing the robot to maneuver more diversely and agilely. In particular, bio inspired Central Pattern Generators (CPGs) based locomotion control is adopted for online swimming gait generation. Aquatic testing has been carried out to demonstrate the improved maneuverability and stability of the robotic fish underwater as well as the effectiveness of the conceived multi-fin mechatronic design.

This paper presents a control architecture for the underwater locomotion control of a biomimetic amphibious robot with multi-mobility mechanism. In view of both hydrodynamic problem and engineering approach, we develop a robotic prototype capable of multi-mode motion. A robust gait control for steady swimming using the central pattern generator (CPG) is proposed and has been successfully applied to the robot. The CPG can produce coordinated patterns of rhythmic activity while being simply modulated by control parameters including input drive, frequency, amplitude, threshold, etc., which will be suitable for manually interactive modulation. Using the CPG model, the robot is capable of performing and switching between various locomotion modes such as swimming forwards and backwards, turning and pitching, with the speed, direction and gait types modulated accordingly. A test-bed is provided and results are presented demonstrating interesting properties of the CPG-based control approach and feasibility of the CPG control for efficient propulsion.

This paper deals with a novel closed-loop maneuvering control method to enhance the turning precision and turning response speed of a robotic fish propelled via the body and/or caudal fin (BCF) mode. Although the BCF propulsion is favorable for the cases requiring greater thrust and accelerations, its maneuverability can be compensated by effective turning control. In our method, the turning maneuver is divided into three phases: the bending, holding, and unbending phases. After much consideration on turning details, the functions of each phase and the basic control laws are further identified. Results of experiments on in-situ direction tracking and direction maintaining verify the effectiveness of the proposed turning control.

This paper presents an efficient kinematical solution to a multi-robot system with serial and parallel mechanisms. JL-I is a reconfigurable robot featuring active spherical joints formed by serial and parallel mechanisms endowing the robotic system with the ability of changing shapes in three dimensions. The active joint here can combine the advantages of the high rigidity of a parallel mechanism and the extended workspace of a serial mechanism. However, the kinematic analysis of the serial and parallel mechanism is always the bottleneck in designing a robot and control realization. In order to deal with this problem, the whole kinematical analysis is organized in the sequence from the direct mechanical analysis related to the serial and parallel mechanism over the numerical solutions to the simplified kinematics expression. The latest results obtained demonstrate that the deduced closure-form solution is time efficient and easy to implement while offering a satisfactory motion performance in on-site experiments.

This paper presents a new turning maneuver generation method for a multilink biomimetic robotic fish, in which smooth step functions are introduced to dynamically trigger directed offsets in active and asymmetric swimming. With the proposed method, three basic turning modes can be unified into a general framework by choosing appropriate step-function combinations and dynamic bias. Furthermore, this method can be employed to maneuver the robotic fish agilely in the path planning, which promises more flexibility and steadiness in potential applications to bio-inspired autonomous underwater vehicles.

This paper deals with maneuver issues of a multilink biomimetic robotic fish, particularly focusing on turning control in free swimming. The characteristic parameters determining turning performance involve magnitude, position, and time of the deflections applied to the links, which are discussed via a series of simulation calculations and actual experiments.

This paper focuses on the design of fishlike underwater robots using an optimization approach to choose relative link lengths. Considering both ichthyologic characteristics and mechatronic constraints, the optimal link-length ratios are numerically calculated by an improved constrained cyclic variable method. Comparative results, before and after the optimization, demonstrate the enhanced performance

This paper is concerned with a novel problem of underwater cooperative transportation of multiple fishlike robots. A situation based action selection mechanism is proposed for the robots to transport a floating object to its destination. There are various control methods for cooperation of multi-robot system, but few of them deal with the underwater applications. In this paper, we first present the development of a series of biomimetic fish-like robots. Then, employing these robotic fish, we design and implement a cooperative transportation task utilizing a situation based action selection approach. We fully implement our method on three fish-like robots and make various experiments in a lab environment

This paper is concerned with the design, construction, and control of a biomimetic robotic dolphin equipped with mechanical flippers, based on a simplified engineered propulsive model. The robotic dolphin is modeled as a three-segment organism composed of rigid anterior body, flexible rear body, and an oscillating fluke. The dorsoventral movement of the tail produces the thrust, and bending of the body in the horizontal plane enables turning maneuvers. A dual-microcontroller structure is proposed to drive the oscillating multi-link rear body and the mechanical flippers. Swimming performance of the prototype robotic dolphin is tested, and the results confirm the effectiveness of the dolphin-like movement in propulsion and maneuvering

This paper presents a three-dimensional, dynamic model of robotic fish which synthesizes both the carangiform and anguilliform swimming modes. The designed robotic fish is composed of three parts: stiff anterior body with a pair of pectoral fins for up-and-down motion, flexible rear body, and an oscillating lunate caudal fin. We use unsteady flow theory to analyze the motion of the anterior part and the links, and adopt experimental result from oscillating foil for the caudal fin. So the dynamic equation of each part can be obtained, and by summing up these equations, the dynamic equation for total swimming robot can be derived. The desired propulsive characteristics including forward velocity, sway velocity, pitch velocity, three velocity of Eulerian angles of the stiff anterior body, motion trajectory as well as propulsive efficiency can then be obtained by solving ordinary differential equation. The circular motion, based on asymmetrical kinematics of flexible rear body, can be achieved by adding different deflections to the oscillatory links. Comparisons between simulation results and real experiments are then conducted and discussed. A good agreement on dynamic characteristics demonstrates the validity of the proposed model

This paper presents a new method for formation control of multiple underwater fish-like robots. Considering both geometrical and mechanical constrains of the fish-like robots and based on the leader-following approach, a curvature coordinate is introduced to describe relative positions between different members within formations. Both the static and dynamic formations are concerned. We conduct simulations and physical experiments to verify effectiveness of the proposed algorithms

This paper deals with maneuver issues, in particular turning control of a free-swimming, multi-link biomimetic robotic fish propelled by a flexible rear body and oscillatory tail fin. Taking into account of both hydrodynamic model and engineering use, we develop a practical method to execute a circular maneuver. The turning control, based on asymmetrical kinematics and dynamics, can be achieved by adding different deflections to the oscillatory links. The characteristic parameters affecting turning performance include magnitude, position, and time of the deflections applied to the links, which are discussed with a series of simulation calculations. The experiments of circular motion with a four-link, infrared sensor-equipped robotic fish acquire a satisfactory maneuverability. Moreover, the feasibility of adding deflection angles to maneuver the robotic fish in the path planning is explored both analytically and experimentally

This paper presents the construction and motion control of a biologically inspired, multi-mode biomimetic robotic fish capable of three-dimensional locomotion. The mechanical configuration and the control system are described. The rhythmic movements of swimming are driven by the biological neural networks, called the central pattern generators (CPGs). The CPGs are modeled as nonlinear oscillators for joints and inter-joint coordination is achieved by altering the connection weights between joints. The CPG-based online gait generation method allows elegant and smooth transitions between swimming gaits, which result in multi-mode swimming to implement more lifelike locomotion. Several swimming modes can be obtained to mimic diverse actions of the real fish in nature, or designed according to special assignments by changing CPG parameters. The up-and-down motion can be implemented by adjusting the rotating angle of pectoral fins. The depth of the robotic fish is controlled utilizing a proportional-integral-derivative (PID) control algorithm according to the feedback of depth information measured by the sonar sensor installed at the bottom of the fish. The experimental results confirm the effectiveness of the control methods

This paper is concerned with the parameter optimization for a simplified propulsive model of biomimetic robot fish propelled by a multiple linked mechanism. Taking account of both theoretic hydrodynamic issues and practical problems in engineering realization, the optimal link-length ratio is numerically calculated by an improved constrained cyclic variable method. The result is successfully applied to the 4-linked robot fish developed in our laboratory. The comparative experiments on forward swimming speed of the robot fish before and after parameter optimization verify the effectiveness of our method.

This paper presents a new method for controlling a group of nonholonomic mobile robots to achieve predetermined formations without using global knowledge. Based on the dynamic leader-follower model, a reactive tracking controller is proposed to make each following robot maintain a desired pose to its leader, and the stability property of this controller is discussed using Lyapunov theory. By employing such controllers, the N-robot formation control problem can be decomposed into decentralized tracking problems between N-l followers and designated leaders. Additionally, graph theory is introduced to formalize general formation patterns in a simple but effective way and two types of switching between these formations are also proposed. Numerical simulations and physical robots experiments show the effectiveness of our approach.

This paper presents a hierarchical framework for controlling a group of biomimetic fish robots to achieve cooperative tasks. Based on our previous successful work on the design and development of a robotic fish prototype, we attempt further to investigate the cooperation in groups of these fish. Employing top-down design approach, we propose a hierarchical architecture consisting of five levels: task level, role (or mode) level, behavior level, action level and controller level, to formalize the processes from task decomposition, role assignments and control performance. Two typical cases are developed to demonstrate the feasibility of the architecture and corresponding experimental results show that high efficiency and much greater capabilities are exhibited when the fish try to cooperate.

Investigation in biomimetic robotic fish is a multidisciplinary problem entirely involving hydrodynamics-based control and robotic technology. In this paper, grounded on an optimized kinematic propulsive model of fish that relate frequency to speed and joint angle bias to turns, a robotic fish and its motion control are designed. Information of multiple fishes' position and orientation, with real-time visual tracking, is then obtained for positioning in the coordinate experiment. The coordinated motions of the fishes are achieved by behavior-based strategies composed of role assignment, behavior selection and role transition. An experimental system for multiple robotic fishes' coordination is finally constructed to verify the proposed scheme and algorithms, and the running experiments of passing-hole and pushing-ball primarily show the effectiveness of the presented strategies.

This paper is concerned with the problem of motion planning for multiple robotic fish swimming together to achieve a disk-pushing task. Based on our previous successful work on the design and development of robotic fish prototypes, we attempt further to investigate how to implement cooperation in groups of these fish. The motivation of this work is that the capability of a single fish robot is always limited while there are many complex missions in practice which should be accomplished by effective cooperation of multiple fish. A disk-pushing task is employed as an example to demonstrate the design of cooperative behaviors as well as strategies. Corresponding experimental results show that higher efficiency and much greater capabilities can be obtained when these fish planning their motion properly.

This paper is concerned with the design of a robotic fish and its motion control algorithms. A radio-controlled, four-link biomimetic robotic fish is developed using a flexible posterior body and an oscillating foil as a propeller. The swimming speed of the robotic fish is adjusted by modulating joint's oscillating frequency, and its orientation is tuned by different joint's deflections. Since the motion control of a robotic fish involves both hydrodynamics of the fluid environment and dynamics of the robot, it is very difficult to establish a precise mathematical model employing purely analytical methods. Therefore, the fish's motion control task is decomposed into two control systems. The online speed control implements a hybrid control strategy and a proportional-integral-derivative (PID) control algorithm. The orientation control system is based on a fuzzy logic controller. In our experiments, a point-to-point (PTP) control algorithm is implemented and an overhead vision system is adopted to provide real-time visual feedback. The experimental results confirm the effectiveness of the proposed algorithms.