Legged robots possess inherent advantages in traversing complex 3D terrains. However, previous work on lowcost quadruped robots with egocentric vision systems has been limited by a narrow front-facing view and exteroceptive noise, restricting omnidirectional mobility in such environments. While building a voxel map through a hierarchical structure can refine exteroception processing, it introduces significant computational overhead, noise, and delays. In this paper, we present MOVE, a one-stage end-to-end learning framework capable of multi-skill omnidirectional legged locomotion with limited view in 3D environments, just like what a real animal can do. When movement aligns with the robot's line of sight, exteroceptive perception enhances locomotion, enabling extreme climbing and leaping. When vision is obstructed or the direction of movement lies outside the robot's field of view, the robot relies on proprioception for tasks like crawling and climbing stairs. We integrate all these skills into a single neural network by introducing a pseudo-siamese network structure combining supervised and contrastive learning which helps the robot infer its surroundings beyond its field of view. Experiments in both simulations and real-world scenarios demonstrate the robustness of our method, broadening the operational environments for robotics with egocentric vision.

Quadruped robots are able to exhibit a range of gaits, each with its own traversability and energy efficiency characteristics. By actively coordinating between gaits in different scenarios, energy-efficient and adaptive locomotion can be achieved. This study investigates the performances of learned energy-efficient policies for quadrupedal gaits under different commands. We propose a training–synthesizing framework that integrates learned gait-conditioned locomotion policies into an efficient multiskill locomotion policy. The resulting control policy achieves low-cost smooth switching and controllable gaits. Our results of the learned multiskill policy demonstrate seamless gait transitions while maintaining energy optimality across all commands.

Gait planning is one of the most critical technologies for quadruped robots. However, far too little attention has been paid to the tight coupling mechanism of gait planning with terrain understanding and energy efficiency. To date, it is still challenging to plan optimal gait strategies that are highly adapted to terrain features with stable and efficient transitions. Accordingly, this paper proposes an adaptive gait control framework for quadruped robots that combines terrain recognition, Cost of Transport (CoT), and the Froude (Fr) number. More specifically, an optimal gait selection strategy for quadruped robots is designed based on different terrain texture features and the CoT characteristics of different gaits. To address the gait transition process induced thereby, an adaptive method for gait parameters based on the Fr number is further proposed, which can make the process more stable. Besides, model predictive control (MPC) and whole-body control (WBC) are employed as the motion controllers for the quadruped robot. Furthermore, simulation and experimental results indicate that the proposed method possesses superior terrain adaptability, energy efficiency, and motion stability during gait transitions, which is beneficial for the quadruped robots to maintain stable motion and reduce energy consumption when performing tasks in changeable terrains.Note to Practitioners—This paper is motivated by the problem of adaptive gait planning for quadruped robots that walks through different terrains. We propose a method that ensures optimal gait selection by robots facing diverse terrains and maintains the stability of gait transition. The proposed control framework, upon testing in a simulated environment, can be directly deployed on real-world robot without further adjustments and allows the robot to traverse various terrains with minimal sim-to-real issues. Hopefully, our proposed method can provide valuable guidance and support for facilitating the enhancement of capabilities in performing prolonged endurance tasks in unstructured environments for quadruped robots.

The realm of multi-robot autonomous exploration tasks underscores the critical role of communication in coordinating group activities. This paper introduces an innovative decentralized multi-robot exploration algorithm, meticulously crafted to ensure unbroken communication within robotic groups, a crucial element for effective coordination. The motivation for our work is two-fold: Firstly, seamless communication is vital for coordinating multi-robot autonomous exploration tasks. Secondly, in applications such as disaster rescue operations or military maneuvers, there are numerous scenarios where spatial congregation of multiple robots is imperative for joint task accomplishment. Our approach addresses these challenges through a stringent communication constraint, ensuring that each robot remains in constant communicative contact with the rest of the group. This is realized by employing a decentralized policy that integrates Graph Neural Network (GNN) layers with self-attention mechanism. Such policy network design allows adaptation to different numbers of robots and varied environments. After an initial imitation learning phase, the policy is refined through learning from experiences generated via a tree-search-based lookahead technique. Our experimental analysis validates that the algorithm not only maintains consistent communication links among all group members but also improve the exploration efficiency under the communication constraints. These results highlight the potential of our method in enhancing the effectiveness of robotic group explorations while ensuring robust communication connection.

Accurate state estimation plays a critical role in ensuring the robust control of humanoid robots, particularly in the context of learning-based control policies for legged robots. However, there is a notable gap in analytical research concerning estimations. Therefore, we endeavor to further understand how various types of estimations influence the decision-making processes of policies. In this paper, we provide quantitative insight into the effectiveness of learned state estimations, employing saliency analysis to identify key estimation variables and optimize their combination for humanoid locomotion tasks. Evaluations assessing tracking precision and robustness are conducted on comparative groups of policies with varying estimation combinations in both simulated and real-world environments. Results validated that the proposed policy is capable of crossing the sim-to-real gap and demonstrating superior performance relative to alternative policy configurations.

The problems of a 7-degree of freedom (DOF) manipulator with rapid and continuous response to uncertain fast-flying objects are addressed: 1) how to effectively solve trajectory planning of the 7-DOF manipulator with multiple criteria; and 2) how to make the 7-DOF manipulator realize the rapid and continuous response to uncertain fast-flying objects. In the proposed approach, based on the trajectory parameterization of the 7-DOF manipulator, a multiobjective teaching-learning-based optimization (MOTLBO) algorithm is adopted to find a close representation of the Pareto optimal set rather than a single solution. As such, an optimal solution can be chosen as digital knowledge information. A new methodology based on a knowledge base representing and learning the operation environment, that is, skill digitization, is presented, which enables the 7-DOF manipulator to realize the rapid and continuous response skill. Simulation and practical testing results of a ping-pong robot validate the feasibility and effectiveness of the proposed approach, in which the online trajectory generation spends only around 1 ms.

Global localization plays a critical role in many robot applications. LiDAR-based global localization draws the community's focus with its robustness against illumination and seasonal changes. To further improve the localization under large viewpoint differences, we propose RING++ that has roto-translation-invariant representation for place recognition and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multichannel features. To the best of our knowledge, this is the first learning-free framework to address all the subtasks of global localization in the sparse scan map. Validations on real-world datasets show that our approach demonstrates better performance than state-of-the-art learning-free methods and competitive performance with learning-based methods. Finally, we integrate RING++ into a multirobot/session simultaneous localization and mapping system, performing its effectiveness in collaborative applications.

Legged robots, specifically quadruped robots, have good locomotion performance in complex and rugged terrain and are becoming widely used in field exploration and rescue missions. To achieve full autonomy in such scenarios, robots need not only accurate localization but also an accurate understanding of the surrounding terrain, which will be used for robots path planning and foothold planning. However, due to the kinetic characteristic and limitation of size, quadruped robots have the disadvantages of high-frequency jitter and limited field of sensors, which lead to some challenges in environmental perception. In this paper, we propose a vision-assisted rugged terrain environment reconstruction and localization method for quadruped robots. We use a depth camera to assist in the generation of high-precision localization and terrain reconstruction results, which can help achieve the autonomous mobility of quadruped robots in this environment. We test our method on a quadruped robot platform. Our experimental results show less error and lower drift in different stairs terrain types than the commonly used lidar-based localization method.

Opening a lock without vision sensors remains a challenge for robots. Inspired by the ability of a human to open a lock through touch and intuition, a peg-in-hole assembly method for recognizing the relative position and inclination angle of a hole is proposed. We use supervised learning to generate a contact-state model to judge the relative contact state and introduce force control strategies that ensure stable and safe interaction with the environment. Adaptive impedance control is adopted to ensure the stability of the alignment and insertion process. The proposed method is not restricted by the object shape. The system can learn an effective classification model with a small volume of force and torque data and predict the relative contact state of a peg and hole. The proposed method is verified in an experiment in which a bicycle lock is opened at different inclination angles. The proposed method has potential application in the field of industrial assembly.

In this paper, an approach for automatic peg-in-hole assembly is proposed. The task is divided into two main steps: searching phase and inserting phase. First, a multilayer perceptron network is designed to address the hole search problem and a hybrid force position controller is introduced to ensure a safe and stable interaction with the external environment. Then, for the inserting phase, a variable impedance controller is adopted based on the fuzzy Q-learning algorithm to yield compliant behavior from the robot during the hole insertion process. This approach is a practical and general approach to solve complex peg-in-hole assembly problems by taking advantage of both learning-based algorithms and force control strategies, which can greatly improve the efficiency and safety of the industrial manufacturing process without identifying the unknown contact model and tuning tedious parameters. Finally, the peg-in-hole experimental results for an industrial robot verified the effectiveness and robustness of the proposed approach.

A novel control synthesis is proposed for humanoids to demonstrate unique foot-tilting behaviors that are comparable to humans in balance recovery. Our study of model-based behaviors explains the underlying mechanism and the significance of foot tilting well. Our main algorithms are composed of impedance control at the center of mass, virtual stoppers that prevent overtilting of the feet, and postural control for the torso. The proof of concept focuses on the sagittal scenario and the proposed control is effective to produce human-like balancing behaviors characterized by active foot tilting. The successful replication of this behavior on a real humanoid proves the feasibility of deliberately controlled underactuation. The experimental validation was rigorously performed, and the data from the submodules and the entire control were presented and analyzed.

We propose a novel control framework to demonstrate a unique foot tilting maneuver based on ankle torque control for humanoid balance recovery. The framework consists of the variable impedance regulation at the center of mass of the robot based on the ankle torque control, the virtual stoppers to prevent over tilting of the feet, and the body attitude control. The scope of our paper focuses on the sagittal scenario as the first proof of concept on the balance recovery by means of active foot tilting without losing stability. Our study demonstrates the success of the control implementation for the humanoid push recovery and the feasibility of having actively controlled foot tilting. The experimental data are presented and analyzed.

This paper presents a control algorithm for push recovery, which particularly focuses on the hip strategy when an external disturbance is applied on the body of a standing under-actuated biped. By analyzing a simplified dynamic model of a bipedal robot in the stance phase, it is found that horizontal stability can be maintained with a suitably controlled torque applied at the hip. However, errors in the angle or angular velocity of body posture may appear, due to the dynamic coupling of the translational and rotational motions. To solve this problem, different hip strategies are discussed for two cases when (1) external disturbance is applied on the center of mass (CoM) and (2) external torque is acting around the CoM, and a universal hip strategy is derived for most disturbances. Moreover, three torque primitives for the hip, depending on the type of disturbance, are designed to achieve translational and rotational balance recovery simultaneously. Compared with closed-loop control, the advantage of the open-loop methods of torque primitives lies in rapid response and reasonable performance. Finally, simulation studies of the push recovery of a bipedal robot are presented to demonstrate the effectiveness of the proposed methods.

For stable motions of position controlled humanoid robots, ZMP (Zero Moment Point) control is widely adopted, but needs to be integrated with other controllers due to its relatively slow response and the execution error of robots. While CoM (Center of Mass) control is much more directly than ZMP feedback control in the view of rejecting unknown external disturbance. In the meanwhile, we hope the position-based humanoid robot can have the whole body compliance in standing maintenance. So we proposes a CoM compliance controller to achieve stable standing of position controlled humanoid robot under unknown disturbance. The controller uses the concept of force control and integrates virtual model control with admittance control, where an AMPM (Angular Momentum including inverted Pendulum Model)-based virtual model with variable gain is designed to not only generate desired recovery force but also take the GRF (Grand Reaction Force) constraints into account, while an admittance controller is employed to transform the desired force to expected CoM position and body attitude. The experiments were conducted on the humanoid robot ‘Kong’ by exerting external force disturbance and changing the slope of the ground to demonstrate the effectiveness and robustness of our method.