Humanoid robots are drawing significant attention as versatile platforms for complex motor control, human-robot interaction, and general-purpose physical intelligence. However, achieving efficient whole-body control (WBC) in humanoids remains a fundamental challenge due to sophisticated dynamics, underactuation, and diverse task requirements. While learning-based controllers have shown promise for complex tasks, their reliance on labor-intensive and costly retraining for new scenarios limits real-world applicability. To address these limitations, behavior(al) foundation models (BFMs) have emerged as a new paradigm that leverages large-scale pre-training to learn reusable primitive skills and broad behavioral priors, enabling zero-shot or rapid adaptation to a wide range of downstream tasks. In this paper, we present a comprehensive overview of BFMs for humanoid WBC, tracing their development across diverse pre-training pipelines. Furthermore, we discuss real-world applications, current limitations, urgent challenges, and future opportunities, positioning BFMs as a key approach toward scalable and general-purpose humanoid intelligence. Finally, we provide a curated and regularly updated collection of BFM papers and projects to facilitate further research, which is available at https://github.com/yuanmingqi/awesome-bfm-papers.

Unknown dynamic load carrying is one important practical application for quadruped robots. Such a problem is non-trivial, posing three major challenges in quadruped locomotion control. First, how to model or represent the dynamics of the load in a generic manner. Second, how to make the robot capture the dynamics without any external sensing. Third, how to enable the robot to interact with load handling the mutual effect and stabilizing the load. In this work, we propose a general load modeling approach called load characteristics modeling to capture the dynamics of the load. We integrate this proposed modeling technique and leverage recent advances in Reinforcement Learning (RL) based locomotion control to enable the robot to infer the dynamics of load movement and interact with the load indirectly to stabilize it and realize the sim-to-real deployment to verify its effectiveness in real scenarios. We conduct extensive comparative simulation experiments to validate the effectiveness and superiority of our proposed method. Results show that our method outperforms other methods in sudden load resistance, load stabilizing and locomotion with heavy load on rough terrain. Project Page.

Object pose refinement is essential for robust object pose estimation. Previous work has made significant progress to-wards instance-level object pose refinement. Yet, category-level pose refinement is a more challenging problem due to large shape variations within a category and the discrep-ancies between the target object and the shape prior. To address these challenges, we introduce a novel architecture for category-level object pose refinement. Our approach in-tegrates an HS-Iayer and learnable affine transformations, which aims to enhance the extraction and alignment of Geometric information. Additionally, we introduce a cross-cloud transformation mechanism that efficiently merges di-verse data sources. Finally, we push the limits of our model by incorporating the shape prior information for translation and size error prediction. We conducted extensive ex-periments to demonstrate the effectiveness of the proposed framework. Through extensive quantitative experiments, we demonstrate significant improvement over the baseline method by a large margin across all metrics.11Project page: https://lynne-zheng-linfang.github.io/georef.github.io

This paper studies the problem of extracting planar regions in uneven terrains from unordered point cloud measurements. Such a problem is critical in various robotic applications such as robotic perceptive locomotion. While existing approaches have shown promising results in effectively extracting planar regions from the environment, they often suffer from issues such as low computational efficiency or loss of resolution. To address these issues, we propose a multi-resolution planar region extraction strategy in this paper that balances the accuracy in boundaries and computational efficiency. Our method begins with a pointwise classification preprocessing module, which categorizes all sampled points according to their local geometric properties to facilitate multi-resolution segmentation. Subsequently, we arrange the categorized points using an octree, followed by an in-depth analysis of nodes to finish multi-resolution plane segmentation. The efficiency and robustness of the proposed approach are verified via synthetic and real-world experiments, demonstrating our method’s ability to generalize effectively across various uneven terrains while maintaining real-time performance, achieving frame rates exceeding 35 FPS.

This manuscript primarily aims to enhance the performance of whole-body controllers(WBC) for underactuated legged locomotion. We introduce a systematic parameter design mechanism for the floating-base feedback control within the WBC. The proposed approach involves utilizing the linearized model of unactuated dynamics to formulate a Linear Quadratic Regulator(LQR) and solving a Riccati gain while accounting for potential physical constraints through a second-order approximation of the log-barrier function. And then the user-tuned feedback gain for the floating base task is replaced by a new one constructed from the solved Riccati gain. Extensive simulations conducted in MuJoCo with a point bipedal robot, as well as real-world experiments performed on a quadruped robot, demonstrate the effectiveness of the proposed method. In the different bipedal locomotion tasks, compared with the user-tuned method, the proposed approach is at least 12% better and up to 50% better at linear velocity tracking, and at least 7% better and up to 47% better at angular velocity tracking. In the quadruped experiment, linear velocity tracking is improved by at least 3% and angular velocity tracking is improved by at least 23% using the proposed method.

In this paper, we focus on the problem of category-level object pose estimation, which is challenging due to the large intra-category shape variation. 3D graph convolution (3D-GC) based methods have been widely used to extract local geometric features, but they have limitations for complex shaped objects and are sensitive to noise. Moreover, the scale and translation invariant properties of 3D-GC restrict the perception of an object's size and translation information. In this paper, we propose a simple network structure, the HS-layer, which extends 3D-GC to extract hybrid scope latent features from point cloud data for category-level object pose estimation tasks. The proposed HS-layer: 1) is able to perceive local-global geometric structure and global information, 2) is robust to noise, and 3) can encode size and translation information. Our experiments show that the simple replacement of the 3D-GC layer with the proposed HS-layer on the baseline method (GPV-Pose) achieves a significant improvement, with the performance increased by 14.5% on 5°2cm metric and 10.3% on IoU75. Our method outperforms the state-of-the-art methods by a large margin (8.3% on 5°2cm, 6.9% on IoU75) on REAL275 dataset and runs in real-time (50 FPS)11Codeisavailable: https://github.com/Lynne-Zheng-Linfang/HS-Pose.

We study six-dimensional (6D) perceptive peg-in-hole problem for practical connector insertion task in this paper. To enable the manipulator system to handle different types of pegs in complex environment, we develop a perceptive robotic assembly system that utilizes an in-hand RGB-D camera for peg-in-hole with multiple types of pegs. The proposed framework addresses the critical hole detection and pose estimation problem through combining the learning-based detection with model-based pose estimation strategies. By exploiting the structure of the peg-in-hole task, we consider a rectangle-shape based characterization for modeling the candidate socket. Such a characterization allows us to design simple learning-based methods to detect and estimate the 6D pose of the target socket that balances between processing speed and accuracy. To validate our method, we test the performance of the proposed perceptive peg-in-hole solution using a KUKA iiwa7 robotic arm to accomplish the socket insertion task with two types of practical sockets (RJ45/HDMI). Without the need of additional search, our method achieves an acceptable success rate in the connector insertion tasks. The results confirm the reliability of our method and show that our method is suitable for real world application.

In robotic insertion tasks where the uncertainty exceeds the allowable tolerance, a good search strategy is essential for successful insertion and significantly influences efficiency. The commonly used blind search method is time-consuming and does not exploit the rich contact information. In this paper, we propose a novel search strategy that actively utilizes the information contained in the contact configuration and shows high efficiency. In particular, we formulate this problem as a Partially Observable Markov Decision Process (POMDP) with carefully designed primitives based on an in-depth analysis of the contact configuration's static stability. From the formulated POMDP, we can derive a novel search strategy. Thanks to its simplicity, this search strategy can be incorporated into a Finite-State-Machine (FSM) controller. The behaviors of the FSM controller are realized through a low-level Cartesian Impedance Controller. Our method is based purely on the robot's proprioceptive sensing and does not need visual or tactile sensors. To evaluate the effectiveness of our proposed strategy and control framework, we conduct extensive comparison experiments in simulation, where we compare our method with the baseline approach. The results demonstrate that our proposed method achieves a higher success rate with a shorter search time and search trajectory length compared to the baseline method. Additionally, we show that our method is robust to various initial displacement errors.

This article studies capturability and push recovery for quadruped locomotion. Despite the rich literature on capturability analysis and push recovery for legged robots, existing tools have been developed mainly with the requirement of reaching static or quasi-static balance following a push. In practice, this requirement commonly restricts capturability analysis to cases with simple dynamics and fails to encode the time dependence of capturable states for legged locomotion with time-based gaits. To address these issues, we apply switched systems to model quadruped locomotion and extend capturability notions through a novel specification ofdynamic balance. We also provide an explicit model predictive control (EMPC) scheme to compute the dynamic balance and capturable tubes and offer a way of using the capturable tube to synthesize push recovery controllers. Such a generalization allows for a rigorous characterization of disturbance timing on the capturability of quadrupedal locomotion and opens the door of disturbance-timing-aware push recovery control strategies. Extensive simulation and hardware experiments illustrate the necessity of considering dynamic balance for quadrupedal push recovery, reveal how disturbance timing affects capturability, and demonstrate the significant improvement in disturbance rejection with the proposed strategy. Hardware experimental validations on a replica of the Mini Cheetah quadruped further verify that the proposed approach performs statistically better than the state-of-the-art baseline considered.

Emergence of massive dynamic objects will diversify spatial structures when robots navigate in urban environments. Therefore, the online removal of dynamic objects is critical. In this paper, we introduce a novel online removal framework for highly dynamic urban environments. The framework consists of the scan-to-map front-end and the map-to-map back-end modules. Both the front- and back-ends deeply integrate the visibility-based approach and map-based approach. The experiments validate the framework in highly dynamic simulation scenarios and real-world dataset.

This paper studies the distributed multi-robot constrained navigation problem. While the multi-robot collision avoidance has been extensively studied in the literature with safety being the primary focus, the individual robot's destination convergence is not necessarily guaranteed. In particular, robots may get stuck in the local equilibria or periodic orbits of the multi-robot system, some of which are practically known as the deadlock and the livelock behaviors. Inspired by the combination of Control Lyapunov Function (CLF) and Control Barrier Function (CBF) for the nonlinear system's constrained stabilization, the authors present a guaranteed safe feedback control policy with improved convergence performance. The proposed Parametric CLF (PCLF) scheme adaptively determines the appropriate CLF parameterization within the in-stantaneous feasible action space. The algorithm also induces a conditional global asymptotic convergence guarantee for multi-robot system of single-integrator dynamics, and is empirically effective for nonlinear nonholonomic vehicle model. Empiri-cally, the proposed PCLF-CBF framework exhibits superior performance than state-of-the-art methods, including its de-generated counterpart of various CLF-CBF solutions.

Fast and accurate tracking of an object's motion is one of the key functionalities of a robotic system for achieving reliable interaction with the environment. This paper focuses on the instance-level six-dimensional (6D) pose tracking problem with a symmetric and textureless object under occlusion. We propose a Temporally Primed 6D pose tracking framework with Auto-Encoders (TP-AE) to tackle the pose tracking problem. The framework consists of a prediction step and a temporally primed pose estimation step. The prediction step aims to quickly and efficiently generate a guess on the object's real-time pose based on historical information about the target object's motion. Once the prior prediction is obtained, the temporally primed pose estimation step embeds the prior pose into the RGB-D input, and leverages auto-encoders to reconstruct the target object with higher quality under occlusion, thus improving the framework's performance. Extensive experiments show that the proposed 6D pose tracking method can accurately estimate the 6D pose of a symmetric and textureless object under occlusion, and significantly outperforms the state-of-the-art on T-LESS dataset while running in real-time at 26 FPS.

This paper presents a novel framework for point- foot biped running in three-dimensional space. The proposed approach generates center of mass (CoM) reference trajectories based on a differentially flat spring-loaded inverted pendulum (SLIP) model. A foothold planner is used to select touch down location that renders optimal CoM trajectory for upcoming step in real time. Dynamically feasible trajectories of CoM and orientation are subsequently generated by a simplified single rigid body (SRB) model based model predictive control (MPC). A task-space controller is then applied online to compute whole- body joint torques which embeds these target dynamics into the robot. The proposed approach is evaluated on physical simulation of a 12 degree-of-freedom (DoF), 7.95 kg point-foot bipedal robot. The robot achieves stable running at at varying speeds with maximum value of 1.1 m/s. The proposed scheme is shown to be able to reject vertical disturbances of 8 N. s and lateral disturbance of 6.5 N. s applied at the robot base.

This paper studies the problem of constructing polytopic representations for planar regions from depth camera readings. This problem is of great importance for terrain mapping in complicated environment and has great potentials in legged locomotion applications. To address the polytopic planar region characterization problem, we propose a two-stage solution scheme. At the first stage, the planar regions embedded within a sequence of depth images are extracted individually and then merged to establish a terrain map containing only planar regions in a selected frame. To simplify the representations of the planar regions that are applicable to foothold planning for legged robots, we further approximate the extracted planar regions via convex polytopes at the second stage. With the polytopic representation, the proposed approach achieves a great balance between accuracy and simplicity. Experimental validations with RGB-D cameras are conducted to demonstrate the performance of the proposed scheme. The proposed scheme successfully characterizes the planar regions via polytopes with acceptable accuracy. More importantly, the run time of the overall scheme is less than 10ms (i.e., > 100Hz) throughout the tests, which strongly illustrates the advantages of our approach developed in this paper.

In this work, an advanced quadruped robot with abundant kinematic loops and passive joints is introduced. Due to the existence of many closed chains, the robot dynamic model is quite complex, and is derived using the Gauss's principle of least constraint. To explicitly consider the loop-closure constraints, we propose a task-space inverse dynamics based approach to obtain the robot locomotion controller. Besides, to meet the demand of high frequency (≥ 500Hz) in controller, an alternative method is provided. It uses the projected dynamics to find an analytical mapping from the desired contact force to the desired torque of actuators under full consideration of passive joints and loop-closure constraints. The effectiveness and efficiency of the proposed algorithms in this paper have been validated by simulation with a reliable physical engine MuJoCo.

This paper studies push recovery for humanoid robots based on a variable-height inverted pendulum (VHIP) model. We first develop an approach for treating zero-step capturability of the VHIP with a novel methodology based on Hamilton-Jacobi (HJ) reachability analysis. Such an approach uses the sub-zero level set of a value function to encode capturability of the VHIP, where the value function is obtained by numerically solving a HJ variational inequality offline. Based on this analysis, a simple and effective method for adjusting foothold locations is then devised for cases where the VHIP state is not zero-step capturable. In addition, the HJ reachability analysis naturally induces an optimal control law that allows for rapid planning with the VHIP during push recovery online. To enable use of the strategy with a position-controlled humanoid robot, an associated differential inverse kinematics based tracking controller is employed. The effectiveness of the overall framework is demonstrated with the UBTECH Walker robot in the MuJoCo simulator. Simulation validations show a significant improvement in push robustness as compared to the methods based on the classical linear inverted pendulum model.

This paper presents a control strategy for quadruped robots to hop on their rear legs in three-dimensional space. The proposed approach generates nominal center of mass (CoM) trajectories based on a template spring-loaded inverted pendulum (SLIP) model. Tracking this reference remains a challenge due to the underactauted nature of balance with point feet. To address this challenge, a control-Lyapunov function based quadratic programming (CLF-QP) controller is proposed, which modulates nominal ground reaction forces (GRFs) to balance the torso while considering friction limits. The CLF construction is guided by a variational-based linearization (VBL) applied to a reduced-order single-rigid-body (SRB) model, and treats underactuation via solving a Riccati equation to obtain the CLF. A new balance control approach is presented that effectively decouples sagittal plane control (via re-planning) with lateral and rotational control (via the CLF and VBL). The proposed approach shows more robust balancing performance than the conventional CLF-QP approach. Simulations of the Mini Cheetah demonstrate in-place hopping with up to a 0.71m apex height.

This paper studies autonomous stair climbing for quadrupedal robots with perception. Enabling quadrupeds to reliably climb staircases greatly expands their applicability in practical scenarios. For this structured task, we develop a simple yet effective perception and control framework for autonomous quadrupedal stair climbing. By exploiting the structural knowledge about the staircases, the proposed framework first extracts the geometric information about the staircase from measurements of the perception system. Then, the climbing velocity and associated foothold references during stair climbing are generated via simple optimization algorithms based on the geometric information about the staircase. Given these references, we use model predictive control based approach to generate input joint torques for controlling the quadruped to complete the whole stair climbing task. Simulation validations using the full dynamic model of the Unitree’s Aliengo quadruped with the MuJoCo simulator are performed, which demonstrate successful autonomous climbing of various staircases with different geometries. Effectiveness of the proposed strategy is further validated through hardware experiments on the real Aliengo robot with different real-world staircases.

This paper studies a novel encirclement guaranteed cooperative pursuit problem involving N pursuers and a single evader in an unbounded two-dimensional game domain. Throughout the game, the pursuers are required to maintain encirclement of the evader, i.e., the evader should always stay inside the convex hull generated by all the pursuers, in addition to achieving the classical capture condition. To tackle this challenging cooperative pursuit problem, a robust model predictive control (RMPC) based formulation framework is first introduced, which simultaneously accounts for the encirclement and capture requirements under the assumption that the evader’s action is unavailable to all pursuers. Despite the reformulation, the resulting RMPC problem involves a bilinear constraint due to the encirclement requirement. To further handle such a bilinear constraint, a novel encirclement guaranteed partitioning scheme is devised that simplifies the original bilinear RMPC problem to a number of linear tube MPC (TMPC) problems solvable in a decentralized manner. Simulation experiments demonstrate the effectiveness of the proposed solution framework. Furthermore, comparisons with existing approaches show that the explicit consideration of the encirclement condition significantly improves the chance of successful capture of the evader in various scenarios.

This paper studies stabilizer design for position-controlled humanoid robots. Stabilizers are an essential part for position-controlled humanoids, whose primary objective is to adjust the control input sent to the robot to assist the tracking controller to better follow the planned reference trajectory. To achieve this goal, this paper develops a novel force-feedback based whole-body stabilizer that fully exploits the six-dimensional force measurement information and the whole-body dynamics to improve tracking performance. Relying on rigorous analysis of whole-body dynamics of position-controlled humanoids under unknown contact, the developed stabilizer leverages quadratic-programming based technique that allows cooperative consideration of both the center-of-mass tracking and contact force tracking. The effectiveness of the proposed stabilizer is demonstrated on the UBTECH Walker robot in the MuJoCo simulator. Simulation validations show a significant improvement in various scenarios as compared to commonly adopted stabilizers based on the zero-moment-point feedback and the linear inverted pendulum model.

This paper studies dynamic obstacle avoidance strategies of quadruped robots in an unknown and dynamically changing environment. Different from classical navigation problems with static environment, an unknown number of moving obstacles with unknown dynamics are present in the scenario considered in this paper. This unknown and rapidly changing environment prevents us from directly applying existing navigation approaches for quadruped robots. To address the additional challenges introduced by the dynamic obstacles, an improved dynamic window approach (DWA) is proposed in which both evaluation function and constraints are modified. To enable real-time implementation of the proposed algorithm, multi-layer techniques for processing camera point cloud data are applied for online extraction of environmental information. The proposed algorithm is tested both in simulation and on hardware with a quadruped robot equipped with a low-cost RGB-D camera. The results show that the proposed algorithm significantly increases the rate of success for avoiding dynamical obstacles and reduces the time spent for reaching the target.

In this paper, we present the dynamic modeling and model predictive tracking control for a fin-actuated robot with barycentre regulating mechanism in multiple motions. Specifically, a dynamic model for the robot is established firstly. Based on the dynamic model, a model predictive tracking control algorithm is proposed. And simulations of of tracking rectangle trajectory, sine-like trajectory, ascending trajectory, and spiral trajectory are conducted to validate the algorithm. The simulation results demonstrate that the proposed algorithm is able to implement trajectory tracking of the robot with small position error and orientation error. This paper contributes to trajectory tracking for an underwater robot with controllable barycentre in multiple motions, which has been rarely explored.