Robot skateboarding is a novel and challenging task for legged robots. Accurately modeling the dynamics of dual floating bases and developing effective planning and control methods present significant complexities in accomplishing skateboarding behavior. This paper focuses on enabling the quadrupedal platform CyberDog2 to achieve dynamic balancing and acceleration on a skateboard. An optimization-based control pipeline is developed through careful derivation of the system’s equations of motion, considering both the robot and skateboard dynamics. By accounting for system physical constraints, an advanced offline trajectory optimization method is employed to generate various acceleration trajectories, creating a motion library for the system. An online linear model predictive control with whole body control framework is used to track the generated trajectories and stablize the system in real-time. To validate its effectiveness, we conducted experiments in various scenarios. The quadrupedal robot successfully performed acceleration from a static state to various velocities and demonstrated the ability to balance and steer the skateboard.

We propose a state estimation method that can accurately predict the robot’s privileged states to push the limits of quadruped robots in executing advanced skills such as jumping in the wild. In particular, we present the State Estimation Transformers (SET), an architecture that casts the state estimation problem as conditional sequence modeling. SET outputs the robot states that are hard to obtain directly in the real world, such as the body height and velocities, by leveraging a causally masked Transformer. By conditioning an autoregressive model on the robot’s past states, our SET model can predict these privileged observations accurately even in highly dynamic locomotions. We evaluate our methods on three tasks — running jumping, running backflipping, and running sideslipping — on a low-cost quadruped robot, Cyberdog2. Results show that SET can outperform other methods in estimation accuracy and transferability in the simulation as well as success rates of jumping and triggering a recovery controller in the real world, suggesting the superiority of such a Transformer-based explicit state estimator in highly dynamic locomotion tasks.

Quadrupedal robots are performing increasingly more real-world capabilities, but are primarily limited to locomotion tasks. To expand their task-level abilities of object acquisition, i.e., run-to-catch as frisbee catching for dogs, this paper developed a control pipeline using stereo vision for legged robots which allows for dynamic catching balls while the robot is in motion. To achieve high-frame-rate tracking, we designed a ball that can actively emit homogeneous infrared (IR) light and then located the flying ball based on binocular vision positioning using the onboard RealSense D450 camera with an additional IR bandpass filter. The camera was mounted on top of a 2-DoF head to gain a full view of the target ball. A state estimation module was developed to fuse the vision positioning, camera motor readings, localization result of RealSense T265 equipped on the back, and the legged odometry output altogether. With the use of a ballistic model, we achieved a robust estimation of both the ball and robot positions in an inertial coordinate. Additionally, we developed a close-loop catching strategy and employed trajectory prediction so that tracking and run-to-catch were performed simultaneously, which is critical for such drastically dynamic and precise tasks. The proposed approach was validated through both static testing and dynamic catch experiments conducted on the CyberDog robot with a high success rate.

This paper introduces a general state estimation framework fusing multiple sensor information for hybrid wheeled-legged robots performing mobile manipulation tasks. At the core of the state estimator is a novel unified odometry for hybrid locomotion which can seamlessly maintain tracking and has no need to switch between stepping and rolling modes. To the best of our knowledge, the proposed odometry is the first work in this area. It is calculated based on the robot kinematics and instantaneous contact points of wheels with sensor inputs from IMU, joint encoders, joint torque sensors estimating wheel contact status, as well as RGB-D camera detecting geometric features of the terrain (e.g. elevation and surface normal vector). Subsequently, the odometry output is utilized as the motion model of a 3D Lidar map-based Monte Carlo Localization module for drift-free state estimation. As part of the framework, visual localization is integrated to provide high precision guidance for the robot movement relative to an object of interest. The proposed approach was verified thoroughly by two experiments conducted on the Pholus robot with OptiTrack measurements as ground truth.

This paper presents a reactive legged locomotion generation scheme that enables our quadruped robot CEN-TAURO to adapt to varying payloads while walking. The center-of-mass (CoM) trajectories are generated in real time in a model predictive control (MPC) fashion, trading off large stability margins against evenly stretched legs. Vertex-based zero-moment-point (ZMP) constraints are imposed to ensure quasi-static walking stability. A Kalman filter is then implemented to estimate the CoM states and the impact of external payloads which can vary online and affect/disturb the locomotion differently. The CoM estimation is used to update the MPC motion planner at every replanning instant so that the robot can react to unknown and time-varying payloads on the fly.We validate the proposed scheme through experimental trials where the robot walks on flat ground or steps on different surface levels while carrying heavy payloads. It is shown that the proposed reactive locomotion strategy enables the robot to carry 20 kg payloads, which is close to the maximum capacity of the robot arms.

This paper proposes an improved supervised autonomy framework for remote teleoperation of a quadrupedal bimanual mobile manipulator in an unknown environment, with the usage of advanced perception technology and allowing the operator to easily assist the robot with decision making for executing tasks on the fly. First, the perception system uses lightweight deep neural network-based Single Shot Detector (SSD) MobileNet on RGB images to detect objects and highlight them to the human operator via an intuitive interactive visualization interface. After object and action selections are made by the operator, segmentation of object point cloud and 3D surfaces based on random sample consensus is performed, followed by object pose localization by using keypoint extraction. Based on the localized object, mobile manipulation motion to perform the operator-selected action is planned and executed with the help of a state estimator for the hybrid wheel-legged robot. Thanks to the autonomy of the robot in perception and manipulation, the complexity of teleoperating the robot is reduced to specifying the essential task objectives. Experimental results on the real robot, with full system integration, for 2 task scenarios, namely passage clearing and object retrieval, demonstrate a high average success rate of 92.2% over a total of 90 trials.

This paper addressed a challenging problem of wheeled-legged robots with high degrees of freedom exploring in unknown rough environments. The proposed method works as a pipeline to achieve prioritized exploration comprising three primary modules: traversability analysis, frontier-based exploration and hybrid locomotion planning. Traversability analysis provides robots an evaluation about surrounding terrain according to various criteria ( roughness, slope etc.) and other semantic information (small step, stair, bridge etc.), while novel gravity point frontier-based exploration algorithm can effectively decide which direction to go even in unknown environments based on robots' current pose and desired one. Given all these information, hybrid locomotion planner will generate a path with motion mode (driving or walking) encoded by optimizing among different objectives and constraints. Lastly, our approach was well verified in both simulation and experiment on a wheeled quadrupedal robot Pholus.

To generate dynamic motions such as hopping and running on legged robots, model-based approaches are usually used to embed the well studied spring-loaded inverted pendulum (SLIP) model into the whole-body robot. In producing controlled SLIP-like behaviors, existing methods either suffer from online incompatibility or resort to classical interpolations based on lookup tables. Alternatively, this paper presents the application of a data-driven approach which obviates the need for solving the inverse of the running return map online. Specifically, a deep neural network is trained offline with a large amount of simulation data based on the SLIP model to learn its dynamics. The trained network is applied online to generate reference foot placements for the humanoid robot. The references are then mapped to the whole-body model through a QP-based inverse dynamics controller. Simulation experiments on the WALK-MAN robot are conducted to evaluate the effectiveness of the proposed approach in generating bio-inspired and robust running motions.

This paper offers a novel insight of using nonlinear models for the control to produce more robust and natural walking gaits for humanoid robots. The sagittal and lateral gait control needs to be treated differently, hence, we proposed two types of suitable nonlinear models, which allow forward simulations to look ahead, and thus, predict accurately the future trajectory/state at the end of the current step. Subsequently, by performing multiple forward simulations in a similar manner for the next step and using the gradient descent method, an appropriate foot placement can be found to achieve precise walking speed. By doing this two-step lookahead, all trajectories of the support and the swing leg can be generated. Our proposed controller can plan trajectories at the beginning of each step or actively re-plan according to task state errors. It is validated effectively in simulations performed in both ADAMS and Open Dynamic Engine. The robot can successfully traverse up/down a stair and recover from pushes with more natural looking gaits compared to the conventional bent-knee style. The reasonable computational time also indicates the feasibility of real-time implementation on real robots.

This paper is motivated by the questions: What would happen if a humanoid robot is put on a Segway? Is it possible for the humanoid robot to use this transportation device that is specifically designed for human? Simulation involving a two-wheeled mobile platform (TWMP) and our humanoid robot COMAN (COmpliant HuMANoid Platform) shows that it is indeed feasible without any hardware modification. Regarding the implementation, the full dynamics of the humanoid robot is considered and quadratic optimization is employed to generate whole-body joint torques to realise two types of tasks according to the interaction type between the TWMP and the humanoid robot. The TWMP is considered as unknown disturbance and the humanoid robot has to keep balancing on it in the first type of task. On the contrary, the active movement of the humanoid robot is utilised as an interface to intuitively drive the TWMP in the second type of task. For both tasks, tracking the position of center of mass (CoM) and regulating the angular momentum around it are considered as primary objectives, stabilizing the posture of certain part of its body is optional. In addition, both tasks are repeated on uneven terrain to demonstrate the robustness of the control method.

This paper aims at developing a unified and adaptive foot placement control for legged robots. The locomotion control of legged robots can be classified into three parts as body height control, body attitude control, and forward velocity control. In our study, the body attitude is controlled at stance phase by the hip actuator, and the height is controlled by the motion of the stance leg. In this case, the foot placement has a nearly linear correlation with forward velocity. Hereby, a generic foot placement controller is developed to control the forward velocity based on the online linear regression analysis of their coupled correlation. Our proposed algorithm is capable of adjusting the control parameters automatically, and is featured by good adaptability and higher control accuracy that outperforms the empirical tuning. The very same controller is able to produce stable hopping with accurate forward velocity tracking even with unknown mass offset, as well as stable bipedal running and walking with accurate velocity tracking.