Gait planning is one of the most critical technologies for quadruped robots. However, far too little attention has been paid to the tight coupling mechanism of gait planning with terrain understanding and energy efficiency. To date, it is still challenging to plan optimal gait strategies that are highly adapted to terrain features with stable and efficient transitions. Accordingly, this paper proposes an adaptive gait control framework for quadruped robots that combines terrain recognition, Cost of Transport (CoT), and the Froude (Fr) number. More specifically, an optimal gait selection strategy for quadruped robots is designed based on different terrain texture features and the CoT characteristics of different gaits. To address the gait transition process induced thereby, an adaptive method for gait parameters based on the Fr number is further proposed, which can make the process more stable. Besides, model predictive control (MPC) and whole-body control (WBC) are employed as the motion controllers for the quadruped robot. Furthermore, simulation and experimental results indicate that the proposed method possesses superior terrain adaptability, energy efficiency, and motion stability during gait transitions, which is beneficial for the quadruped robots to maintain stable motion and reduce energy consumption when performing tasks in changeable terrains.Note to Practitioners—This paper is motivated by the problem of adaptive gait planning for quadruped robots that walks through different terrains. We propose a method that ensures optimal gait selection by robots facing diverse terrains and maintains the stability of gait transition. The proposed control framework, upon testing in a simulated environment, can be directly deployed on real-world robot without further adjustments and allows the robot to traverse various terrains with minimal sim-to-real issues. Hopefully, our proposed method can provide valuable guidance and support for facilitating the enhancement of capabilities in performing prolonged endurance tasks in unstructured environments for quadruped robots.

In the field of multisensor fusion for simultaneous localization and mapping (SLAM), monocular cameras and IMUs are widely used to build simple and effective visual-inertial systems. However, limited research has explored the integration of motor-encoder devices to enhance SLAM performance. By incorporating such devices, it is possible to significantly improve active capability and field of view (FOV) with minimal additional cost and structural complexity. This article proposes a novel visual-inertial-encoder tightly coupled odometry (VIEO) based on a video detection and ranging (ViDAR) device. A ViDAR calibration method is introduced to ensure accurate initialization for VIEO. In addition, a platform motion decoupled active SLAM method based on deep reinforcement learning (DRL) is proposed. Experimental data demonstrate that the proposed ViDAR and the VIEO algorithm significantly increase cross-frame co-visibility relationships compared to its corresponding visual-inertial odometry (VIO) algorithm, improving state estimation accuracy. Additionally, the DRL-based active SLAM algorithm, with the ability to decouple from platform motion, can increase the diversity weight of the feature points and further enhance the VIEO algorithm's performance. The proposed methodology sheds fresh insights into both the updated platform design and decoupled approach of active SLAM systems in complex environments.

Sampling-based planning algorithms such as RRT have been proved to be efficient in solving path planning problems for robotic systems. Various improvements to the RRT algorithm have been presented to improve the performance of the extension and convergence of the random trees, such as Informed RRT*. However, with the growth of spatial dimensions, the time consumption of randomly sampling the entire state space and incrementally rewiring the random trees raises drastically before a feasible solution is found. In this paper, to enhance the convergence performance of optimal solutions, we present Reconstructed Bi-directional Informed RRT* (RBI-RRT*) path planning algorithm. The algorithm acts as RRT-Connect to rapidly find a feasible solution, which helps compress the sampling space as Informed RRT* does. After the random trees are transformed into RRT* structure by the reconstruction process in RBI-RRT*, the algorithm continues to find the near-optimal path. A series of simulations and real-world robot experiments were conducted to evaluate the algorithm against existing planning algorithms. Compared to Informed RRT* Connect, RBI-RRT* reduced the computation time of achieving a specific cost by 22.1% on average in simulations and 11.2% in the real-world robotic arm experiments. The results show that RBI-RRT* is more efficient in high-dimensional planning problems.

To enrich legged robots with fast energy-efficient mobility on even terrain, wheeled-legged robots have emerged as a valued robot form in robotics research. This paper describes the complete development of a new wheeled-legged quadruped robot named Max, ranging from its mechanical design over system architecture to core algorithms implemented for it to realize various motion behaviors. Instead of attaching wheels to the distal ends of legs as in the existing wheeled-legged robot designs, this robot has wheels installed on the knees with a special switching mechanism to convert a leg between the legged and wheeled locomotion modes. This design keeps the wheeled leg lightweight, enabling the robot to preserve the motion agility as a quadruped robot while gaining the energy-efficiency as a four-wheel or even two-wheel mobile robot. An online locomotion generation method is proposed to compute the 6-D body trajectory of the robot in walking on the perceived terrain, while dynamic movements such as leaps and flips are generated by a unified trajectory optimizer, which is also used to generate the transition motions of the robot to transform into the wheeled mode. The diverse mobility of the proposed robot Max is verified with extensive experiments.Note to Practitioners—Empowering robots with all-terrain mobility is a fundamental open problem in developing a new generation of robots. To this end, combinations of wheels and legs have been explored for robots to possess both traversability on uneven terrains and efficiency on even terrains. This paper proposes a new wheeled-legged quadruped robot with focuses on the integrated design of wheeled legs, system architecture, and core algorithms implemented for various legged and wheeled locomotion behaviors. To embed wheels without adding additional motors and keep the light weight of original legs, a special switching mechanism is designed and integrated at the knee joints where wheels are installed. Algorithms for generating quadrupedal walk according to online perceived terrain information as well as other dynamic legged and wheeled motions are discussed and demonstrated. The system architecture for allocating all vision and motion algorithms is also presented. This work is intended to provide a whole picture of developing this new robot including both hardware and software aspects.

In this paper, we present a general learning framework for controlling a quadruped robot that can mimic the behavior of real animals and traverse challenging terrains. Our method consists of two steps: an imitation learning step to learn from motions of real animals, and a terrain adaptation step to enable generalization to unseen terrains. We capture motions from a Labrador on various terrains to facilitate terrain adaptive locomotion. Our experiments demonstrate that our policy can traverse various terrains and produce a natural-looking behavior. We deployed our method on the real quadruped robot$\boldsymbol{Max}$[1] via zero-shot simulation-to-reality transfer, achieving a speed of 1.1 m/s on stairs climbing.

Centroidal dynamics, which describes the overall linear and angular motion of a robot, is often used in locomotion generation and control of legged robots. However, the equation of centroidal dynamics contains nonlinear terms mainly caused by the robot's angular motion and needs to be linearized for deriving a linear model-predictive motion controller. This paper proposes a new linearization of the robot's centroidal dynamics. By expressing the angular motion with exponential coordinates, more linear terms are identified and retained than in the existing methods to reduce the loss from the model linearization. As a consequence, a model-predictive control (MPC) algorithm is derived and shows a good performance in tracking angular motions on a quadruped robot.

When handling occlusion in unsupervised stereo matching, existing methods tend to neglect the supportive role of occlusion and to perform inappropriate disparity smoothing around the occlusion. To address these problems, we propose an occlusion-aware stereo network that contains a specific module to first estimate occlusion as an additional depth cue. In the occlusion inference module, a pixel is classified with a three-category label based on whether an area is occluded by an object on the left, occluded by an object on the right, or unoccluded. After the occluders are detected, we introduce a directed disparity smoothing loss that allows valid disparity estimates to be propagated to fill the occluded region, while ambiguous matches in the occluded region do not affect other regions. Disparity and occlusion are trained alternately in an unsupervised manner with detached backpropagation to enable the directed smoothness. Experiments show that our method achieves 3-pixel threshold error rates of 6.51% and 5.69% on the KITTI 2015 and KITTI 2012 validation sets, state-of-the-art results among unsupervised learning networks at the time of submission.

Completing sparse data captured with commercial depth sensors is a vital and fundamental procedure for many computer vision applications. For execution in real-world scenarios, a good trade-off between accuracy and speed is increasingly in demand for depth completion methods. Most previous methods achieve satisfactory accuracy on standard benchmarks. However, they extensively rely on heavy models to handle diverse structures and require additional run time on multimodal data. In this paper, we present an efficient method of depth completion. We propose a grouped fusion strategy for efficiently extracting depth and guidance features in parallel and fusing them naturally in the feature spaces to achieve high performance. Instead of a monolithic architecture, we employ cascaded hourglass networks, each of which is specialized for certain structures and has a lightweight architecture. Given the sparsity of the depth maps, we downsample the inputs to multiple scales to further accelerate the computation. Our model runs at over 39 FPS on an embedded GPU with high-resolution inputs. Evaluations on the KITTI benchmark demonstrate that the proposed model is an ideal approach for real-world applications.

Despite the notable progress made in action recognition tasks, not much work has been done in action recognition specifically for human-robot interaction. In this paper, we deeply explore the characteristics of the action recognition task in interaction scenes and propose an attention-oriented multi-level network framework to meet the need for real-time interaction. Specifically, a Pre-Attention network is employed to roughly focus on the interactor in the scene at low resolution firstly and then perform fine-grained pose estimation at high resolution. The other compact CNN receives the extracted skeleton sequence as input for action recognition, utilizing attention-like mechanisms to capture local spatial-temporal patterns and global semantic information effectively. To evaluate our approach, we construct a new action dataset specially for the recognition task in interaction scenes. Experimental results on our dataset and high efficiency (112 fps at 640 × 480 RGBD) on the mobile computing platform (Nvidia Jetson AGX Xavier) demonstrate excellent applicability of our method on action recognition in real-time human-robot interaction.

Depth completion, a task to estimate the dense depth map from sparse measurement under the guidance from the high-resolution image, is essential to many computer vision applications. Most previous methods building on fully convolutional networks can not handle diverse patterns in the depth map efficiently and effectively. We propose a multi-scale guided cascade hourglass network to tackle this problem. Structures at different levels are captured by specialized hourglasses in the cascade network with sparse inputs in various sizes. An encoder extracts multi-scale features from color image to provide deep guidance for all the hourglasses. A multi-scale training strategy further activates the effect of cascade stages. With the role of each sub-module divided explicitly, we can implement components with simple architectures. Extensive experiments show that our lightweight model achieves competitive results compared with state-of-the-art in KITTI depth completion benchmark, with low complexity in run-time.

This paper presents an anchor-pair network for crowded human detection, which can overcome and solve the difficulties caused by occlusion in crowded scenes. Specifically, we use a function-aware network structure to extract more distinctive and discriminative features for head and full-body respectively, and then a CNN module is also exploited to fuse the features by learning the correlations between head and full-body to reduce crowd errors. Meanwhile, a novel paired form for anchors, denoted as anchor-pair, is proposed to estimate the head regions and full-body regions simultaneously. Furthermore, a new ingenious Joint-NMS is introduced to perform on the detected head and full-body box pairs, which produces significant performance improvement in heavily occluded scenarios at tiny computational cost. Our anchor-pair network achieves a state-of-the-art result on the CrowdHuman dataset which reduces the MR−2to 55.43%, achieving 11.59% relative improvement over our dataset baseline.

A two-dimensional perching model is proposed first for perching with quadrotors. Then a perching mechanism by means of grasping is designed based on the model. The kinematic specifications of the perching mechanism are optimized to maximize the force transfer ratio so that sufficient grasping force can be generated for reliable perching. A controller of the gripper based on the control strategy from previous development is designed for autonomous perching with a quadrotor. Experiments on the grasping capability and reliability of the mechanism and its effectiveness with the controller for autonomous perching are conducted. Results show that the perching mechanism can generate sufficient grasping force and achieve autonomous perching to a target pole with a quadrotor both effectively and reliably.