Legged robots possess inherent advantages in traversing complex 3D terrains. However, previous work on lowcost quadruped robots with egocentric vision systems has been limited by a narrow front-facing view and exteroceptive noise, restricting omnidirectional mobility in such environments. While building a voxel map through a hierarchical structure can refine exteroception processing, it introduces significant computational overhead, noise, and delays. In this paper, we present MOVE, a one-stage end-to-end learning framework capable of multi-skill omnidirectional legged locomotion with limited view in 3D environments, just like what a real animal can do. When movement aligns with the robot's line of sight, exteroceptive perception enhances locomotion, enabling extreme climbing and leaping. When vision is obstructed or the direction of movement lies outside the robot's field of view, the robot relies on proprioception for tasks like crawling and climbing stairs. We integrate all these skills into a single neural network by introducing a pseudo-siamese network structure combining supervised and contrastive learning which helps the robot infer its surroundings beyond its field of view. Experiments in both simulations and real-world scenarios demonstrate the robustness of our method, broadening the operational environments for robotics with egocentric vision.

Quadruped robots are able to exhibit a range of gaits, each with its own traversability and energy efficiency characteristics. By actively coordinating between gaits in different scenarios, energy-efficient and adaptive locomotion can be achieved. This study investigates the performances of learned energy-efficient policies for quadrupedal gaits under different commands. We propose a training–synthesizing framework that integrates learned gait-conditioned locomotion policies into an efficient multiskill locomotion policy. The resulting control policy achieves low-cost smooth switching and controllable gaits. Our results of the learned multiskill policy demonstrate seamless gait transitions while maintaining energy optimality across all commands.

We focus on the task of unknown object rearrangement, where a robot is supposed to reconfigure the objects into a desired goal configuration specified by an RGB-D image. Recent works explore unknown object rearrangement systems by incorporating learning-based perception modules. However, they are sensitive to perception error, and pay less attention to task-level performance. In this article, we aim to develop an effective system for unknown object rearrangement amidst perception noise. We theoretically reveal that the noisy perception impacts grasp and place in a decoupled way, and show such a decoupled structure is valuable to improve task optimality. We propose grasp, see, and place (GSP), a dual-loop system with the decoupled structure as prior. For the inner loop, we learn a see policy for self-confident in-hand object matching. For the outer loop, we learn a grasp policy aware of object matching and grasp capability guided by task-level rewards. We leverage the foundation model CLIP for object matching, policy learning, and self-termination. A series of experiments indicate that GSP can conduct unknown object rearrangement with higher completion rates and fewer steps.

Global trajectory planning is crucial for long-range formation navigation tasks of multi-robot systems in efficiency improvement and energy saving, whose main challenges are the joint space constraints of the whole team and the long-range deployment. To overcome the above difficulties, we reformulate the original problem into an affine formation planning problem in parameter space. Further, we propose a front-end & back-end framework for global trajectory planning of Multi-Robot Systems (MRS) with affinely deformable formation. For the front-end, an RL-steering affine formation RRT* method is designed to search a global formation-level trajectory in affine parameter space, combining the efficient BVP-solving capability of RL and the global guidance and generalizing ability of RRT*. For the back-end, we propose a formationlevel affine parameter trajectory optimization method to refine the front-end trajectory, and further transform it into peragent trajectories for execution. Extensive benchmarks and ablation experiments in simulation show the effectiveness of our framework for the global trajectory generation of a multiUAV system with affinely deformable formation. The appendix can be seen here3.

The realm of multi-robot autonomous exploration tasks underscores the critical role of communication in coordinating group activities. This paper introduces an innovative decentralized multi-robot exploration algorithm, meticulously crafted to ensure unbroken communication within robotic groups, a crucial element for effective coordination. The motivation for our work is two-fold: Firstly, seamless communication is vital for coordinating multi-robot autonomous exploration tasks. Secondly, in applications such as disaster rescue operations or military maneuvers, there are numerous scenarios where spatial congregation of multiple robots is imperative for joint task accomplishment. Our approach addresses these challenges through a stringent communication constraint, ensuring that each robot remains in constant communicative contact with the rest of the group. This is realized by employing a decentralized policy that integrates Graph Neural Network (GNN) layers with self-attention mechanism. Such policy network design allows adaptation to different numbers of robots and varied environments. After an initial imitation learning phase, the policy is refined through learning from experiences generated via a tree-search-based lookahead technique. Our experimental analysis validates that the algorithm not only maintains consistent communication links among all group members but also improve the exploration efficiency under the communication constraints. These results highlight the potential of our method in enhancing the effectiveness of robotic group explorations while ensuring robust communication connection.

Accurate state estimation plays a critical role in ensuring the robust control of humanoid robots, particularly in the context of learning-based control policies for legged robots. However, there is a notable gap in analytical research concerning estimations. Therefore, we endeavor to further understand how various types of estimations influence the decision-making processes of policies. In this paper, we provide quantitative insight into the effectiveness of learned state estimations, employing saliency analysis to identify key estimation variables and optimize their combination for humanoid locomotion tasks. Evaluations assessing tracking precision and robustness are conducted on comparative groups of policies with varying estimation combinations in both simulated and real-world environments. Results validated that the proposed policy is capable of crossing the sim-to-real gap and demonstrating superior performance relative to alternative policy configurations.

A novel in-contact three-dimensional (3D) measuring device, called MultiCal, is proposed as a convenient, low-cost (less than US$5000), and robust facility for onsite kinematic calibration and online measurement of robot manipulator accuracy. The device has µm-level accuracy and can be easily embedded in robot cells. During the calibration procedure, the robot manipulator first moves automatically to multiple end-effector orientations with its tool center point (TCP) constrained on a fixed point by a 3D displacement measuring device (single point constraint), and the corresponding joint angles are recorded. Then, the measuring device is precisely mounted at different positions using a well-designed fixture, and the above measurement process is repeated to implement a multi-point constraint. The relative mounting positions are accurately measured and used as prior information to improve calibration accuracy and robustness. The results of theoretical analysis indicate that MultiCal reduces calibration accuracy by 10% to 20% in contrast to traditional non-contact 3D or six-dimensional (6D) measuring devices (such as laser trackers) when subject to the same level of artificial measurement noise. The results of a calibration experiment conducted on a Staubli TX90 robot show that MultiCal has only 7% to 14% lower calibration accuracy compared to a measuring arm with a laser scanner, and 21% to 30% lower time efficiency compared to a 6D binocular vision measuring system, yielding maximum and mean absolute position errors of 0.831 mm and 0.339 mm, respectively.

Global localization plays a critical role in many robot applications. LiDAR-based global localization draws the community's focus with its robustness against illumination and seasonal changes. To further improve the localization under large viewpoint differences, we propose RING++ that has roto-translation-invariant representation for place recognition and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multichannel features. To the best of our knowledge, this is the first learning-free framework to address all the subtasks of global localization in the sparse scan map. Validations on real-world datasets show that our approach demonstrates better performance than state-of-the-art learning-free methods and competitive performance with learning-based methods. Finally, we integrate RING++ into a multirobot/session simultaneous localization and mapping system, performing its effectiveness in collaborative applications.

Current RGB-based 6D object pose estimation methods have achieved noticeable performance on datasets and real world applications. However, predicting 6D pose from single 2D image features is susceptible to disturbance from changing of environment and textureless or resemblant object surfaces. Hence, RGB-based methods generally achieve less competitive results than RGBD-based methods, which deploy both image features and 3D structure features. To narrow down this performance gap, this paper proposes a framework for 6D object pose estimation that learns implicit 3D information from 2 RGB images. Combining the learned 3D information and 2D image features, we establish more stable correspondence between the scene and the object models. To seek for the methods best utilizing 3D information from RGB inputs, we conduct an investigation on three different approaches, including Early-Fusion, Mid-Fusion, and Late-Fusion. We ascertain the Mid-Fusion approach is the best approach to restore the most precise 3D keypoints useful for object pose estimation. The experiments show that our method outperforms state-of-the-art RGB-based methods, and achieves comparable results with RGBD-based methods.

Legged robots, specifically quadruped robots, have good locomotion performance in complex and rugged terrain and are becoming widely used in field exploration and rescue missions. To achieve full autonomy in such scenarios, robots need not only accurate localization but also an accurate understanding of the surrounding terrain, which will be used for robots path planning and foothold planning. However, due to the kinetic characteristic and limitation of size, quadruped robots have the disadvantages of high-frequency jitter and limited field of sensors, which lead to some challenges in environmental perception. In this paper, we propose a vision-assisted rugged terrain environment reconstruction and localization method for quadruped robots. We use a depth camera to assist in the generation of high-precision localization and terrain reconstruction results, which can help achieve the autonomous mobility of quadruped robots in this environment. We test our method on a quadruped robot platform. Our experimental results show less error and lower drift in different stairs terrain types than the commonly used lidar-based localization method.

The longitudinal control of the platoon of connected and automated vehicles (CAVs) has gained extensive attention in recent transportation research. A majority of existing results are based on linearized third-order vehicular models, under the premise that a complete priori knowledge of vehicle dynamics is available. This article focuses on a general class of third-order nonlinear CAVs with parametric uncertainty and unknown external disturbance which cannot be linearized. A vehicle-to-vehicle (V2V) communication-based cooperative adaptive backstepping control scheme is proposed, in which unknown parameters and disturbance bounds are estimated on-line. Since the transfer function of linear systems cannot be applied to nonlinear systems to guarantee string stability, asymmetric time-varying constraints are employed to prevent the spacing errors from growing up. A realistic example is considered to verify the feasibility of the control algorithm.

This article focuses on the investigation of finite-time dissipative state estimation for Markov jump neural networks. First, in view of the subsistent phenomenon that the state estimator cannot capture the system modes synchronously, the hidden Markov model with partly unknown probabilities is introduced in this article to describe such asynchronization constraint. For the upper limit of network bandwidth and computing resources, a novel dynamic event-triggered transmission mechanism, whose threshold parameter is constructed as an adjustable diagonal matrix, is set between the estimator and the original system to avoid data collision and save energy. Then, with the assistance of Lyapunov techniques, an event-based asynchronous state estimator is designed to ensure that the resulting system is finite-time bounded with a prescribed dissipation performance index. Ultimately, the effectiveness of the proposed estimator design approach combining with a dynamic event-triggered transmission mechanism is demonstrated by a numerical example.

This article is devoted to the investigation of reduced-order dissipative filtering for Takagi–Sugeno (T–S) fuzzy Markov jump systems with the event-triggered mechanism. For the proposed event-triggered mechanism, its threshold parameter is constructed as a special diagonal matrix which can improve system performance by flexibly adjusting the matrix elements. Due to the impact of the sampling behaviors and the environmental disturbance, the asynchronization between the filter and the estimated system is considered in this article, which can be characterized by the hidden Markov model. Through handling the linear matrix inequalities (LMIs) with some slack matrices, event-triggered fuzzy filters are designed to guarantee the resulting system is stochastically stable and strictly dissipative. The proposed filter parameters are obtained by solving LMIs. Ultimately, both the effectiveness and advantages of the proposed reduced-order filter with the event-triggered mechanism are verified by a practical example.

Opening a lock without vision sensors remains a challenge for robots. Inspired by the ability of a human to open a lock through touch and intuition, a peg-in-hole assembly method for recognizing the relative position and inclination angle of a hole is proposed. We use supervised learning to generate a contact-state model to judge the relative contact state and introduce force control strategies that ensure stable and safe interaction with the environment. Adaptive impedance control is adopted to ensure the stability of the alignment and insertion process. The proposed method is not restricted by the object shape. The system can learn an effective classification model with a small volume of force and torque data and predict the relative contact state of a peg and hole. The proposed method is verified in an experiment in which a bicycle lock is opened at different inclination angles. The proposed method has potential application in the field of industrial assembly.

In this paper, the problem of resilient observer-based robust control is considered for discrete-time Markov jump systems subject to asynchronous models and extended dissipativity. The model uncertainty is in the interval type, which has an ability to describe parameter fluctuating phenomenon more accurately than using the norm-bounded uncertainty. A hidden Markov chain is employed to depict the mismatch between the original system and the observer-based controller. Conditions are provided to ensure the stability of the resulting closed-loop system with a desired dissipation performance regardless of the uncertainties. An example is presented to illustrate the effectiveness and potential of the proposed new design techniques.

In this paper, an approach for automatic peg-in-hole assembly is proposed. The task is divided into two main steps: searching phase and inserting phase. First, a multilayer perceptron network is designed to address the hole search problem and a hybrid force position controller is introduced to ensure a safe and stable interaction with the external environment. Then, for the inserting phase, a variable impedance controller is adopted based on the fuzzy Q-learning algorithm to yield compliant behavior from the robot during the hole insertion process. This approach is a practical and general approach to solve complex peg-in-hole assembly problems by taking advantage of both learning-based algorithms and force control strategies, which can greatly improve the efficiency and safety of the industrial manufacturing process without identifying the unknown contact model and tuning tedious parameters. Finally, the peg-in-hole experimental results for an industrial robot verified the effectiveness and robustness of the proposed approach.

This article considers intelligent fuel cell/battery hybrid vehicles (FCHVs) that can make autonomous decisions at both the vehicle and powertrain levels. Since the vehicle and powertrain level dynamics are inherently integrated, we propose an integrated motion and powertrain model predictive control approach for intelligent FCHVs by jointly optimizing the vehicle acceleration and fuel cell current. The control goals are to achieve vehicle mobility, minimal hydrogen consumption, and battery state-of-charge maintenance within system constraints. The main challenge in an integrated control is that the electric motor can operate in both propelling and generating modes coupling with vehicle and powertrain states. This hybrid operation is handled by the mixed logical dynamical modeling resulting in a mixed integer nonlinear control problem. To relieve the possible heavy computational burden, two simplification approaches are proposed: hierarchical control and successive linearizations. Two standard driving cycles and a typical vehicle cruising scenario are employed to test the effectiveness of the proposed modeling and control algorithms. Simulation results show that the hierarchical linear control is more suitable for real-time applications with comparable control performance with that of the integrated control. However, additional constraints must be carefully designed to compensate for the ignored coupling dynamics and constraints.

This paper presents a control algorithm for push recovery, which particularly focuses on the hip strategy when an external disturbance is applied on the body of a standing under-actuated biped. By analyzing a simplified dynamic model of a bipedal robot in the stance phase, it is found that horizontal stability can be maintained with a suitably controlled torque applied at the hip. However, errors in the angle or angular velocity of body posture may appear, due to the dynamic coupling of the translational and rotational motions. To solve this problem, different hip strategies are discussed for two cases when (1) external disturbance is applied on the center of mass (CoM) and (2) external torque is acting around the CoM, and a universal hip strategy is derived for most disturbances. Moreover, three torque primitives for the hip, depending on the type of disturbance, are designed to achieve translational and rotational balance recovery simultaneously. Compared with closed-loop control, the advantage of the open-loop methods of torque primitives lies in rapid response and reasonable performance. Finally, simulation studies of the push recovery of a bipedal robot are presented to demonstrate the effectiveness of the proposed methods.

The best measure quantifying FWL (finite word length) stability is the one that bases on the largest stable perturbation hypercube. But the computing of this FWL stability measure has not been solved. For second order digital systems, this paper develops an analytic computing method. Through solving 12 linear equations and 12 quadratic equations, the measure value can be obtained exactly.

The author present a study of the finite word length (FWL) implementation for digital controller structures using /spl delta/-operator. The closed-loop roundoff noise gain using /spl delta/-operator is defined and analyzed based on stochastic control theory. The optimal controller realizations are obtained via minimizing the roundoff noise gain. A numerical example is presented to illustrate the design procedure and the effectiveness of the proposed strategy.