Developing general robotic systems capable of manipulating in unstructured environments is a significant challenge, particularly as the tasks involved are typically long-horizon and rich-contact, requiring efficient skill transfer across different task scenarios. To address these challenges, we propose knowledge graph-based skill library construction method. This method hierarchically organizes manipulation knowledge using "task graph" and "scene graph" to represent task-specific and scene-specific information, respectively. Additionally, we introduce "state graph" to facilitate the interaction between high-level task planning and low-level scene information. Building upon this foundation, we further propose a novel hierarchical skill transfer framework based on the skill library and tactile representation, which integrates high-level reasoning for skill transfer and low-level precision for execution. At the task level, we utilize large language models (LLMs) and combine contextual learning with a four-stage chain-of-thought prompting paradigm to achieve subtask sequence transfer. At the motion level, we develop an adaptive trajectory transfer method based on the skill library and the heuristic path planning algorithm. At the physical level, we propose an adaptive contour extraction and posture perception method based on tactile representation. This method dynamically acquires high-precision contour and posture information from visual-tactile images, adjusting parameters such as contact position and posture to ensure the effectiveness of transferred skills in new environments. Experiments demonstrate the skill transfer and adaptability capabilities of the proposed methods across different task scenarios. Project website: https://github.com/MingchaoQi/skill_transfer

A novel efficient reinforcement learning paradigm combining human knowledge, model-based and model-free methods is presented for optimal robot manipulation control during complex multi-phase robot manipulation tasks, e.g., the peg-in-hole tasks with tight fit and nut-and-bolt assembly. Firstly, human demonstration is conducted to collect the data during successful robot manipulation, and manipulation phase estimation method integrating with human knowledge is presented to obtain the higher-level planning of the multi-phase robot manipulation tasks. Typical robot manipulation tasks can usually be decomposed into three types of phases, namely free motion, discontinuous contact, and continuous contact. For phase with free motion, the motion planning method is utilized for generating smooth trajectory. For phase with discontinuous contact in the axes of interest during the pre-manipulation process, the rule-based model-free method, namely the Policy Gradients with Human-Guided Parameter-based Exploration (PGHGPE) method is utilized. For the manipulation phase with continuous contacts, the model-based method is utilized because of its higher sample efficiency. Finally, the simulation and experimental studies verify the effectiveness of the presented algorithm. Note to Practitioners—The important premise for the future robot assistants is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable method for solving this kind of problems. However, HRL faces the challenge of low computational efficiency. To this end, efficient manipulation skill learning for complex manipulation tasks via human knowledge, model-based and model-free reinforcement learning methods are presented, which improves the efficiency of the skill learning process to a practical level.

Robotic manipulation holds the potential to replace humans in the execution of tedious or dangerous tasks. However, control-based approaches are not suitable due to the difficulty of formally describing open-world manipulation in reality, and the inefficiency of existing learning methods. Therefore, applying manipulation in a wide range of scenarios presents significant challenges. In this study, we propose a novel framework for skill learning in robotic manipulation called Tactile Active Inference Reinforcement Learning (TactileAIRL), aimed at achieving efficient learning. To enhance the performance of reinforcement learning (RL), we introduce active inference, which integrates model-based techniques and intrinsic curiosity into the RL process. This integration improves the algorithm’s training efficiency and adaptability to sparse rewards. Additionally, we have designed universal tactile static and dynamic features based on vision-based tactile sensors, making our framework scalable to many manipulation tasks learning involving tactile feedback. Simulation results demonstrate that our method achieves significantly high training efficiency in objects pushing tasks. It enables agents to excel in both dense and sparse reward tasks with just few interaction episodes, surpassing the SAC baseline. Furthermore, we conduct physical experiments on a gripper screwing task using our method, which showcases the algorithm’s rapid learning capability and its potential for practical applications.

This paper proposes a novel hierarchical reinforcement learning (HRL) framework of complex manipulation tasks which integrates the human prior knowledge. The framework involves the following steps:$ \textbf{1)} $The manipulation process is divided into several stages based on human prior knowledge.$ \textbf{2)} $Transition conditions between stages are determined in the form of “if-then” rules.$ \textbf{3)} $The key features of each stage are selected, and the corresponding control policies are designed via human knowledge.$ \textbf{4)} $The policy gradient with parameter-based exploration (PGPE) method is employed to optimize the policy parameters because it does not require the policy to be derivable for the parameters. To increase the convergence speed of this framework, importance sampling and adaptive adjustment of exploration variance are employed to improve it.$ \textbf{5)} $On this basis, to facilitate the transfer of simulation results to practical experiments, half sim-to-real method is presented, which fully utilizes the simulation results, and the differences between simulation and experimental environments are considered. Simulation and experimental studies show that our framework can deal with the peg-hole-insertion task with a high quality in less than 1600 episodes and can safely adapt the skill into the practical scene with little iterations, which verify the efficiency of the presented method.Note to Practitioners—Intelligent robots will become the right assistants of human beings in the future, especially in various areas of complex manipulation occasions. The important premise is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable and efficient method for solving this kind of problems. This paper proposes a novel HRL framework which can better integrate the human prior knowledge. In addition, improved PGPE method is proposed to obtain the optimized policy parameters more quickly. More importantly, a novel half sim-to-real transfer method is presented to better integrate the simulation and experiment results. This provides a common paradigm, which leverages the simulation results to reduce the interaction between robot and practical environment and then utilizes the experiment results to further optimize the policy parameters.

In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via the Gaussian process regression (GPR) method as the external dynamics. Integrating the learned external dynamics as well as the stochastic uncertainties, the complete interaction system dynamics are obtained. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LEDs) method is presented to obtain the optimal manipulation control parameters, namely, the feedforward force, the reference trajectory, as well as the impedance parameters, subject to time-varying environment dynamics. The comparative simulation studies verify the advantages of the presented method, and the experimental studies of the peg-hole-insertion task prove that this method can deal with complex manipulation tasks.

This article considers the implementation of an observer-based logarithmic control scheme for physical human-robot interaction, which is a typical Lagrangian system. The novelty lies in using a switching term in the logarithmic sliding mode observer to describe the operator’s behavior without any sensors, and applying adaptive parameters in the logarithmic sliding mode controller (SMC) to practically stabilize reaching the sliding surface using chattering-free nonsingular reaching law in finite time. A reset mechanism is synthesized into the control system to enhance the transient response. The motion on the sliding surface is analyzed from the perspective of practical finite-time stability, from which both the convergence regions of the tracking and estimate errors can be determined. The numerical and experimental results verify the effectiveness and advantage of the proposed reset logarithmic SMC and observer for human-robot interaction compared to the existing linear SMC and terminal SMC. With regards to settling time and rising time, the superiority of transient performance in experimental results is coincident with the stability analysis.

This article investigates the stabilization of the bilateral teleoperation system with inherent strong nonlinearities and model parametric uncertainties, and a fractional-order non-singular terminal sliding mode control (FONTSMC) strategy for high-precision master-slave tracking is generated by combining fractional calculus and sliding mode control. The stability of the closed-loop system and finite-time convergence are analyzed using the Lyapunov stability theory. For simulation verification, the presented control scheme is applied to a pair of 3-DoF Phantom Omni haptic manipulators. The results reveal that the performance of non-singular terminal sliding mode control with fractional calculus is significantly better than that of the integer-order non-singular terminal sliding mode control in terms of trajectory tracking accuracy and response time.

In this paper, the control performance of the teleoperation system subjected to dynamics uncertainty and external disturbance is investigated. To improve control performance, an adaptive neural learning prescribed-time controller was developed, which ensures that the system’s output tracks the desired trajectory with a predetermined accuracy within a user-defined time. Unlike other general finite-time or fixed-time controllers, the predetermined convergence time can be exactly obtained rather than approximated. Moreover, the proposed control scheme can solve the issue with and without constraints uniformly. With the aid of the Lyapunov method, the stability of the system is analyzed. Finally, the effectiveness of the proposed method is further verified by numerical simulations.

Nowadays the robot manipulation skills are usually learned by human demonstration via trajectory-level learning, which somewhat lacks robustness and generalization. In this paper, we propose a novel contact state level learning method for robot manipulation skill acquisition via human demonstration. The robot-environment contact states are described via environment dynamics modelling and geometric constraints modelling for flexible contact and rigid contact cases, respectively. During human demonstration process, the robot-environment interaction force, the robot position, and velocity data are collected. After that, the environment dynamics and geometric constraints modelling methods are presented to determine the contact state changes during the robot manipulation process. Then the robot manipulator learns the contact state information rather than specific manipulation trajectory. On this basis, the manipulation control law using active exploration method is presented to control the robot during the button pressing process and peg-hole-insertion process, respectively. Finally, the performance of the presented methodology has been verified via experimental studies. Note to Practitioners—Intelligent robots will become the right assistants of human beings in the future, especially in various areas of manipulation occasions. The important premise of realizing this vision is that the robots should have certain ability of manipulation skill learning. A lot of research has been carried out in this field, many of which are focusing on trajectory level manipulation skill learning and reproduction. Other than the trajectory level learning, human beings can learn many other higher levels of manipulation skills, such as the contact state level and semantic level learning, which makes the learning results more robust and general. In this paper, the contact state estimation and learning method via environment dynamics and geometric constraints modelling is presented to learn the robot manipulation skill based on the contact state transition conditions. In this way, the robot needs less data in the skill learning process, and the trajectory level learning is avoided. After learning the contact state level manipulation skill, the lower trajectory level command is autonomously generated. Experiments on button pressing and peg-hole-insertion tasks by KUKA iiwa robot have obtained very good results. Other than the button pressing and peg-hole-insertion tasks, the presented methodology can be applied to many other manipulation tasks, as long as there are contact state changes in the manipulation process. The work of this paper lays a foundation for the robot learning of higher-level manipulation skills.

In a space telerobot system (STS), effectiveness of the control method in eliminating the time delay’s influences is advisable to be verified under the real circumstance. However, it is difficult and costly for many scholars to obtain confidential information that would allow them to establish an STS. It may be feasible, using some existing results, to model the time delay as close to reality as possible, and to then program a simulation system to generate the simulated time delay, thus verifying validity. In this article, time-delay modeling and simulation problems for relay communication-based STS are first studied. The time delay in relay communication-based STS consists of both processing and communication time delays; the latter is divided into ground and ground-space parts. By extending the available results, processing and ground communication time delays are modeled with the probability distribution function modeling approach. An optimal communication link identification and minimum time-delay realization (OCLIMTDR) method is proposed to model the ground-space communication time delay. In this method, the novel point–vector–sphere (PVS) algorithm serves to judge link connectivity. The PVS algorithm is based on geometric theory, which gives the OCLIMTDR method good extensibility and renders it suitable for any relay communication network in theory. All three parts of the time-delay models are integrated to form the loop time-delay model of the STS. Subsequently, a time-delay simulation system is created by programming the loop time-delay model. Finally, the correctness of the simulation system is further authenticated based on simulations and some prior knowledge.

In this paper, a novel fuzzy-observer-based hybrid force/position control method is investigated for a bimanual teleoperation system in the presence of dynamics uncertainties, random network-induced time delays, and multiple sampling rates of remote control signals and local measured data. The system structure consists of two pairs of position observers and contact force/torque estimators. The position observers are designed based on Takagi-Sugeno fuzzy inference rules to estimate the delayed remote state with low sampling rates. The force/torque estimators are designed for estimating the coupled item of uncertain dynamics and contact forces without acceleration information. By adding a compensatory item based on an auxiliary model, the force estimation and motion-tracking errors caused by varying dynamics uncertainties decrease, which is certified by two comparative force estimation techniques. The stability condition for the closed-loop system is also proved by the linear matrix inequality method based on the Lyapunov function. Finally, two simulations verify the effectiveness of the proposed method. The results indicate that the proposed method enables a better motion synchronization effect in soft-handling environment.

Facing the new challenges of the spacecraft developing, the concept of cellular space robot (CSR) for both space-craft system construction and on-orbit operation is presented in this paper. The system description and design principles are introduced to ensure the flexibility of the system. And dynamics model for takeover control is developed. After that, the regression models for the parameter identification are deduced based on the dynamics model. An interaction model identification algorithm is presented to solve the parameter identification problem for the distributed cells. Besides, the interactive model identification is validated by simulations. The simulations show that the interactive model identification method can achieve the consensus and convergence.