A novel efficient reinforcement learning paradigm combining human knowledge, model-based and model-free methods is presented for optimal robot manipulation control during complex multi-phase robot manipulation tasks, e.g., the peg-in-hole tasks with tight fit and nut-and-bolt assembly. Firstly, human demonstration is conducted to collect the data during successful robot manipulation, and manipulation phase estimation method integrating with human knowledge is presented to obtain the higher-level planning of the multi-phase robot manipulation tasks. Typical robot manipulation tasks can usually be decomposed into three types of phases, namely free motion, discontinuous contact, and continuous contact. For phase with free motion, the motion planning method is utilized for generating smooth trajectory. For phase with discontinuous contact in the axes of interest during the pre-manipulation process, the rule-based model-free method, namely the Policy Gradients with Human-Guided Parameter-based Exploration (PGHGPE) method is utilized. For the manipulation phase with continuous contacts, the model-based method is utilized because of its higher sample efficiency. Finally, the simulation and experimental studies verify the effectiveness of the presented algorithm. Note to Practitioners—The important premise for the future robot assistants is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable method for solving this kind of problems. However, HRL faces the challenge of low computational efficiency. To this end, efficient manipulation skill learning for complex manipulation tasks via human knowledge, model-based and model-free reinforcement learning methods are presented, which improves the efficiency of the skill learning process to a practical level.

Recent works have combined monocular event camera and inertial measurement unit to estimate the SE(3) trajectory. However, the asynchronicity of event cameras brings a great challenge to conventional fusion algorithms. In this paper, we present an asynchronous event-inertial odometry under a unified Gaussian Process (GP) regression framework to naturally fuse asynchronous data associations and inertial measurements. A GP latent variable model is leveraged to build data-driven motion prior and acquire the analytical integration capacity. Then, asynchronous event-based feature associations and integral pseudo measurements are tightly coupled using the same GP framework. Subsequently, this fusion estimation problem is solved by underlying factor graph in a sliding-window manner. With consideration of sparsity, those historical states are marginalized orderly. A twin system is also designed for comparison, where the traditional inertial preintegration scheme is embedded in the GP-based framework to replace the GP latent variable model. Evaluations on public event-inertial datasets demonstrate the validity of both systems. Comparison experiments show competitive precision compared to the state-of-the-art synchronous scheme.

Robotic manipulation holds the potential to replace humans in the execution of tedious or dangerous tasks. However, control-based approaches are not suitable due to the difficulty of formally describing open-world manipulation in reality, and the inefficiency of existing learning methods. Therefore, applying manipulation in a wide range of scenarios presents significant challenges. In this study, we propose a novel framework for skill learning in robotic manipulation called Tactile Active Inference Reinforcement Learning (TactileAIRL), aimed at achieving efficient learning. To enhance the performance of reinforcement learning (RL), we introduce active inference, which integrates model-based techniques and intrinsic curiosity into the RL process. This integration improves the algorithm’s training efficiency and adaptability to sparse rewards. Additionally, we have designed universal tactile static and dynamic features based on vision-based tactile sensors, making our framework scalable to many manipulation tasks learning involving tactile feedback. Simulation results demonstrate that our method achieves significantly high training efficiency in objects pushing tasks. It enables agents to excel in both dense and sparse reward tasks with just few interaction episodes, surpassing the SAC baseline. Furthermore, we conduct physical experiments on a gripper screwing task using our method, which showcases the algorithm’s rapid learning capability and its potential for practical applications.

This paper proposes a distributed on-orbit spacecraft assembly algorithm, where future spacecraft can assemble modules with different functions on orbit to form a spacecraft structure with specific functions. This form of spacecraft organization has the advantages of reconfigurability, fast mission response and easy maintenance. Reasonable and efficient on-orbit self-reconfiguration algorithms play a crucial role in realizing the benefits of distributed spacecraft. This paper adopts the framework of imitation learning combined with reinforcement learning for strategy learning of module handling order. A robot arm motion algorithm is then designed to execute the handling sequence. We achieve the self-reconfiguration handling task by creating a map on the surface of the module, completing the path point planning of the robotic arm using A*. The joint planning of the robotic arm is then accomplished through forward and reverse kinematics. Finally, the results are presented in Unity3D.

This paper proposes a novel hierarchical reinforcement learning (HRL) framework of complex manipulation tasks which integrates the human prior knowledge. The framework involves the following steps:$ \textbf{1)} $The manipulation process is divided into several stages based on human prior knowledge.$ \textbf{2)} $Transition conditions between stages are determined in the form of “if-then” rules.$ \textbf{3)} $The key features of each stage are selected, and the corresponding control policies are designed via human knowledge.$ \textbf{4)} $The policy gradient with parameter-based exploration (PGPE) method is employed to optimize the policy parameters because it does not require the policy to be derivable for the parameters. To increase the convergence speed of this framework, importance sampling and adaptive adjustment of exploration variance are employed to improve it.$ \textbf{5)} $On this basis, to facilitate the transfer of simulation results to practical experiments, half sim-to-real method is presented, which fully utilizes the simulation results, and the differences between simulation and experimental environments are considered. Simulation and experimental studies show that our framework can deal with the peg-hole-insertion task with a high quality in less than 1600 episodes and can safely adapt the skill into the practical scene with little iterations, which verify the efficiency of the presented method.Note to Practitioners—Intelligent robots will become the right assistants of human beings in the future, especially in various areas of complex manipulation occasions. The important premise is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable and efficient method for solving this kind of problems. This paper proposes a novel HRL framework which can better integrate the human prior knowledge. In addition, improved PGPE method is proposed to obtain the optimized policy parameters more quickly. More importantly, a novel half sim-to-real transfer method is presented to better integrate the simulation and experiment results. This provides a common paradigm, which leverages the simulation results to reduce the interaction between robot and practical environment and then utilizes the experiment results to further optimize the policy parameters.