With the growing popularity of electric vehicles, the demand for robot-based unmanned automatic charging has become both urgent and challenging. Two key challenges need to be addressed: how to efficiently locate the charging port, and how to compliantly insert the connector into the port. In this paper, we propose an incremental learning method based on the broad learning system to address the visual positioning error of the charging port. This method allows the robot to transfer and generalize the search skills learned in simulation to real-world scenarios. As a result, the robot can rapidly locate the charging port in real-world environments without the need for complex contact state modeling, time-consuming data collection, or model retraining. Subsequently, a biomimetic admittance controller is designed to enable the robot to adapt its compliant behavior online during the plugging process. Finally, experiments are performed on a UR robot to verify the effectiveness of our method.

This paper presents a framework for learning and transferring robot tool-use skills based on Dynamic Movement Primitives (DMPs) for robot fine manipulation. DMPs and their enhanced methods are employed to acquire a specific tool-use skill applicable to tools with similar sizes, shapes, and uses. However, the acquired skills may not be transferable to other scenarios and tools with variations. The new framework introduces two new types of skills based on DMPs: Object Operating (O2) skill and Tool Flipping (TF) skill. The O2 skill enables robots to handle tools for manipulating objects to achieve desired effects. The learning process for the O2 skill considers limitations imposed by tools and the environment during human demonstrations. Distinguishing between whether constraints can be modelled or not, we propose both a model-based and a constraint-based method to separate a constraint-irrelevant (CI) skill and the constrained conditions. The CI skill is generalized using a novel method called constrained -DMP lite, enabling adaptation to new tasks with special tools. The TF skill addresses situations where tools must generate an action to alter contacting positions on both objects and tools while avoiding conflicts during movement. Finally, the TF and O2 skills are generalized to be applied in creating a continuous action chain. We conduct several experiments to compare and analyze the advantages and disadvantages of the proposed methods with other approaches in terms of generalizability and calculation complexity.Note to Practitioners—Strengthening robot tool-use ability has been a hot research topic in recent years because these tools can extend the reachability and enhance the flexibility of robots. The previous research on DMPs has been utilized for learning tool-use skills. However, the learned skills few considered the tools’ special use regulations, therefore the skill of using a tool is hard to transfer to another tool-use case. This paper explores tool-use skill learning and transfer between different tools by developing a framework based on the DMPs for this problem. The framework consists of two kinds of skills: O2 skill and TF skill with different purposes as well as a series of newly developed algorithms, such as constrained -DMP lite, a model-based and a constraint-based CI skill learning methods. These methods can separate the constraints from human demonstrations of using tools to achieve a CI skill and generalize the CI skill according to the constraints generated from a new tool-use manipulation task. We verify the effectiveness of the proposed framework through some typical tool-use experiments, including pushing objects, cutting and obstacle avoidance in actuality. The development of this framework can be used in industrial and house working scenarios.

Tele-echography has emerged as a promising and effective solution, leveraging the expertise of sonographers and the autonomy of robots to perform ultrasound scanning for patients residing in remote areas, without the need for in-person visits by the sonographer. Designing effective and natural human-robot interfaces for tele-echography remains challenging, with patient safety being a critical concern. In this article, we develop a teleoperation system for robot-assisted sonography with two different interfaces, a haptic device-based interface and a low-cost 3D Mouse-based interface, which can achieve continuous and intuitive telemanipulation by a leader device with a small workspace. To achieve compliant interaction with patients, we design impedance controllers in Cartesian space to track the desired position and orientation for these two teleoperation interfaces. We also propose comprehensive evaluation metrics of robot-assisted sonography, including subjective and objective evaluation, to evaluate tele-echography interfaces and control performance. We evaluate the ergonomic performance based on the estimated muscle fatigue and the acquired ultrasound image quality. We conduct user studies based on the NASA Task Load Index to evaluate the performance of these two human-robot interfaces. The tracking performance and the quantitative comparison of these two teleoperation interfaces are conducted by the Franka Emika Panda robot. The results and findings provide guidance on human-robot collaboration design and implementation for robot-assisted sonography.Note to Practitioners—Robot-assisted sonography has demonstrated efficacy in medical diagnosis during clinical trials. However, deploying fully autonomous robots for ultrasound scanning remains challenging due to various constraints in practice, such as patient safety, dynamic tasks, and environmental uncertainties. Semi-autonomous or teleoperation-based robot sonography represents a promising approach for practical deployment. Previous work has produced various expensive teleoperation interfaces but lacks user studies to guide teleoperation interface selection. In this article, we present two typical teleoperation interfaces and implement a continuous and intuitive teleoperation control system. We also propose a comprehensive evaluation metric for assessing their performance. Our findings show that the haptic device outperforms the 3D Mouse, based on operators’ feedback and acquired image quality. However, the haptic device requires more learning time and effort in the training stage. Furthermore, the developed teleoperation system offers a solution for shared control and human-robot skill transfer. Our results provide valuable guidance for designing and implementing human-robot interfaces for robot-assisted sonography in practice.

Imitation learning is an important method for the human-robot skill transfer. However, ensuring that skills learned through imitation remain effective in different environments is a challenge. This article addresses the challenge by proposing a stable autonomous dynamic system that can effectively handle obstacles and disturbances while maintaining trajectory accuracy. We introduce an energy-approximated dynamic subattractor (EADA) method that enhances disturbance resistance by dynamically selecting subattractors through Neum (an energy function derived from demonstration data). By combining velocity modulation algorithms with EADA, the system achieves global stability, precise obstacle avoidance, autonomous trajectory recovery, and rapid response. The proposed framework effectively handles complex scenarios, including environments with multiple obstacles, dynamic obstacles, and disturbances. We validate the proposed approach through simulations on the LASA dataset and real-world robotic experiments (both single-arm and dual-arm robots), demonstrating its effectiveness in achieving smooth and accurate obstacle avoidance trajectories with generalization capability.

In this paper, a distributed observer-based prescribed performance control method is proposed for using a multi-robot teleoperation system to manipulate a common deformable object. To achieve a stable position-tracking effect and realize the desired cooperative operational performance, we first define a new hybrid error matrix for both the relative distances and absolute positions of robots and then decompose the matrix into two new error terms for cooperative and independent robot control. Then, we improve the Kelvin-Voigt (K-V) contact model based on the new error terms. Because the center position and deformation of the object cannot be measured, the object dynamics are then expressed by the relative distances of robots and an equivalent impedance term. Each robot incorporates an observer to estimate contact force and object dynamics based on its own measurements. To address the position errors caused by biases in force estimation and realize the position-tracking effect of each robot, we improve the barrier Lyapunov functions (BLFs) by incorporating the errors into system control. which allows us to achieve a predefined position-tracking effect. We conduct an experiment to verify the proposed controller’s ability in a dual-telerobot cooperative manipulation task, even when the object is subjected to unknown disturbances.Note to Practitioners—This article is inspired by the limitations of multi-telerobot manipulation with a deformable object, where the deformation of the object cannot be measured directly. Meanwhile, force sensors, especially 6-axis force sensors, are very expensive. To realize the purpose that objects manipulated by multiple robots match the same state as operated on the leader side, we propose an object-centric teleoperation framework based on the estimates of contact forces and object dynamics and the improved barrier Lyapunov functions (BLFs). This framework contributes to two aspects in practice: 1) propose a control diagram for deformable object co-teleoperation of multi-robots for unmeasurable object’s centre position and deformation; 2) propose an improved BLFs controller based on the estimation of contact force and robot dynamics. The estimation errors are considered and transferred using an equivalent impedance to be integrated into the Lyapunov function to minimize both force and motion-tracking errors. The experimental results verify the effectiveness of the proposed method. The developed framework can be used in industrial applications with a similar scenario.

Tactile sensors can be used for motion detection and object perception in robot manipulation. The contact detection within the camera's visual inspection area has been well-developed, but perception outside the field of view of the camera is overlooked. In this paper, we present a new tendon-linked tactile sensor, MechTac, to achieve perceptions inside and outside the field of view. The MechTac is an evolution of the typical TacTip sensor with the following two advantages. 1) The ability to provide perception outside the field of view. This is achieved by using a network of braided tendons to transfer deformation from the blind perception regions (TacSide) to the visual areas (TacTip). 2) The tactility of the TacSide and TacTip is reflected by the movements of multiple papillae pins and visible markers on the pin tips on the inner surface of the TacTip. The pins and markers are differentially sensitive to various touch features, which is similar to the differentiated perceptual ability of humans. TacTip is more sensitive to small touches, corresponding to the fingertip, while the TacSide is less sensitive but has a larger perceptual area, corresponding to the middle part of the finger. Moreover, we propose a new deep learning method to decompose the mixed information affected by the TacSide and the TacTip. A modified DenseNet121 was specifically designed for object perception at the TacTip and localization at the TacSide. The experimental results show that prediction accuracy reaches about 98% for object perception or localization and over 99% for the case requiring two functions.

Due to changes in the environment and errors that occurred during skill initialization, the robot's operational skills should be modified to adapt to new tasks. As such, skills learned by the methods with fixed features, such as the classical Dynamical Movement Primitive (DMP), are difficult to use when the using cases are significantly different from the demonstrations. In this work, we propose an incremental robot skill learning and generalization framework including an incremental DMP (IDMP) for robot trajectory learning and an adaptive neural network (NN) control method, which are incrementally updated to enable robots to adapt to new cases. IDMP uses multi-mapping feature vectors to rebuild the forcing function of DMP, which are extended based on the original feature vector. In order to maintain the original skills and represent skill changes in a new task, the new feature vector consists of three parts with different usages. Therefore, the trajectories are gradually changed by expanding the feature and weight vectors, and all transition states are also easily recovered. Then, an adaptive NN controller with performance constraints is proposed to compensate dynamics errors and changed trajectories after using the IDMP. The new controller is also incrementally updated and can accumulate and reuse the learned knowledge to improve the learning efficiency. Compared with other methods, the proposed framework achieves higher tracking accuracy, realizes incremental skill learning and modification, achieves multiple stylistic skills, and is used for obstacle avoidance with different heights, which are verified in three comparative experiments.

Abstract In this paper, composite dynamic movement primitives (DMPs) based on radial basis function neural networks (RBFNNs) are investigated for robots’ skill learning from human demonstrations. The composite DMPs could encode the position and orientation manipulation skills simultaneously for human-to-robot skills transfer. As the robot manipulator is expected to perform tasks in unstructured and uncertain environments, it requires the manipulator to own the adaptive ability to adjust its behaviours to new situations and environments. Since the DMPs can adapt to uncertainties and perturbation, and spatial and temporal scaling, it has been successfully employed for various tasks, such as trajectory planning and obstacle avoidance. However, the existing skill model mainly focuses on position or orientation modelling separately; it is a common constraint in terms of position and orientation simultaneously in practice. Besides, the generalisation of the skill learning model based on DMPs is still hard to deal with dynamic tasks, e.g., reaching a moving target and obstacle avoidance. In this paper, we proposed a composite DMPs-based framework representing position and orientation simultaneously for robot skill acquisition and the neural networks technique is used to train the skill model. The effectiveness of the proposed approach is validated by simulation and experiments.

Different from previous work on single skill learning from human demonstrations, an incremental motor skill learning, generalization and control method based on dynamic movement primitives (DMP) and broad learning system (BLS) is proposed for extracting both ordinary skills and instant reactive skills from demonstrations, the latter of which is usually generated to avoid a sudden danger (e.g., touching a hot cup). The method is completed in three steps. First, the ordinary skills are basically learned from demonstrations in normal cases by using DMP. Then, the incremental learning idea of BLS is combined with DMP to achieve multistylistic reactive skill learning such that the forcing function of the ordinary skills will be reasonably extended into multiple stylistic functions by adding enhancement terms and updating weights of the radial basis function kernels. Finally, electromyography signals are collected from human muscles and processed to achieve stiffness factors. By using fuzzy logic system, the two kinds of skills learned are integrated and generalized in new cases such that not only start, end and scaling factors but also the environmental conditions, robot reactive strategies and impedance control factors will be generalized to lead to various reactions. To verify the effectiveness of the proposed method, an obstacle avoidance experiment that enables robots to approach destinations flexibly in various situations with barriers will be undertaken.

Robots are often required to generalize the skills learned from human demonstrations to fulfil new task requirements. However, skill generalization will be difficult to realize when facing with the following situations: the skill for a complex multistep task includes a number of features; some special constraints are imposed on the robots during the process of task reproduction; and a completely new situation quite different with the one in which demonstrations are given to the robot. This work proposes a new framework to facilitate robot skill generalization. The basic idea lies in that the learned skills are first segmented into a sequence of subskills automatically, then each individual subskill is encoded and regulated accordingly. Specifically, we adapt each set of the segmented movement trajectories individually instead of the whole movement profiles, thus, making it more convenient for the realization of skill generalization. In addition, human limb stiffness estimated from surface electromyographic signals is considered in the framework for the realization of human-to-robot variable impedance control skill transfer, as well as the generalization of both movement trajectories and stiffness profiles. Experimental study has been performed to verify the effectiveness of the proposed framework.

This paper reports the development of an intelligent shared control system for a robotic manipulator that is commanded by the user's mind. The target objects are detected by a vision system and then displayed to the user in a video that shows them fused with flicking diamonds that are designed to excite electroencephalograph (EEG) signals at different frequency bands. Through the analysis of the invoked EEG signals, a brain-computer interface is developed to infer the exact object that is required by the user. These results are then transferred to the shared control system, which is enabled by visual servoing techniques to achieve accurate object manipulation. The task motion and self-motion (CTS) methods are coordinated to enhance the intelligence of the shared control system by equipping the robot with an autonomous obstacle avoidance function. Extensive experimental studies are performed to verify that the adaptive object tracking algorithm, the CTS method, and the least-squares method are helpful in improving the performance of the intelligent robotic system.