Enabling robots to grasp disorganized cloth for efficient storage is valuable in robot-assisted room organization. Diverse deformations of cloth and the stacking of multiple items limit grasping-pose estimation that relies on annotations. This necessitates segmenting each cloth item in an unsupervised manner before estimating the grasping position. However, existing segmentation methods primarily focus on improving metrics such as Intersection-over-Union and Pixel Accuracy, which cannot effectively measure the segmentation errors of the cloth area and thus lead to failure grasping position estimation. To address this challenge, we use False Discovery Rate (FDR) as a novel measure of segmentation errors and analyze its impact on grasping success. Our preliminary study reveals a negative correlation between segmentation FDR and grasping success rate, highlighting the need for more reliable segmentation in cluttered cloth scenarios. Therefore, we propose an unsupervised cloth segmentation network based on feature distance-weighted constraints, designed to reduce the false discovery rate in cloth area perception without requiring expensive pixel-level manual annotations. Additionally, to estimate the grasping position on the perceived cloth area, we introduce a strategy based on cloth surface wrinkle analysis, which operates without the need for annotations or training. By integrating the proposed segmentation network and grasping strategy, we develop a robotic system capable of sequentially grasping cluttered cloth from a table. Extensive real-world robotic experiments demonstrate the effectiveness of our approach, outperforming multiple baseline methods in segmentation FDR and grasping success rate.

As one of the most fundamental control modes in robotics, force/torque (F/T) control plays an essential role in a wide range of applications. However, classical F/T control fails to offer effective means to regulate the convergence sequence of the controlled states, which is beneficial in many real-world tasks, e.g., unknown surface contact, where the force should preferably converge later than the alignment angles to ensure sufficient contact and avoid dangerous misalignment. In this work, a novel nested fast terminal sliding mode control approach is proposed. This approach establishes a hierarchical structure for the controlled states, such that the Lyapunov stabilities of controlled states can be achieved in both a sequential and a time-synchronized manner within finite time, which is named as ‘Sequen-Sync’. Extensive experiments are conducted for various tasks in two different environments. The experimental results show that the proposed approach successfully achieves Sequen-Sync stability, which leads to improved contact quality and enhanced safety.

Impedance control can be achieved within a model predictive control (MPC) framework for optimization and constraint compliance. However, user-defined or optimization-derived impedance models can be too conservative to achieve a timely convergence, or too aggressive to ensure safety. To address this, an MPC-based impedance control framework with learning-based tuning for predefined-time (PdT) convergence is proposed. On the low level, the framework dynamically selects between a task-oriented and a safety-oriented impedance model based on real-time interaction force modeling and safety assessments, ensuring optimal performance and maintaining safety while interacting with unknown and complex environments. On the high level, the framework achieves PdT convergence via reinforcement learning for meta-parameter tuning, allowing users to specify the desired convergence time upper bound. Lastly, the superiority of the proposed framework is validated on interaction safety and PdT convergence via experiments.

Recognizing various types of clothing is crucial for robotic clothing manipulation tasks, such as garment organization and robot-assisted dressing. Unlike rigid object recognition, clothing recognition remains a challenging task due to the diverse forms introduced by flexible deformations. However, existing classification models primarily focus on clothing color and texture while overlooking structural features, limiting their ability to distinguish between deformable clothing categories with similar color and texture. Moreover, due to the insufficient representation of structural features, these models heavily rely on manually annotated labels, making it difficult to accurately recognize unseen clothing items with new colors or textures. To address these challenges, we propose a novel topological structure representation and optimization strategy for category-level clothing structural feature learning. Additionally, we design a multi-clothing classification framework based on multiple mask generation to identify clothing regions within a scene. By leveraging our proposed structural feature learning strategy, our framework effectively generalizes to unseen clothing items. Finally, we introduce a fabric-specific grasping position estimation method and develop a corresponding robotic grasping system capable of selecting and grasping specified clothing items based on user instructions. Extensive real-world robotic experiments demonstrate the effectiveness of our system, and comprehensive comparisons with multiple baselines further validate the superiority of our approach.

The potential of tactile sensing for dexterous robotic manipulation has been demonstrated by its ability to enable nuanced real-world interactions. In this study, the retrieval of unknown objects from confined spaces, which is unsuitable for conventional visual perception and gripper-based manipulation, is identified and addressed. Specifically, a tactile-sensorized tool stick that well fits in the narrow space is utilized to provide multi-point contact sensing for object manipulation. A reinforcement learning (RL) agent with a hybrid action space is then proposed to acquire the optimal policy for manipulating the objects without prior knowledge of their physical properties. To accelerate on-hardware training, a focused training strategy is adopted with the hypothesis that an agent trained on a small set of representative shapes can be generalized to a wide range of everyday objects. Additionally, a curriculum on terminal goals is designed to further accelerate the hardware-based training process. Comparative experiments and ablation studies have been conducted to evaluate the effectiveness and robustness of the proposed approach, which highlights the high success rate of our solution for retrieving everyday objects.

Optical flow estimation aims to find the 2D motion field by identifying corresponding pixels between two images. Despite the tremendous progress of deep learning-based optical flow methods, it remains a challenge to accurately estimate large displacements with motion blur. This is mainly because the correlation volume, the basis of pixel matching, is computed as the dot product of the convolutional features of the two images. The locality of convolutional features makes the computed correlations susceptible to various noises. On large displacements with motion blur, noisy correlations could cause severe errors in the estimated flow. To overcome this challenge, we propose a new architecture “CRoss-Attentional Flow Trans-former” (CRAFT), aiming to revitalize the correlation volume computation. In CRAFT, a Semantic Smoothing Trans-former layer transforms the features of one frame, making them more global and semantically stable. In addition, the dot-product correlations are replaced with trans-former Cross-Frame Attention. This layer filters out feature noises through the Query and Key projections, and computes more accurate correlations. On Sintel (Final) and KITTI (foreground) benchmarks, CRAFT has achieved new state-of-the-art performance. Moreover, to test the robust-ness of different models on large motions, we designed an image shifting attack that shifts input images to generate large artificial motions. Under this attack, CRAFT per-forms much more robustly than two representative meth-ods, RAFT and GMA. The code of CRAFT is is available at https://github.com/askerlee/craft.

In deep reinforcement learning (DRL), the agent is usually trained on seen environments by optimizing a policy network. However, it is difficult to be generalized to unseen environments properly, even when the environmental variations are insignificant. This is partly because the policy network cannot effectively learn the representation of visual difference that is subtle among highly similar states in the environments. Because a bilinear structured model containing two feature extractors allows pairwise feature interactions in a translation-ally invariant manner which makes it particularly useful for subtle difference recognition among highly similar states, in this work, a bilinear policy network is employed to enhance representation learning, and thus to improve generalization of the DRL. The proposed bilinear policy network is tested on various DRL task, including a control task on path planning for active object detection, and Grid World, an AI game task. The test results show that the generalization of DRL can be improved by the proposed network.

Planar pushing is a fundamental robot manipulation task with most algorithms built upon the quasi-static as-sumption. Under this assumption the end-effector should apply force on the pushed object along the full moving trajectory. This means that the target position must lie in the robot's workspace. To enable a robot to deliver objects outside of its workspace and facilitate faster delivery, the quasi-static assumption should be lifted in favour of dynamical manipulation. In this work, we propose a two-staged data-driven manipulation method to hit an unknown object to reach a target position. This expands the reachability of the manipulated object beyond the robot's workspace. The robot equipped with a tactile sensor first explores for the stable pushing region (SPR) on the given object by using a gain-scheduling PD control with the contact centre estimated to maintain full contact between the object and the end-effector. In the second stage, a learning-based approach is used to generate the impulse the object should receive at the SPR to reach a target sliding distance. The performance of proposed method is evaluated on a KUKA LBR iiwa 14 R820 robot manipulator and a XELA tactile sensor.

Previous research on finite-time control focuses on forcing a system state (vector) to converge within a certain time moment, regardless of how each state element converges. In the present work, we introduce a control problem with unique finite/fixed-time stability considerations, namely time-synchronized stability (TSS), whereat the same time, all the system state elements converge to the origin, and fixed-TSS, where the upper bound of the synchronized settling time is invariant with any initial state. Accordingly, sufficient conditions for (fixed-) TSS are presented. On the basis of these formulations of the time-synchronized convergence property, the classical sign function, and also anorm-normalized sign function, are first revisited. Then in terms of this notion of TSS, we investigate their differences with applications in control system design for first-order systems (to illustrate the key concepts and outcomes), paying special attention to their convergence performance. It is found that while both these sign functions contribute to system stability, nevertheless an important result can be drawn that norm-normalized sign functions help a system to additionally achieve TSS. Furthermore, we propose a fixed-time-synchronized sliding-mode controller for second-order systems; and we also consider the important related matters of singularity avoidance there. Finally, numerical simulations are conducted to present the (fixed-) time-synchronized features attained; and further explorations of the merits of the proposed (fixed-) TSS are described.

When deploying a robot to a new task, one often has to train it to detect novel objects, which is time-consuming and labor- intensive. We present TAILOR - a method and system for ob- ject registration with active and incremental learning. When instructed by a human teacher to register an object, TAILOR is able to automatically select viewpoints to capture informa- tive images by actively exploring viewpoints, and employs a fast incremental learning algorithm to learn new objects without potential forgetting of previously learned objects. We demonstrate the effectiveness of our method with a KUKA robot to learn novel objects used in a real-world gearbox as- sembly task through natural interactions.

Manipulation in a densely cluttered environment creates complex challenges in perception to close the control loop, many of which are due to the sophisticated physical interaction between the environment and the manipulator. Drawing from biological sensory-motor control, to handle the task in such a scenario, tactile sensing can be used to provide an additional dimension of the rich contact information from the interaction for decision making and action selection to manoeuvre towards a target. In this paper, a new tactile-based motion planning and control framework based on bioinspiration is proposed and developed for a robot manipulator to manoeuvre in a cluttered environment. An iterative two-stage machine learning approach is used in this framework: an autoencoder is used to extract important cues from tactile sensory readings while a reinforcement learning technique is used to generate optimal motion sequence to efficiently reach the given target. The framework is implemented on a KUKA LBR iiwa robot mounted with a SynTouch BioTac tactile sensor and tested with real-life experiments. The results show that the system is able to move the end-effector through the cluttered environment to reach the target effectively.

Active vision is a desirable perceptual feature for robots. Existing approaches usually make strong assumptions about the task and environment, thus are less robust and efficient. This study proposes an adaptive view planning approach to boost the efficiency and robustness of active object detection. We formulate the multi-object detection task as an active multiview object detection problem given the initial location of the objects. Next, we propose a novel adaptive action prediction (A2P) method built on a deep Q-learning network with a dueling architecture. The A2P method is able to perform view planning based on visual information of multiple objects; and adjust action ranges according to the task status. Evaluated on the AVD dataset, A2P leads to 21.9% increase in detection accuracy in unfamiliar environments, while improving efficiency by 22.7%. On the T-LESS dataset, multi-object detection boosts efficiency by more than 30% while achieving equivalent detection accuracy.

The ability to simultaneously distinguish objects, materials, and their associated physical properties is one fundamental function of the sense of touch. Recent advances in the development of tactile sensors and machine learning techniques allow more accurate and complex modelling of robotic tactile sensations. However, many state-of-the-art (SotA) approaches focus solely on constructing black-box models to achieve ever higher classification accuracy and fail to adapt across sensors with unique spatial-temporal data formats. In this work, we propose an Explainable and Sensor-Adaptable Recurrent Networks (ExSARN) model for tactile texture representation. The ExSARN model consists of a two-stage recurrent networks fed by a sensor-specific header network. The first stage recurrent network emulates our human touch receptors and decouples sensor-specific tactile sensations into different frequency response bands, while the second stage codes the overall temporal signature as a variational recurrent autoen-coder. We infuse the latent representation with ternary labels to qualitatively represent texture properties (e.g. roughness and stiffness), which facilitates representation learning and provide explainability to the latent space. The ExSARN model is tested on texture datasets collected with two different tactile sensors. Our results show that the proposed model not only achieves higher accuracy, but also provides adaptability across sensors with different sampling frequencies and data formats. The addition of the crudely obtained qualitative property labels offers a practical approach to enhance the interpretability of the latent space, facilitate property inference on unseen materials, and improve the overall performance of the model.

6D object pose estimation is widely applied in robotic tasks such as grasping and manipulation. Prior methods using RGB-only images are vulnerable to heavy occlusion and poor illumination, so it is important to complement them with depth information. However, existing methods using RGB-D data cannot adequately exploit consistent and complementary information between RGB and depth modalities. In this paper, we present a novel method to effectively consider the correlation within and across both modalities with attention mechanism to learn discriminative and compact multi-modal features. Then, effective fusion strategies for intra- and inter-correlation modules are explored to ensure efficient information flow between RGB and depth. To our best knowledge, this is the first work to explore effective intra- and inter-modality fusion in 6D pose estimation. The experimental results show that our method can achieve the state-of-the-art performance on LineMOD and YCB-Video dataset. We also demonstrate that the proposed method can benefit a real-world robot grasping task by providing accurate object pose estimation.

In this paper we present a framework to learn skills from human demonstrations in the form of geometric nullspaces, which can be executed using a robot. We collect data of human demonstrations, fit geometric nullspaces to them, and also infer their corresponding geometric constraint models. These geometric constraints provide a powerful mathematical model as well as an intuitive representation of the skill in terms of the involved objects. To execute the skill using a robot, we combine this geometric skill description with the robot's kinematics and other environmental constraints, from which poses can be sampled for the robot's execution. The result of our framework is a system that takes the human demonstrations as input, learns the underlying skill model, and executes the learnt skill with different robots in different dynamic environments. We evaluate our approach on a simulated industrial robot, and execute the final task on the iCub humanoid robot.

The sense of touch is an essential sensing modality for a robot to interact with the environment as it provides rich and multimodal sensory information upon contact. It enriches the perceptual understanding of the environment and closes the loop for action generation. One fundamental area of perception that touch dominates over other sensing modalities, is the understanding of the materials that it interacts with, for example, glass versus plastic. However, unlike the senses of vision and audition which have standardized data format, the format for tactile data is vastly dictated by the sensor manufacturer, which makes it difficult for large-scale learning on data collected from heterogeneous sensors, limiting the usefulness of publicly available tactile datasets. This paper investigates the joint learnability of data collected from two tactile sensors performing a touch sequence on some common materials. We propose a supervised recurrent autoencoder framework to perform joint material classification task to improve the training effectiveness. The framework is implemented and tested on the two sets of tactile data collected in sliding motion on 20 material textures using the iCub RoboSkin tactile sensors and the SynTouch BioTac sensor respectively. Our results show that the learning efficiency and accuracy improve for both datasets through the joint learning as compared to independent dataset training. This suggests the usefulness for large-scale open tactile datasets sharing with different sensors.

We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicle Routing Problem (SC-VRP) is formulated and subsequently solved by a modified Biased Random Key Genetic Algorithm (BRKGA) with novel, efficient encoding strategies and local improvement heuristics. We test our proposed method for several complex 3D structures with the 3D model extracted from OpenStreetMap. The proposed method outperforms previous methods, by reducing the length of the planned inspection path by up to 48%.

Robotic systems are increasingly required not only to generate precise motions to complete their tasks but also to handle the interactions with the environment or human. Significantly, soft interaction brings great challenges on the force control due to the nonlinear, viscoelastic and inhomogeneous properties of the soft environment. In this paper, a robust impedance control scheme utilizing integral backstepping technology and integral terminal sliding mode control is proposed to achieve force tracking for an ultrasonic motor-actuated end-effector in a soft environment. In particular, the steady-state performance of the target impedance while in contact with soft environment is derived and analyzed with the nonlinear Hunt-Crossley model. Finally, the dynamic force tracking performance of the proposed control scheme is verified via several experiments.

Touch is arguably the most important sensing modality in physical interactions. However, tactile sensing has been largely under-explored in robotics applications owing to the complexity in making perceptual inferences until the recent advancements in machine learning or deep learning in particular. Touch perception is strongly influenced by both its temporal dimension similar to audition and its spatial dimension similar to vision. While spatial cues can be learned episodically, temporal cues compete against the system's re-sponse/reaction time to provide accurate inferences. In this paper, we propose a fast tactile-based texture classification framework which makes use of the spiking neural network to learn from the neural coding of the conventional tactile sensor readings. The framework is implemented and tested on two independent tactile datasets collected in sliding motion on 20 material textures. Our results show that the framework is able to make much more accurate inferences ahead of time as compared to that by the state-of-the-art learning approaches.

The sense of touch is arguably the first human sense to develop. Empowering robots with the sense of touch may augment their understanding of interacted objects and the environment beyond standard sensory modalities (e.g., vision). This paper investigates the effect of hybridizing touch and sliding movements for tactile-based texture classification. We develop three machine-learning methods within a framework to discriminate between surface textures; the first two methods use hand-engineered features, whilst the third leverages convolutional and recurrent neural network layers to learn feature representations from raw data. To compare these methods, we constructed a dataset comprising tactile data from 23 textures gathered using the iCub platform under a loosely constrained setup, i.e., with nonlinear motion. In line with findings from neuroscience, our experiments show that a good initial estimate can be obtained via touch data, which can be further refined via sliding; combining both touch and sliding data results in 98% classification accuracy over unseen test data.

Soft robots have exhibited significant advantages compared to conventional rigid robots due to the high-energy density and strong environmental compliance. Among the soft materials explored for soft robots, dielectric elastomers (DEs) stand out with the muscle-like actuation behaviors. However, recently, modeling and control of a DE-based soft robot still remain a challenging because of the nonlinearity and viscoelasticity of DE actuators. This paper focuses on the design, modeling and control of a soft circular robot which is able to achieve a 2D motion. To facilitate the design of a motion controller, a dynamic model of the robot is investigated through experimental identification. Based on the model, a feedforward plus feedback control scheme is adopted for the motion control of the robot. Finally, both simulations and experiments are conducted to verify the effectiveness of the proposed model and control approach.

Recent studies show that the growth of social robotics market will dominate the robotics sector by 2025. Social robots are set to enter and co-exist in daily living, making human-robot interaction studies a crucial research direction in these robotics applications. One such application is the deployment of robotics in information dissemination to augment the shrinking workforce in developed economies, such as advertisement campaigns. This work seeks to understand whether live robot advertiser can better engage audience and improve audience perception of a marketed product as compared to pre-filmed video advertising clips. In this preliminary study, a humanoid robot, the NAO, was programmed to act according to a marketing campaign script, engaging the audience through voice, simulated eye-contacts, gestures and interaction with the Keepon, a robot to be advertised. We carried out three sets of experiments in the CBD of Singapore on teenage pedestrians. The results of this study suggest that physical robot presence will enhance information dissemination and hence improve advertising result, providing value-add to the advertised product.

Robot learning by demonstration gives robots the ability to learn tasks which they have not been programmed to do before. The paradigm allows robots to work in a greater range of real-world applications in our daily life. However, this paradigm has traditionally been applied to learn tasks from a single demonstration modality. This restricts the approach to be scaled to learn and execute a series of tasks in a real-life environment. In this paper, we propose a multi-modal learning approach using DMP+ with linear decay integrated in a dialogue system with speech and ontology for the robot to learn seamlessly through natural interaction modalities (like an apprentice) while learning or re-learning is done on the fly to allow partial updates to a learned task to reduce potential user fatigue and operational downtime in teaching. The performance of new DMP+ with linear decay system is statistically benchmarked against state-of-the-art DMP implementations. A gluing demonstration is also conducted to show how the system provides seamless learning of multiple tasks in a flexible manufacturing set-up.

This paper is concerned with divisible shared control, which decomposes the motion space into complementary subspaces and distributes the control to the human and the robot so that each can independently effect motion control in its subspace. We present a divisible shared control scheme to assist teleoperation tasks on a curved object surface, which is difficult for a human to perform without assistance. We designed and carried an experiment to investigate its effect of user performance and work load. Experimental evaluation, based on both quantitative and qualitative measures, suggests that divisible shared control improves accuracy, speed, and smoothness, while at the same time reduces cognitive load, effort, and frustration.

In the last few years, a shift from mass production to mass customisation is observed in the industry. Easily reprogrammable robots that can perform a wide variety of tasks are desired to keep up with the trend of mass customisation while saving costs and development time. Learning by Demonstration (LfD) is an easy way to program the robots in an intuitive manner and provides a solution to this problem. In this work, we discuss and evaluate LAP, a three-stage LfD method that conforms to the criteria for the high-mix-low-volume (HMLV) industrial settings. The algorithm learns a trajectory in the task space after which small segments can be adapted on-the-fly by using a human-in-the-loop approach. The human operator acts as a high-level adaptation, correction and evaluation mechanism to guide the robot. This way, no sensors or complex feedback algorithms are needed to improve robot behaviour, so errors and inaccuracies induced by these subsystems are avoided. After the system performs at a satisfactory level after the adaptation, the operator will be removed from the loop. The robot will then proceed in a feed-forward fashion to optimise for speed. We demonstrate this method by simulating an industrial painting application. A KUKA LBR iiwa is taught how to draw an eight figure which is reshaped by the operator during adaptation.

The global trend of population ageing has magnified the shortage of qualified staff in the elderly care industry. This study evaluates the feasibility and user experience of introducing robots in elderly care services. A robot instructor was being benchmarked against a human instructor administering two types of activities with 41 elderly participants. The results show that robot was more effective and better preferred by users over human instructor on instructing physical exercise, while reaching similar level of effectiveness and user acceptance on information delivery. Additionally, user perception of robots improved after the robot experiment session. These findings could be useful for future design of robots for elderly users and for social robots in general.

This paper describes the current work on using humanoid robot to augment traditional storytelling for educational and entertainment purposes in casual contexts such as home and classroom. We explore a novel method of Human-Robot Collaboration (HRC) for storytelling and address the question how robots may best augment storytelling. In the pilot study, a humanoid robot, Aldebaran's Nao, was programmed to recite a story to 60 students aged 14 to 15. Nao delivered the performance as either an independent storyteller, or as a collaborator with a human storyteller. We assessed the effectiveness of HRC by comparing the participants' preference over the two settings. We found that 1) most participants prefer HRC over the robot-only performance (RO) and considered HRC effective; and 2) the preference in HRC is explained by the complementary strengths of the robot and human storyteller in interacting with the participants. These results, provide a first step towards effective use of robots for collaborative storytelling in daily situations.

Dynamic Movement Primitives (DMPs) are a generic approach for trajectory modeling in an attractor land-scape based on differential dynamical systems. DMPs guarantee stability and convergence properties of learned trajectories, and scale well to high dimensional data. In this paper, we propose DMP+, a modified formulation of DMPs which, while preserving the desirable properties of the original, 1) achieves lower mean square error (MSE) with equal number of kernels, and 2) allows learned trajectories to be efficiently modified by updating a subset of kernels. The ability to efficiently modify learned trajectories i) improves reusability of existing primitives, and ii) reduces user fatigue during imitation learning as errors during demonstration may be corrected later without requiring another complete demonstration. In addition, DMP+ may be used with existing DMP techniques for trajectory generalization and thus complements them. We compare the performance of our proposed approach against DMPs in learning trajectories of handwritten characters, and show that DMP+ achieves lower MSE in position deviation. We demonstrate in a second experiment that DMP+ can efficiently update a learned trajectory by updating only a subset of kernels. The update algorithm achieves modeling accuracy comparable to learning the adapted trajectory with the original DMPs.

In this paper, we propose a framework to analyze the interactive behaviors of humans and robots in physical interactions. Game theory is employed to describe the system under study, and policy iteration is adopted to provide a solution of Nash equilibrium. The human's control objective is estimated based on the measured interaction force, and it is used to adapt the robot's objective such that human-robot coordination can be achieved. The validity of the proposed method is verified through a rigorous proof and experimental studies.

Learning from demonstration (LfD) provides an easy and intuitive way to program robot behaviours, potentially reducing development time and costs tremendously. This is especially appealing for manufacturers interested in using industrial manipulators for high-mix production, since this technique enables fast and flexible modifications to the robot behaviours and is thus suitable to teach the robot to perform a wide range of tasks regularly. We define a set of criteria to assess the applicability of state-of-the-art LfD frameworks in the industry. A three-stage LfD method is then proposed, which incorporates human-in-the-loop adaptation to iteratively correct a batch-learned policy to improve accuracy and precision. The system will then transit to open-loop execution of the task to enhance production speed, by removing the human teacher from the feedback loop. The proposed LfD framework addresses all criteria set in this work.

In robot-assisted upper limb rehabilitation, detecting the intentions of hemiplegic patients is essential towards assisting the patients to actively exercise instead of driving passive motions. Many interactive channels, such as voice, EMG and EEG, have been studied to estimate the motion intentions. However, limitations of these techniques, such as high complexity, have constrained their applications in practice. In this paper, we integrate a virtual environment and a low-cost motion sensor into a novel control strategy to detect motion intentions for a rehabilitation robot. Several bimanual motion sequences are intuitively programmed by a professional therapist for subjects to repeat. The strategy uses the unaffected arm and the programmed motion sequence to estimate the motion intentions of the affected arm. We adopt this strategy in Mirror Therapy, a widely-practised therapeutic intervention method. Experiments have been conducted to validate the control strategy.

In this paper, we propose an adaptive optimal control for a robot to collaborate with a human. Game theory and policy iteration are employed to analyze the interactive behaviors of the human and the robot in physical interactions. The human's control objective is estimated and it is used to adapt the robot's own objective, such that human-robot coordination can be achieved. An optimal control is developed to guarantee that the robot's control objective is realized. The validity of the proposed method is verified through rigorous analysis and experiment studies.

Many recent studies describe micromanipulation systems by using complex Analytic Forward Models (AFM), but such models are difficult to build and incapable of describing unmodelable factors, such as manufacturing defects. In this work, we propose the Enhanced Analytic Forward Model (EAFM), an integrated model of the AFM and the Heteroscedastic Gaussian Processes (HGP). The EAFM can compensate the shortfalls of the AFM by training the HGP on the residual of the AFM. This also allows the HGP to learn the repeatability of the micromanipulation system. Based on the EAFM, we further contribute an optimal position controller for improving the accuracy and the repeatability. This optimal EAFM controller is implemented and tested on a three degree-of-freedom micromanipulator based micromanipulation system. Two sets of real-world experiments are carried out to verify our method. The results demonstrate that the controller using EAFM can statistically achieve higher accuracy and repeatability than solely using the AFM.

Recent studies on underactuated manipulation usually describe the system with a Kinematic Model (KM), which is built by adding external constraints to the standard manipulation analysis method. However, such external constraints are easily violated in a real-world dexterous manipulation task which results in significant control errors. In this work, the Enhanced Kinematic Model (E-KM), an integrated model of the KM and the Sparse Online Gaussian Process (SOGP) is proposed. The E-KM can compensate the shortfalls of the KM by on-the-fly training the SOGP on the residual between the prediction of the KM and the ground truth data. Based on the E-KM, we further contribute an optimal controller for underactuated manipulations. This optimal E-KM controller is implemented and tested on the iCub, a humanoid robot with two anthropomorphic underactuated hands. Two sets of real-world experiments are carried out to verify our method. The results demonstrate that the controller using E-KM statistically can achieve higher control accuracy than using solely using the KM for a wide range of objects.

We consider the issue of segmenting an action in the learning phase into a logical set of smaller primitives in order to construct a generative model for imitation learning using a hierarchical approach. Our proposed framework, addressing the “how-to” question in imitation, is based on a one-shot imitation learning algorithm. It incorporates segmentation of a demonstrated template into a series of subactions and takes a hierarchical approach to generate the task action by using a finite state machine in a generative way. Two sets of experiments have been conducted to evaluate the performance of the framework, both statistically and in practice, through playing a tic-tac-toe game. The experiments demonstrate that the proposed framework can effectively improve the performance of the one-shot learning algorithm and reduce the size of primitive space, without compromising the learning quality.

Teaching a robot to learn new knowledge is a repetitive and tedious process. In order to accelerate the process, we propose a novel template-based approach for robot arm movement imitation. This algorithm selects a previously observed path demonstrated by a human and generates a path in a novel situation based on pairwise mapping of invariant feature locations present in both the demonstrated and the new scenes using a combination of minimum distortion and minimum energy strategies. This One-Shot Learning algorithm is capable of not only mapping simple point-to-point paths but also adapting to more complex tasks such as those involving forced waypoints. As compared to traditional methodologies, our work require neither extensive training for generalisation nor expensive run-time computation for accuracy. This algorithm has been statistically validated using cross-validation of grasping experiments as well as tested for practical implementation on the iCub humanoid robot for playing the tic-tac-toe game.