The coordination of the multiple degrees-of-freedom of the human body may be simplified by muscle synergies, motor modules which can be flexibly combined to achieve various goals. Studies investigating adaptation to novel relationships between muscle activity and task outcomes found that altering the recruitment of such modules is faster than the learning of their structures de novo. However, how learning new synergy recruitments or new synergy structures may occur remains unclear. While trial-by-trial learning of novel sensorimotor tasks has been successfully modeled at the level of task variables, few models accounted for the redundancy of the motor system, particularly at the muscular level. However, these models either did not consider a modular architecture of the motor system, or assumed a priori knowledge of the sensorimotor task. Here, we present a computational model for the generation of redundant muscle activity where explicitly defined modules, implemented as spatial muscle synergies, can be updated together with their recruitment coefficients through an error-based learning process dependent on a forward model of the sensorimotor task, which is not assumed to be known a priori. Our model can qualitatively reproduce the experimental observations of slower learning and larger changes in the structure of the muscle activity under sensorimotor tasks that require the learning of novel patterns of muscle activity, providing further insights into the modular organization of the human motor system.

When a human controls a robot directly or via teleoperation, incomplete information and unpredictable environmental conditions can lead to conflicts between their plans. Differential game theory (GT) offers a framework for optimal robotic assistance, but existing methods for identifying the human model require a shared plan. This article introduces an approach to deal with a diverging human plan by leveraging a neuromechanical model of their viscoelasticity to directly estimate their motion intent during movement. The viscoelastic gains can then be integrated into a GT framework to compute optimal contribution of the robot to the common motor task. We evaluated the proposed method in experiments, comparing it with fixed impedance control and nonoptimal variable impedance control. Results demonstrate stable interactions even in the presence of conflicting motion plans, and superior performance relative to these two alternative methods.

Humans activate muscles to shape the mechanical interaction with their environment, but can they harness this control mechanism to best sense the environment? We investigated how participants adapt their muscle activation to visual and haptic information when tracking a randomly moving target with a robotic interface. The results exhibit a differentiated effect of these sensory modalities, where participants' muscle coactivation increases with the haptic noise and decreases with the visual noise, in apparent contradiction to previous results. These results can be explained when considering muscle spring-like mechanics, where stiffness increases with coactivation to regulate motion guidance. Increasing coactivation to more closely follow the motion plan favors accurate visual over haptic information, while decreasing it filters visual noise and relies more on accurate haptic information. We formulated this active sensing mechanism as the optimization of visuo-haptic information and effort. This optimal information and effort (OIE) model can explain the adaptation of muscle activity to unimodal and multimodal sensory information when interacting with fixed or dynamic environments, or with another human, and can be used to optimize human-robot interaction.

Interactive exploration of unknown objects' properties, such as stiffness, mass, center of mass, friction coefficient, and shape, is crucial for autonomous robotic systems operating in unstructured environments. Precise identification of these properties is essential for stable and controlled object manipulation and for anticipating the outcomes of (prehensile or nonprehensile) manipulation actions, such as pushing, pulling, and lifting. Our study focuses on autonomously inferring the physical properties of a diverse set of homogeneous, heterogeneous, and articulated objects using a robotic system equipped with vision and tactile sensors. We propose a novel predictive perception framework to identify object properties by leveraging versatile exploratory actions: nonprehensile pushing and prehensile pulling. A key component of our framework is a novel active shape perception mechanism that seamlessly initiates exploration. In addition, our dual differentiable filtering with graph neural networks learns the object–robot interaction and enables consistent inference of indirectly observable, time-invariant object properties. Finally, we develop a N-step information gain approach to select the most informative actions for efficient learning and inference. Extensive real-robot experiments with planar objects show that our predictive perception framework outperforms state-of-the-art baselines and showcases it in three major applications for object tracking, goal-driven task, and environmental change detection.

While surgical robots, such as the da Vinci Surgical System, have become prevalent in minimally invasive surgery, they are predominantly used by the human operator to directly teleoperate the tools. This paper aims to analyse the different methods of human robot shared control in the surgical domain. We propose a reinforcement learning algorithm, transverse generative adversarial imitation learning (tGAIL), which is employed to train the robot from the expert’s demonstration and show competitive generalization ability compared to inverse reinforcement learning and conventional GAIL. We then propose a priority-changing shared control method to effectively combine the surgeon and robot’s strengths by dynamically adjusting control priority based on the deviation distance. We show that using this method in a supervision framework boosts the performance of the human operator when completing the peg transfer task. By learning from the expert and collaborating with the human during the task, the intelligent agent can help to reduce operation time by 31.7% and the human input by 60.5% compared to direct teleoperation.

The manipulation of instruments under a microscope suffers from physiological tremor and human errors, which are inevitable in long microsurgery interventions. Robotic systems developed in recent years for microsurgery are expensive and not flexible, as they cannot use standard instruments, and need the surgeon to modify their operative skills and strategies. In this paper, we introduce a modular robotic system for microsurgery enabling the surgeon to operate using conventional instruments. Our system was implemented using a commercial Kinova robot and a dedicated modular end-effector that uses standard microsurgery instruments. An initial teleoperation validation was carried out by eleven participants, who could successfully control the microsurgery tools to perform basic surgical movements. Furthermore, participants performed a simple anastomosis task with the robot and compared it to manual control. The results showed that robotic control is superior to manual control in simple surgical tasks and the converse in complex tasks. Participants preferred the proposed robotic system due to its user-friendliness and effort reduction.

The utilization of shared control in the realm of surgical robotics augments precision and safety by amalgamating human expertise with autonomous assistance. This paper proposes a user-centered shared control framework enabling a robot to learn from expert demonstration, predict operators’ intent and modulate control authority to provide natural assistance when needed. We employ deep inverse reinforcement learning (IRL) to enable the robot to learn path planning from expert demonstrations with fast convergence, subsequently enhancing the policy with a potential field method. The control authority is allocated seamlessly between the human operator and the autonomous agent based on the prediction of operators’ movement from an adaptive filter and fuzzy logic inference. The proposed method is executed using the da Vinci Research Kit (dVRK) robot in a simulation environment, and its effectiveness is assessed through user performance evaluation in a trajectory tracking task. Compared to direct control and simple shared control, the proposed shared control scheme exhibits superior tracking accuracy and trajectory smoothness under external disturbances. Subjective responses underscore users’ perception of the method’s efficacy in enhancing their performance.

Physically connected humans have been shown to exploit the exchange of haptic forces and tactile information to improve their performance in joint action tasks. As human interactions are increasingly mediated through robots and networks it is important to understand the impact that network features such as lag and noise may have on human behaviour. In this paper, we investigated interaction with a human-like robot controller that provides similar haptic communication behaviour as human-human interaction and examined the influence and compensation mechanisms for delay and noise on haptic communication. The results of our experiments show that participants can perceive a difference between noise and delay, and make use of compensation mechanisms to preserve performance in both cases. However, while noise is compensated for by increasing co-contraction, delay compensation could not be explained by this strategy. Instead, computational modelling suggested that a distinct mechanism is used to compensate for the delay and yield an efficient haptic communication.

Many surgical tasks require three or more tools working together, where a hands-free interface could extend a surgeon's actions to control a third surgical tool. However, most current interfaces do not allow skilled control of grasping critical to robotic manipulation. Here we first present a systematic study to identify efficient and intuitive interaction strategies to control grasping of a surgical tool. A series of experiments were conducted to evaluate six foot pressure-based gestures. Based on the results, three modular novel foot-machine interfaces were developed, which can be integrated with other motion control interfaces. The identified interaction strategies were implemented to control a laparoscopic tool in a surgical simulator, and evaluated in a user study. The results illustrate how naive participants can operate grasping yielding smooth and pick & place operation.

For robotic systems to interact with objects in dynamic environments, it is essential to perceive the physical properties of the objects such as shape, friction coefficient, mass, center of mass, and inertia. This not only eases selecting manipulation action but also ensures the task is performed as desired. However, estimating the physical properties of especially novel objects is a challenging problem, using either vision or tactile sensing. In this work, we propose a novel framework to estimate key object parameters using non-prehensile manipulation using vision and tactile sensing. Our proposed active dual differentiable filtering (ADDF) approach as part of our framework learns the object-robot interaction during non-prehensile object push to infer the object's parameters. Our proposed method enables the robotic system to employ vision and tactile information to interactively explore a novel object via non-prehensile object push. The novel proposed$N$-step active formulation within the differentiable filtering facilitates efficient learning of the object-robot interaction model and during inference by selecting the next best exploratory push actions (where to push? and how to push?). We extensively evaluated our framework in simulation and real-robotic scenarios, yielding superior performance to the state-of-the-art baseline.

Extending the human field of view could enhance our ability to perceive our surroundings thereby improving user safety and enabling us to perform complex manipulation tasks in industrial assembly processes. We investigated the augmentation of environment perception through a study that systematically measured the performance and perception arising from the use of a third eye placed on the back of the user’s head in virtual reality. 28 participants were asked to conduct two goal-oriented tasks: One requiring the identification of targets that discretely appeared in a set of predefined locations; the other the catching of a continuously moving target. The results show that participants were able to incorporate the additional visual information in real time resulting in changes of their motion behaviour. These changes led to localised improvements in task performance for the discrete target task and more efficient motion for both tasks. Participants also showed a strong preference for performing the tasks with the third eye and perceived no accompanying increase in cognitive load.

Previous works have developed impedance control to increase safety and improve performance in contact tasks, where the robot is in physical interaction with either an environment or a human user. This article investigates impedance learning for a robot guided by a human user while interacting with an unknown environment. We develop automatic adaptation of robot impedance parameters to reduce the effort required to guide the robot through the environment, while guaranteeing interaction stability. For nonrepetitive tasks, this novel adaptive controller can attenuate disturbances by learning appropriate robot impedance. Implemented as an iterative learning controller, it can compensate for position dependent disturbances in repeated movements. Experiments demonstrate that the robot controller can, in both repetitive and nonrepetitive tasks: first, identify and compensate for the interaction, second, ensure both contact stability (with reduced tracking error) and maneuverability (with less driving effort of the human user) in contact with real environments, and third, is superior to previous velocity-based impedance adaptation control methods.

Research in joint action focuses on the psychological, neurological, and physical mechanisms by which humans collabo-rate with other agents, and overlaps with several domains related to human-robot interaction. The development of artificial systems that can support or emulate the requisite aspects of joint action could lead to improved human-robot team performance as well as improvements in subjective metrics (e.g., trust). This workshop highlights theoretical and technical considerations about human-robot joint action and real-time adaptation, with a particular focus on socio-motor entrainment, showing how the emulation of psychological mechanisms (e.g., emotion, intention signaling, mirroring) can lead to improved performance. We will invite speakers with backgrounds in robotics, neuroscience and psychol-ogy, as well as speakers with a focus in adjacent works, such as in human-robot coordinated dance, alignment, or synchronization. We will call for papers that utilize the theory of joint-action in an interactive human-robot context. We will also call for position papers on the application of the theory of joint action to robotics, with a heavy focus on psychological mechanisms that could potentially be emulated or adapted to a human-robot context. Participants will have the opportunity to brainstorm considerations and techniques that would be applicable to joint action inspired works through breakout sessions with the aim to lead to new and improved collaborations across fields.

This paper presents a wearable device for finger assessment that can identify finger joint impedance parameters through harmonic oscillation perturbations. This device is designed to help assess motor impairments related to hypertonic soft-tissue changes, that can arise from a number of conditions such as stroke. By measuring the ratio of the applied torque and resulting velocities, the impedance values for any bending direction of a metacarpophalangeal (MCP) joint can be estimated. The ability of this device to effectively estimate finger parameters was tested in experiments with six participants. The experimental result was validated through comparison to prior works on finger impedance estimation. The user experience of the presented system was also analysed, indicating that the device design is comfortable and acceptable for participants.

Supernumerary robotic limbs can act as intelligent prostheses or augment the motion of healthy people to achieve actions which are not possible with only two natural hands. However, as trimanual control is not typical in everyday activities, it is still unknown how different training could influence its acquisition. We conducted an experimental study to evaluate the impact of different forms of trimanual action on training. Two groups of twelve subjects were each trained in virtual reality for five weeks using either a three independent goals task or one dependent goal task. The success of their training was then evaluated by comparing their task performance and motion characteristics between sessions. The results show that subjects dramatically improved their trimanual task performance as a result of training. However, while they showed improved motion efficiency and reduced workload for tasks with multiple independent goals with practice, no such improvement was observed when they trained with the one coordinated goal task.

Many teleoperation tasks require three or more tools working together, which need the cooperation of multiple operators. The effectiveness of such schemes may be limited by communication issues between individuals. Trimanipulation by a single operator using an artificial third arm controlled together with their natural arms may address this issue. Foot-controlled interfaces have previously shown the capability to be used for the continuous control of robot arms. However, the use of such interfaces for controlling a supernumerary robotic limb in coordination with the natural limbs is not well understood. In this paper, a teleoperation task imitating physically-coupled hands in a virtual reality scene was conducted with 14 subjects to evaluate human performance during trimanipulation. The participants were required to move three limbs together in a coordinated way mimicking three arms holding a shared physical object. It was found that after a short practice session, three-hand trimanipulation with a single subject’s hands and foot was still slower than dyad operation. However, they displayed similar performance in their success rate and higher motion efficiency than two people cooperating.

Human movements with or without vision exhibit timing (i.e. speed and duration) and variability characteristics which are not well captured by existing computational models. Here, we introduce a stochastic optimal feedforward-feedback control (SFFC) model that can predict the nominal timing and trial-by-trial variability of self-paced arm reaching movements carried out with or without online visual feedback of the hand. In SFFC, movement timing results from the minimization of the intrinsic factors of effort and variance due to constant and signal-dependent motor noise, and movement variability depends on the integration of visual feedback. Reaching arm movements data are used to examine the effect of online vision on movement timing and variability, and test the model. This modelling suggests that the central nervous system predicts the effects of sensorimotor noise to generate an optimal feedforward motor command, and triggers optimal feedback corrections to task-related errors based on the available limb state estimate.

It is widely acknowledged that drivers should remain in the control loop before automated vehicles completely meet real-world operational conditions. This paper presents an “indirect shared control” framework for steer-by-wire vehicles, which allows the control authority to be continuously shared between the driver and automation through an weighted-input-summation method. A “best-response” driver steering model based on model predictive control (MPC) for indirect shared control is proposed. Unlike any conventional driver model for manual driving, this model assumes that drivers can learn and incorporate the controller strategy into their internal model for predictive path following. The analytic solution to the driver model is provided to enable off-line simulations. A driving-simulator experiment was conducted to demonstrate the advantages of the indirect shared control system in a highway lane-keeping task. The result showed that the proposed indirect shared control method was effective to improve the subjects’ lane-keeping performance and reduce steering control effort. The proposed driver steering model was also validated by the experiment data, which produced a smaller prediction error than the conventional MPC driver model.

In human-robot collaborative transportation and sawing tasks, the human operator physically interacts with the robot and directs the robot's movement by applying an interaction force. The robot needs to update its control strategy to adapt to the interaction with the human and to minimize the interaction force. To this end, we propose an integrated algorithm of robot's trajectory adaptation and adaptive impedance control to minimize the interaction force in physical humanrobot interaction (pHRI) and to guarantee the performance of the collaboration tasks. We firstly utilize the information of the interaction force to regulate the robot's reference trajectory. Then, an adaptive impedance controller is developed to ensure automatic adaptation of the robot's impedance parameters. While one can reduce the interaction force by using either trajectory adaptation or adaptive impedance control, we investigate the task performance when combining both. Experimental results on a planar robotic platform verify the effectiveness of the proposed method.

Video games have proved very valuable in rehabilitation technologies. They guide therapy and keep patients engaged and motivated. However, in order to realize their full potential, a good understanding is required of the players’ motor control. In particular, little is known regarding player behaviour in tasks demanding bimanual interaction. In this work, an experiment was designed to improve the understanding of such tasks. A driving game was developed in which players were asked to guide a differential wheeled robot (depicted as a rocket) along a trajectory. The rocket could be manipulated by using an Xbox controller’s triggers, each supplying torque to the corresponding side of the robot. Such a task is redundant, i.e. there exists an infinite number of input combinations to yield a given outcome. This allows the player to strategize according to their own preference. 10 participants were recruited to play this game and their input data was logged for subsequent analysis. Two different motor strategies were identified: an "intermittent" input pattern versus a "continuous" one. It is hypothesized that the choice of behaviour depends on motor skill and minimization of effort and error. Further testing is necessary to determine the exact relationship between these aspects.

Sensory feedback from touch is critical for many tasks carried out by robots and humans, such as grasping objects or identifying materials. Electronic skin (e-skin) is a crucial technology for these purposes. Artificial tactile skin that can play the roles of human skin remains a distant possibility because of hard issues in resilience, manufacturing, mechanics, sensorics, electronics, energetics, information processing, and transport. Taken together, these issues make it difficult to bestow robots, or prosthetic devices, with effective tactile skins. Nonetheless, progress over the past few years in relation with the above issues has been encouraging, and we have achieved close to providing some of the abilities of biological skin with the advent of deformable sensors and flexible electronics. The naive imitation of skin morphology and sensing an impoverished set of mechanical and thermal quantities are not sufficient. There is a need to find more efficient ways to extract tactile information from mechanical contact than those previously available. Renewed interest in neuromorphic tactile skin is expected to bring some fresh ideas in this field. This article reviews these new developments, particularly related to the handling of tactile data, energy autonomy, and large-area manufacturing. The challenges in relation with these advances for tactile sensing and haptics in robotics and prosthetics are discussed along with potential solutions.

To move a hard table together, humans may coordinate by following the dominant partner's motion [1-4], but this strategy is unsuitable for a soft mattress where the perceived forces are small. How do partners readily coordinate in such differing interaction dynamics? To address this, we investigated how pairs tracked a target using flexion-extension of their wrists, which were coupled by a hard, medium or soft virtual elastic band. Tracking performance monotonically increased with a stiffer band for the worse partner, who had higher tracking error, at the cost of the skilled partner's muscular effort. This suggests that the worse partner followed the skilled one's lead, but simulations show that the results are better explained by a model where partners share movement goals through the forces, whilst the coupling dynamics determine the capacity of communicable information. This model elucidates the versatile mechanism by which humans can coordinate during both hard and soft physical interactions to ensure maximum performance with minimal effort.

Humans can skilfully use tools and interact with the environment by adapting their movement trajectory, contact force, and impedance. Motivated by the human versatility, we develop here a robot controller that concurrently adapts feedforward force, impedance, and reference trajectory when interacting with an unknown environment. In particular, the robot's reference trajectory is adapted to limit the interaction force and maintain it at a desired level, while feedforward force and impedance adaptation compensates for the interaction with the environment. An analysis of the interaction dynamics using Lyapunov theory yields the conditions for convergence of the closed-loop interaction mediated by this controller. Simulations exhibit adaptive properties similar to human motor adaptation. The implementation of this controller for typical interaction tasks including drilling, cutting, and haptic exploration shows that this controller can outperform conventional controllers in contact tooling.

Shared control is an important approach to avoid the driver-out-of-the-loop problems brought by imperfect autonomous driving. Steer-by-wire technology allows the mechanical decoupling between the steering wheel and the road wheels. On steer-by-wire vehicles, the automation can join the control loop by correcting the driver steering input, which forms a new paradigm of shared control. The new framework, under which the driver indirectly controls the vehicle through the automation's input transformation, is called indirect shared control. This paper presents an indirect shared control system, which realizes the dynamic control authority allocation with respect to the driver's authority intention. The simulation results demonstrate the effectiveness and benefits of the proposed control authority adaptation method.

The studies on bio-mechatronics systems and applications have been carried out for more than three decades, to overcome the challenges raised from both theoretical and experimental sides, especially those posed by the application of mechatronics and robotics in healthcare and medical fields. The research on biomedical robotics and bio-mechatronics covers a diverse spectrum of rapid rising interdisciplinary areas including bio-inspired robots for industrial, military, medical, and rehabilitation applications. This special issue aims at showcasing the most exciting and recent advances in the application of robotics and mechatronics in various fields and brings together a broad spectrum of topics covering various definition, development, control, and deployment of bio-mechatronics/robot systems, including social robots, wearable robot systems such as exoskeleton, rehabilitation robot, tele-robot, and a numbers of systems engineering approaches such as modeling, optimization and control. This special issue is to give analysis to the biological systems from a “bio-mechatronic” point of view, and to investigate the engineering and scientific principles behind their remarkable performance. High-quality original papers of innovative ideas and concepts have been included in the special issue of biomedical robotics and bio-mechatronics systems and application. While the design and development of bio-inspired machines and systems with novel and high performance in various applications have been investigated as well. The recent development of multidisciplinary research shall contribute to the promotion of the research on biomedical robotics and bio-mechatronics systems and application, with application to transportation, diagnosis, surgery, assistive technology, prosthetics, personal assistance, rehabilitation, health care, in laboratory, hospital, and the real world.

This paper presents the development and evaluation of an MR compatible haptic interface for human motor control studies, which can be easily installed and removed from the scanner room. The interface is actuated by a powerful shielded DC motor located 2.1 m away from the 3T MR scanner. Rotational movements are transmitted to a subject's wrist through preloaded cable transmission which drives the handle unit. The handle of the interface is designed to be adjustable to different hands size, enabling comfortable and natural wrist movements. The nominal achievable wrist torque of the interface is up to 2Nm. The interface is easily transportable due to its design characteristics. A dynamic model of the interface is presented and identified for position and torque control modes. Phantom MR compatibility test in clinical environment showed that the interface is compatible with strong magnetic field and radio frequency emission and its operation does not affect the quality of MR images.

Interactive technologies can help people acquire movement skills, and one way is by using visual distortions to boost neural adaptation. An extreme version of such approach is to train a movement without moving by creating a synesthetic illusion of movement - displaying virtual motions when there is none. While this approach uses no proprioceptive error to drive adaptation, our results show encouraging evidence that motor skills can be acquired through such illusions of movement.

Current robotic rehabilitation devices have a high cost-to-benefit ratio, which prevents their large scale adoption by the clinical rehabilitation community. This paper first presents H-Man, a low cost planar robot, as a quantitative assessment and training tool. This is followed by a preliminary study to investigate baseline performance measures for motor assessment during reaching tasks as a step toward replacing conventional ordinal scales with continuous quantitative scales. Thirteen healthy and one participant with upper limb motor impairment participated in the study and performed reaching tasks with their dominant and non-dominant hands in three directions. The results from healthy subjects indicate no significant difference between different directions for both limbs and also between corresponding directions of dominant and non-dominant limbs (p > 0.05, all cases). However, differences in measures can be observed for the impaired subject.

This paper introduces an inexpensive prosthetic hand control system designed to reduce the cognitive burden on amputees. It is designed around a vision-based object recognition system with an embedded camera that automates grasp selection and switching, and an inexpensive mechanomyography (MMG) sensor for hand opening and closing. A prototype has been developed and implemented to select between two different grasp configurations for the Bebionic V2 hand, developed by RSLSteeper. Pick and place experiments on 6 different objects in `Power' and `Pinch' grasps were used to assess feasibility on which to base full system development. Experimentation demonstrated an overall accuracy of 84.4% for grasp selection between pairs of objects. The results showed that it was more difficult to classify larger objects due to their size relative to the camera resolution. The grasping task became more accurate with time, indicating learning capability when estimating the position and trajectory of the hand for correct grasp selection; however further experimentation is required to form a conclusion. The limitation of this involves the use of unnatural reaching trajectories for correct grasp selection. The success in basic experimentation provides the proof of concept required for further system development.

This paper describes the reachMAN2, a rehabilitation robot with minimum degrees-of-freedom to train arm reaching and manipulation for typical ADLs, providing assistance in arm flexion/extension, forearm supination/pronation and hand opening/closing. The design, safety, control and performance evaluation of the system are presented. A handle using an innovative cam mechanism enables natural hand opening/closing movements. The interaction force between the device and the subject is measured via four force sensors located on the handle of the device. A preliminary study with healthy subjects was performed to assess the performances of the device.

This work investigates how to design a comfortable wrist exoskeleton which complies with the natural coordination mechanisms in the redundant wrist. Human sensorimotor control is known to impose intrinsic kinematic constraints to solve redundant motor tasks. To this end, the effect of an exoskeleton on natural motor strategies was assessed during pointing tasks performed with the wrist. The exoskeleton was designed based on the kinematic model of one specific subject. Then wrist orientation was observed during pointing tasks with the exoskeleton in the following conditions: i) optimal alignment between human and exoskeleton joints; ii) varying degrees of misalignment between human and exoskeleton joints; iii) optimal alignment while the PS axis was locked (i.e. no redundancy). The results exhibited a modification of the natural coordination mechanisms characterized by a subject-specific Koenderink shape index. Kruskal-Wallis pairwise analyses demonstrated a significant variation between test conditions indicating a change of intrinsic constraints with misalignment and locked PS. The assessment methodologies presented in this paper can be used to test for ergonomic constraints and can guide the design of robotic systems performing kinematically redundant tasks.

This paper develops a learning control algorithm adapting the reference point and force to interact with an object of unknown geometry and elasticity. The controller is inspired by neuroscience studies that investigated the neural mechanisms when human adapt to virtual objects of different properties. The learning control algorithm estimates the shape and stiffness of the given object while maintaining a specified contact force with the environment. Simulations demonstrate the efficiency of the algorithm to identify the geometry and impedance of an unknown object without requiring force sensing. These properties are attractive for robotic haptic exploration with little demand on the sensing.

This work introduces a coordinate-independent method to analyse movement variability of tasks performed with hand-held tools, such as a pen or a surgical scalpel. We extend the classical uncontrolled manifold (UCM) approach by exploiting the geometry of rigid body motions, used to describe tool configurations. In particular, we analyse variability during a static pointing task with a hand-held tool, where subjects are asked to keep the tool tip in steady contact with another object. In this case the tool is redundant with respect to the task, as subjects control position/orientation of the tool, i.e. 6 degrees-of-freedom (dof), to maintain the tool tip position (3dof) steady. To test the new method, subjects performed a pointing task with and without arm support. The additional dof introduced in the unsupported condition, injecting more variability into the system, represented a resource to minimise variability in the task space via coordinated motion. The results show that all of the seven subjects channeled more variability along directions not directly affecting the task (UCM), consistent with previous literature but now shown in a coordinate-independent way. Variability in the unsupported condition was only slightly larger at the endpoint but much larger in the UCM.

This article presents a versatile controller that enables various contact tooling tasks with minimal prior knowledge of the tooled surface. The controller is derived from results of neuroscience studies that investigated the neural mechanisms utilized by humans to control and learn complex interactions with the environment. We demonstrate here the versatility of this controller in simulations of cutting, drilling and surface exploration tasks, which would normally require different control paradigms. We also present results on the exploration of an unknown surface with a 7-DOF manipulator, where the robot builds a 3D surface map of the surface profile and texture while applying constant force during motion. Our controller provides a unified control framework encompassing behaviors expected from the different specialized control paradigms like position control, force control and impedance control.

Most of today's robots have rigid structures and actuators requiring complex software control algorithms and sophisticated sensor systems in order to behave in a compliant and safe way adapted to contact with unknown environments and humans. By studying and constructing variable impedance actuators and their control, we contribute to the development of actuation units which can match the intrinsic safety, motion performance and energy efficiency of biological systems and in particular the human. As such, this may lead to a new generation of robots that can co-exist and co-operate with people and get closer to the human manipulation and locomotion performance than is possible with current robots.

Our goal is to investigate the mechanisms of human-human interaction in collaborative and competitive tasks. In this regard, previous groups have suggested simplified motor tasks involving dual robotic interfaces. However, these interfaces involved arm movements with a high degree of kinematic and muscle redundancy. This paper introduces a simple, yet versatile dual-wrist robotic interface, Hi5, that allows us to investigate the motor processes behind human-human and bimanual interaction control, and avoid confounds inherent of arm movements. This paper presents the design of Hi5 and the implemented safety measures. Performance tests then demonstrate its capacity to yield high dynamics, and exhibit low values for inertia and friction. Preliminary experiments show how Hi5 allows us to analyze the specific kinematic, torque and muscle activation patterns of each partner and disambiguate their roles in force and impedance control.

This paper introduces and analyzes an algorithm for adaptation of the reference trajectory of a human or robot arm interacting with a novel environment. The algorithm, based on the minimization of interaction force and performance error by satisfying a desired impedance, yields a mathematically rigorous model of the underlying mechanism of motion planning adaptation in humans. Simulations demonstrate a decrease of the interaction force to a limited amount as well as identification of the unknown interaction surface shape. These properties are attractive for adaptive motion of robots interacting with unknown surfaces, providing a robust behavior with little demand on the sensing.

This paper presents a novel human-like learning controller to interact with unknown environments. Strictly derived from the minimization of instability, motion error, and effort, the controller compensates for the disturbance in the environment in interaction tasks by adapting feedforward force and impedance. In contrast with conventional learning controllers, the new controller can deal with unstable situations that are typical of tool use and gradually acquire a desired stability margin. Simulations show that this controller is a good model of human motor adaptation. Robotic implementations further demonstrate its capabilities to optimally adapt interaction with dynamic environments and humans in joint torque controlled robots and variable impedance actuators, without requiring interaction force sensing.

Human motor control computational model is an important component in the study and the successful realisation of human-robot interaction. In this paper, the Operational Space Formulation is presented as a suitable framework of human motor control computational model based on the Equilibrium Point Hypothesis (EPH) approach. The iterative adaptive control strategy was incorporated to simulate human motor adaptation to different tasks. The strategy involves the use of an Equilibrium Model which represents the ideal human motor response to a given task. The combined strategy was simulated to match a set of data gathered experimentally from several human subjects. The results were observed to explain many of the features found in the recorded behaviours in the EPH-based approach of human motor modelling.

Interaction of a robot with dynamic environments would require continuous adaptation of force and impedance, which is generally not available in current robot systems. In contrast, humans learn novel task dynamics with appropriate force and impedance through the concurrent minimization of error and energy, and exhibit the ability to modify movement trajectory to comply with obstacles and minimize forces. This article develops a similar automatic motor behavior for a robot and reports experiments with a one degree-of-freedom system. In a postural control task, the robot automatically adapts torque to counter a slow disturbance and shifts to increasing its stiffness when the disturbance increases in frequency. In the presence of rigid obstacles, it refrains from increasing force excessively, and relaxes gradually to follow the obstacle, but comes back to the desired state when the obstacle is removed. A trajectory tracking task demonstrates that the robot is able to adapt to different loads during motion. On introduction of a new load, it increases its stiffness to adapt to the load quickly, and then relaxes once the adaptation is complete. Furthermore, in the presence of an obstacle, the robot adjusts its trajectory to go around it.

This video presents a fully automated desktop workstation for fabricating tissue engineering (TE) scaffolds by assembling microscopic building blocks of dimension 0.5 × 0.5 × 0.2mm3and 60μm thickness. A TE scaffold is a porous supporting structure made of biodegradable material for cells to attach to and proliferate. TE scaffolds fabricated this way enable 3D control of the cells and agents to promote optimal tissue growth. This video outlines the architecture and control strategies of the workstation, and demonstrates its efficiency. Visual and force feedbacks, as well as a haptic exploration, make the assembly robust to large relative errors in the microparts.

Robot-assisted rehabilitation has only recently begun to be applied to improvement of hand function after stroke. In a preliminary study, involving 4 post-stroke subjects, more than 2 years following the stroke, we have been able to show that 8 weeks of robot-assisted training leads to changes in patterns of arm and finger muscle activation. The patterns were quantified in terms of synchronous muscle synergies which allowed for comparison with muscle activation patterns of healthy age-matched subjects. We found that the muscle synergies of the post-stroke subjects became more similar to those of the healthy subject group following training.

This study examines effects of learning 3D micromanipulation in an unstable dynamic environment. A test group trained in an unstable divergent force field while a control group trained the movement in the null force field. The subjects in the test group increased the success rate, in contrast to the control group which had similar rate after training. The error and its standard deviation decreased in the test group but not in the control group. In summary, training in unstable dynamics enable subjects to become more accurate, in contrast to training using only visual feedback.

The systems which have been developed are safe, easily transportable and offer various training possibilities. Virtual-reality games have been implemented to increase motivation during the 8 weeks of training with post-stroke subjects. Clinical trials have shown that chronic stroke patients can perform the required tasks and that training improves their motor function.

Robotic devices able to train both reaching and manipulation are often large and complex and thus not suitable for decentralized use at home or in local rehabilitation centers. This paper describes a compact device with only three degrees of freedom (DOF) to train reaching and manipulation critical to activities of daily living. The design considers only the DOF necessary to train tasks such as pick-and-place of objects, drinking, eating and knob manipulation, based on low-dimensional synergies used in these tasks. Specifications from measured biomechanical parameters yield safety and suitable performance. A prototype demonstrates some of the resulting functions and therapeutic possibilities offered by this design.

When coordinating movements, the nervous system often has to decide how to distribute work across a number of redundant effectors. Here, we show that humans solve this problem by trying to minimize both the variability of motor output and the effort involved. In previous studies that investigated the temporal shape of movements, these two selective pressures, despite having very different theoretical implications, could not be distinguished; because noise in the motor system increases with the motor commands, minimization of effort or variability leads to very similar predictions. When multiple effectors with different noise and effort characteristics have to be combined, however, these two cost terms can be dissociated. Here, we measure the importance of variability and effort in coordination by studying how humans share force production between two fingers. To capture variability, we identified the coefficient of variation of the index and little fingers. For effort, we used the sum of squared forces and the sum of squared forces normalized by the maximum strength of each effector. These terms were then used to predict the optimal force distribution for a task in which participants had to produce a target total force of 4-16 N, by pressing onto two isometric transducers using different combinations of fingers. By comparing the predicted distribution across fingers to the actual distribution chosen by participants, we were able to estimate the relative importance of variability and effort of 1:7, with the unnormalized effort being most important. Our results indicate that the nervous system uses multi-effector redundancy to minimize both the variability of the produced output and effort, although effort costs clearly outweighed variability costs.

This paper presents a novel technique to fabricate scaffold/cell constructs for tissue engineering by robotic assembly of microscopic building blocks (of volume 0.50.50.2 and 60 thickness). In this way, it becomes possible to build scaffolds with freedom in the design of architecture, surface morphology, and chemistry. Biocompatible microparts with complex 3-D shapes were first designed and mass produced using MEMS techniques. Semi-automatic assembly was then realized using a robotic workstation with four degrees of freedom integrating a dedicated microgripper and two optical microscopes. Coarse movement of the gripper is determined by pattern matching in the microscopes images, while the operator controls fine positioning and accurate insertion of the microparts. Successful microassembly was demonstrated using SU-8 and acrylic resin microparts. Taking advantage of parts distortion and adhesion forces, which dominate at micro-level, the parts cleave together after assembly. In contrast to many current scaffold fabrication techniques, no heat, pressure, electrical effect, or toxic chemical reaction is involved, a critical condition for creating scaffolds with biological agents.

Manipulation tasks performed under microscopes are fatigue inducing on account of the difficulty in visualisation of the scene and hand-eye coordination. This work concentrates on providing assistance to the operator in the form of a visual-haptic interactive system which allows intuitive command of the manipulator motion axes. Specific examples of visualisation tools are presented which compensate for problems such as: spatial d is orientation, loss of depth perception, and occluded data.

This paper presents an analysis of a newly developed elastic path controller (EPC) that can be used in assistive devices with human operators. The technique of robust control is also utilized to determine the parameters of the EPC in order to ensure that the EPC can tolerate possible uncertainties or disturbances in the environment. The new EPC is applied to a robotic wheelchair and tested in simulation. The results demonstrate the efficacy of the EPC.

This paper presents a novel type of Elastic Path Controller (EPC) to control and manipulate the assistive devices such as the Collaborative Wheelchair Assistant (CWA). Compared to previous controllers such as Samson’s and Brent’s controller, the proposed EPC is safer and more maneuverable. This paper also proposes the handling of singularity and solutions to backward motion problems of the EPC. Simulation experiments show that the new EPC can fulfill the tasks of path following, obstacle avoiding, singularity handling, backward motion, and so on.

This paper investigates how collaborative robots (cobots) can assist a human by mechanically constraining motion to software-defined guide paths, and introduces simple and efficient tools to design ergonomic paths. Analysis of the movements of seven subjects with the Scooter cobot reveals significant differences between guided movements (GM) and free movements (FM). While FM requires learning for each novel task, movements in GM are satisfying from the first trial, require little effort, are faster, smoother, and with fewer back and forth corrections than in FM. Operators rely on path guidance to rotate the Scooter and direct it along curved trajectories. While these advantages demonstrate the strength of the cobot concept, they do not show how guide paths should be defined. We introduce tools to enable the cobot and its operator to collaboratively learn ergonomic guide paths and adapt to changes in the environment. By relying on the haptic sensing, vision, and planning capabilities of the human operator, we can avoid equipping the cobot with complex sensor processing. Experiments with human subjects demonstrate the efficiency and complementarity of these guide paths design tools

This paper describes a two-degrees-of-freedom haptic interface to investigate the brain mechanisms of human motor control, which is capable of safely and gently interacting with human arm motion during functional magnetic resonance imaging (fMRI). A hydrostatic transmission separates the interface into a master and an MR compatible slave system, allowing the placement of all interfering components outside the electromagnetic shield of the MR room. The transmission mirrors force and motion of the master actuators on the slave system placed close to the MR scanner. The parallel architecture takes advantage of the linear MR compatible actuators and allows human subjects to perform reaching movements comfortably in the small workspace limited by the dimensions of the MR scanner and the biomechanics of the arm. The kinematic structure of the slave interface was optimized with respect to the available space and types of movements to be investigated. Materials were chosen based on their MR compatibility, their stiffness and weight. The interaction force with the subject is measured over two optical force sensors, located close to the output of the interface. Two shielded optoelectronic encoders measure the extension of the slave hydraulic pistons. Detailed tests demonstrated the fMRI compatibility even during movement of the interface

Functional brain exploration methodologies such as functional magnetic resonance imaging (fMRI) are critical tools to study perceptual and cognitive processes. In order to develop complex and well-controlled fMRI paradigms, researchers are interested in using active interfaces with electrically powered actuators and sensors. Due to the particularity of the MR environment, safety and compatibility criteria have to be strictly followed to avoid risks to the subject under test, the operators or the environment, as well as to prevent artifacts in the images. This paper describes the design of an fMRI compatible mechatronic interface based on MR compatibility tests of materials and actuators. In particular, a statistical test is introduced to evaluate the presence of artifacts in the image sequences that could negatively affect the fMRI studies. The device with two degrees of freedom, allowing one translation with position-feedback along a horizontal axis and one rotation about a vertical axis linked to the translation, was realized to investigate the brain mechanisms of dynamic tactile perception tasks. It can be used to move and orient various objects below the finger for controlled tactile stimulation. The MR compatibility of the complete interface is shown using the statistical test as well as a functional study with a human subject

An elastic path controller (EPC) is developed in this paper to add "elasticity" to a path following controller and enable dynamic modification of the guiding path for assistive devices based on path guidance. This permits the users to compensate for changes in the environment such as obstacles or error in position sensing. The EPC is demonstrated both on the Scooter cobot and on the collaborative wheelchair assistant (CWA). Simulation results and psychophysical experiments performed with the Scooter suggest that this novel tool is efficient and can help human operators to adapt to environment changes

This paper describes the development and assessment of a collaborative wheelchair assistant (CWA). The concept at the heart of the CWA is to rely on the user's motion planning skills and assist the maneuvering with path guidance. The user decides where to go and controls the speed (including start and stop), while the system guides the wheelchair along software-defined paths. An intuitive path editor allows the user to modify the path on-line and so avoid dangers or obstacles. By using the human sensory and planning systems, complex sensor processing and artificial decision systems are not needed, making the system safe, simple and low-cost. Experimental results demonstrate this concept

Using haptic interfaces in combination with functional magnetic resonance imaging (fMRI) could lead to important insights into the brain mechanisms of human motor control and related dysfunctions. However, in addition to the usual requirements for haptic interfaces (e.g. smooth force control, back-drivability, low friction and inertia) these devices must also be MR safe and MR compatible. Previous MR-compatible actuation methods for force-feedback present drawbacks with respect to conventional haptic interfaces. Here, we present a novel MR-compatible actuator designed especially for impedance control and to meet the requirements for haptic interfaces. It consists of an ultrasonic motor controlled in speed combined with an electrorheological fluid brake which modulates the output torque over a differential gear. The entire system is integrated into a compact housing with an encoder at the output

The strong impairment of motor functions in stroke survivors affects daily activities such as eating, manipulating objects or writing. Our goal is to induce long lasting improvements in such tasks by having patients perform systematic exercises using haptic interfaces. This paper describes a novel two-degrees-of-freedom interface which we have developed to help stroke patients gradually recover the ability to open and close the hand and manipulate knobs. Different solutions are studied and a design consisting of two parallelogram structures interacting with the fingers is proposed. The mechanical design offers the possibility to adapt the interface to various hand sizes and finger orientations, and to right or left-handed subjects. Design kinematics as well as actuation and system control are described. Several knobs are proposed to interact with patients, especially a cone mechanism to train a complete opening movement from a strongly contracted and closed hand to a large opened position. The interaction force with the subject is measured over four force sensors located close to the output of the interface. A preliminary study has been performed to evaluate the performances of the haptic interface

We analyze the dynamics of an MR-compatible hydrostatic transmission designed to transfer power over distances of up to 10 m. In this system, a master actuates a passive slave connected by two hydrostatic lines in a cyclic arrangement. We derive a nonlinear model of this system and use it to analyze the system's behavior and the design parameters. The transmission acts as a low-pass filter with cut-off frequency decreasing for longer hoses. Even for a length of 10 m the cut-off frequency is about 20 Hz, resulting in a bandwidth that suffices for haptic interfaces interacting with human motion as well as for medical robots. A pragmatic control delivered free movements, position and velocity dependent force fields and trajectory control suitable to investigate how the brain controls movements in interaction with the environment. For short hose lengths (/spl les/1 m) the dynamics can be well approximated by a linear model, and the system is dynamic and stiff. The hydraulic transmission can produce force and motion in any orientation, enabling a more flexible design than other types of transmissions such as by cables.

For use in a Virtual Reality based training system for surgical micromanipulation, we have developed a fast multi-scale FEM algorithm that concentrates detail where needed while still handling global deformations. The resulting 6-to-7-fold speed up is promising for the development of real-time simulation of the mechanical response of a virtual organ or tissue. FEM algorithm uses elements from multiple levels in a hierarchy of mesh similar to the progressive mesh. This algorithm has been integrated with a visual/haptic feedback workstation.

This paper presents a monolithic shape memory alloy (SMA) microgripper, which was developed to assemble microscopic building blocks of width 60 /spl mu/m into tissue engineering scaffolds. It consists of two small fingers for grasping, an actuator which changes its shape upon heating by Joule effect and a parallel elastic structure to provide a pullback force on cooling as well as to guide the finger movement. All the elements are laser-cut from the same Ni-Ti-Cu sheet but have different mechanical properties and are used for different functions. Using local laser annealing, a local shape memory effect is introduced on the actuator while leaving other areas in a cold-worked state, i.e. no shape memory effect occurs. The material has nonlinear mechanics and the actuator undergoes large deflections. A numerical method is introduced to compute the nonlinear dynamics of the coupled system formed by the actuator and pullback spring. The predicted deflection and force are in good agreement with the measured ones, and this model can be used to optimize the design.

Force/torque sensors compatible with magnetic resonance imaging (MRI) are required to develop haptic interfaces for neuroscientific investigations and robotic tools for interventional MRI. In this paper, we analyze the mechanical structure of classical and MRI compatible sensors from literature and demonstrate the critical role of the mechanical design on the sensing performance. A simple and efficient torque sensor based on light intensity measurement over optical fibers is introduced, which allows to place the electronic components outside the scanner room. By using a self-guiding flexible structure and optimal mirror placement, the sensitivity to transverse torque is reduced to 0.03% of the desired output torque.

Magnetically compatible robots are required to develop haptic interfaces for neuroscience studies and MRI guided robots for minimally invasive interventions. This paper introduces an MR compatible master slave concept using a hydraulic transmission, and presents an operational robot able to work within an MRI/fMRI scanner and acquire images continuously during motion. It describes a magnetically inert actuator using a direct drive to power the hydraulic circuitry and a modular set of position and force/torque sensors that we have developed. These were integrated into a haptic interface prototype with on rotary degree of freedom, which can be used in conjuction with an fMRI. The MR compatibility was confirmed experimentally, and the performances show a manipulation accuracy of a few micrometers over a range of several centimeters, and forces up to several thousand Newton.

Describes an integrated approach to design and fabricate scaffold/cell constructs for tissue engineering. With this approach it becomes possible to produce scaffolds with controlled distribution of living cells and growth factors, a critical condition for successful grafting. Our idea consists of building a scaffold/cell construct by robotic micro-assembly of microscopic polymer building blocks. The paper introduces the rationale and concept of this interdisciplinary project and presents some realized steps. A 3D contact FEM simulation has been carried out to study the forces involved on the scaffold elements and micro-gripper during assembly. An error analysis has been performed to evaluate the accumulated error when building a scaffold/cell construct. A dedicated monolithic shape memory alloy micro-gripper has been realized and tested, which is able to handle parts in the range of 50-100 microns.

Many wheelchair users have problems in orienting themselves and maneuvering the wheelchair in congested environments. Dependent upon others to push them, they can feel powerless. The collaborative wheelchair assistant introduced in this paper may help these people to regain autonomy by providing motion guidance along software-defined paths corresponding to their ability. It enables the user to travel independently, maneuvering round sharp corners and along narrow passages much more easily without compromising speed or safety. This paper investigates the hardware, kinematics and control of the collaborative wheelchair and presents tools by which the user can easily define guiding paths and personalizes the control to a particular disability. The user can also modify the predefined path 'en route' using an elastic path planner, and so escape from danger using reflexes and anticipation. This low cost assistant can be realized on both manual and powered wheelchairs.

Investigates nonlinear controllers with dynamic compensation depending solely on the desired state and velocity computed from the position signal using a velocity observer. A feedforward controller and a controller similar to the computed torque controller are compared. The stability is proved, and adaptive versions of these two controllers are implemented on a 6 DOF parallel manipulator with uncoupled and highly nonlinear dynamics. Both controllers improved the tracking performance critically relative to a linear controller. We recommend using the feedforward controller, because it is simpler to implement and the difference of performance between both controllers is minor.

In this paper, 2 control algorithms for learning and compensating for the dynamics of manipulators during the motion are presented and tested experimentally. It is shown that the adaptive feedforward controller (AFFC) is well suitable for learning the parameters of the dynamic equation, even, in presence of friction and noise. The resulting control performances are better than with measured parameters for any trajectory in the workspace. When the task consists of driving a repeated trajectory, an adaptive look-up-table MEMory, introduced and analyzed in this paper, is however simpler to implement and results in even better control performances.

In order to identify the dynamic parameters in nonlinear adaptive control the robot's dynamic equation has to be written in a linear form. Many methods have been proposed for serial robots, but for parallel robots, the few solutions proposed so far lead to complicated equations that are not readily usable for real-time implementation. In this paper we propose a new method based on the virtual work principle to find a linear form of the dynamic equation of robots. Compared to other methods, it has the advantage that it does not need to open the closed loop structure into a tree-structure robot. It considers rather each body separately using its Jacobian matrix to project the forces into the joint space of the robot. Thus, simplification can be made at the very beginning of the modeling. This is very efficient when used to model fully parallel robots. As an illustration, the proposed method is applied to the 3dof DELTA parallel robot.

This paper presents the dynamic equation, nonlinear control and dynamic parameters identification of the Hexaglide, a new 6-DOF parallel manipulator intended to be used as a high speed milling machine. Using a method based on the virtual work principle, the dynamic equation is found in a compact linear form and then used in a nonlinear adaptive control algorithm based on the minimization of the tracking error. The dynamic parameters are learned during motion and introduced in an inverse dynamic model used as a feedforward compensator. Specific trajectories, exciting the parameters separately, enable a fast stepwise learning of the 12 parameters. A simulation demonstrates the validity of the approach.

In this paper, a method is presented for the online learning of visually guided movements. The algorithms presented have been tested with a manipulator tracking manoeuvering targets. Three parameters critical for the visuo-motor coordination are learned in less than one hour with repeated movements. After this learning phase, the robot performs smooth and fast reaching movements and can easily drop small objects into the waggon of a moving model train, independently of the trajectory.

This paper presents a mathematical analysis of image processing, algorithms designed according to the results of this analysis, and their implementation. We prove that the search of objects features can be accelerated without loss of precision by using an inhomogeneous density of the sensitive cells the parameters space is composed of. In other words, the visual analysis should be concentrated in the region of the features space around the expected object position. The improvement relative to an uniform cell density is quantified using a cost function corresponding to time and precision optimisation. We show that a Kohonen neural network can be used for efficient image processing, and simulate this strategy. We introduce a simpler algorithm for the case that the object positions are Gauss-distributed around the expected position. This algorithm has been implemented it on a robot guided by a vision system. The robot learned to process images efficiently during the manoeuvres and after that was able to track objects moving in a fast and unpredictable manner.