Tactile interfaces, that can convey information to humans via tactile feedback, are still relatively rare. In this study we present a prototype ‘tactile scanner’, that fixes onto a user’s arm, and using arrays of capacitive sensors and vibratory motors, provides users with a sense of the proximity of objects near their arm. Through two experiments, we show that the device enables users to detect not just the position of objects, but also estimate their shapes and orientations. Finally, in a third experiment, we compare the user accuracy with the tactile scanner and their accuracy with real touch.

Tactile perception on our fingers is a key sensory feedback that enables us to perceive and explore our world using our hands as probes, and is essential for efficient gripping and manipulation of objects. A tactile feedback system can therefore greatly improve the quality of life of individuals with partial or complete sensory loss like during stroke, or with artificial limbs after an amputation. However, most existing tactile texture feedback technologies suffer from two constraints. First, texture decoding and texture feedback have been traditionally examined separately and not as parts of the same problem, and second, texture information has been popularly fed back using sensory modality other than tactile itself. In this study, we propose a prototype on-line direct-texture decoding and feedback system in which the texture touched by a user is decoded using an accelerometer attached to the finger. The feedback is realized by rubbing the user’s skin with the actual material and the speed of the user swipes. The efficacy of the proposed system was tested in two user experiments with five test materials. The results and the corresponding hints for future improvements are discussed.

Human physical assistance requires the assistant to tune both his trajectory and impedance in order to assist an individual as well as be guided by him. In this study we propose a controller for teleoperated human assistance that allows the assistant to guide the assisting robot in both trajectory and impedance. We propose to use the inherent perturbations in the task, induced by the elderly or stroke patient, for impedance estimation, while a simple neuroscience based filter allows the reference estimation of the operator. We tested our impedance estimation and the controller as a whole in two experiments in which a human operator guided a robot suffering force perturbations that simulated a human patient.

A significant problem in brain-computer interface (BCI) research is decoding - obtaining required information from very weak noisy electroencephalograph signals and extracting considerable information from limited data. Traditional intention decoding methods, which obtain information from induced or spontaneous brain activity, have shortcomings in terms of performance, computational expense and usage burden. Here, a new methodology called prediction error decoding was used for motor imagery (MI) detection and compared with direct intention decoding. Galvanic vestibular stimulation (GVS) was used to induce subliminal sensory feedback between the forehead and mastoids without any burden. Prediction errors were generated between the GVS-induced sensory feedback and the MI direction. The corresponding prediction error decoding of the front/back MI task was validated. A test decoding accuracy of 77.83-78.86% (median) was achieved during GVS for every 100[Formula: see text]ms interval. A nonzero weight parameter-based channel screening (WPS) method was proposed to select channels individually and commonly during GVS. When the WPS common-selected mode was compared with the WPS individual-selected mode and a classical channel selection method based on correlation coefficients (CCS), a satisfactory decoding performance of the selected channels was observed. The results indicated the positive impact of measuring common specific channels of the BCI.

Humans have the amazing ability to learn the dynamics of the body and environment to develop motor skills. Traditional motor studies using arm reaching paradigms have viewed this ability as the process of 'internal model adaptation'. However, the behaviors have not been fully explored in the case when reaches fail to attain the intended target. Here we examined human reaching under two force fields types; one that induces failures (i.e., target errors), and the other that does not. Our results show the presence of a distinct failure-driven adaptation process that enables quick task success after failures, and before completion of internal model adaptation, but that can result in persistent changes to the undisturbed trajectory. These behaviors can be explained by considering a hierarchical interaction between internal model adaptation and the failure-driven adaptation of reach direction. Our findings suggest that movement failure is negotiated using hierarchical motor adaptations by humans.

This paper addresses the recognition of daily-life objects by a robot equipped with tactile sensors. The main contribution is a deep learning framework that can recognize objects already touched as well as objects never touched before. To this end, we train a deconvolutional neural network that generates synthetic tactile data for novel classes. Then, we use both these synthetic data and the real data collected by touching objects, to train a convolutional neural network to recognize both known (trained) objects and novel objects. Furthermore, we propose a method for integrating newly encountered data into novel classes. Finally, we evaluate the framework using the largest available dataset of tactile objects descriptions.

This study proposes a visuo-tactile Zero-Shot object recognition framework. The proposed framework recognizes a set of novel objects for which no tactile or visual training data are available. It uses visuo-tactile training data collected from known objects to recognize the novel ones, given their attributes. This framework extends the haptic Zero-Shot Learning framework that we proposed in [1] with vision, which enables a multimodal recognition system. In our test with the PHAC-2 dataset, the system was able to get a recognition accuracy of 72% among 6 objects that were never touched or seen during the training phase.

Humans are adept at tool use. We can intuitively and immediately improvise and use unknown objects in our environment as tools, to assist us in performing tasks. In this study, we provide similar cognition and capabilities to robots. Neuroscientific studies on tool use have suggested that human dexterity with tools is enabled by the embodiment of the tools, which in effect, allows humans to immediately transfer prior skills acquired without tools, onto tasks requiring tool use. Here, utilizing the theoretical results from our investigations on embodiment and tool use in humans over the last years, we propose a concept and algorithm to enable similar skill transfer by robots. Our algorithm enables a robot that has had no prior learning with tools, to automatically recognize an object (seen for the first time) in its environment as a potential tool for an otherwise unattainable task, and use the tool to perform the task thereafter.

Multiple studies have shown that the mere observation of movements by a robot can affect an observing human's movement; effects referred to as motor contagions. However, previous studies have either analyzed motor contagions induced during (which we call on-line contagions), or induced after (off-line contagions) observation of the robot, but never both together. It thus remains unclear whether and how these two contagions differ from each other. Here, in an empirical industrial co-worker setting, we examine the differences in the off-line and on-line contagions induced in participants by the observation of the same movements performed by a human, or a humanoid robot co-worker. We observed that while the off-line contagions predominantly affect the participant's movement velocity, the on-line contagions affect their movement frequency. Furthermore, the off-line contagions were prominent after observing another human, while the on-line contagions were equally strong with either a human or a humanoid coworker. These results suggest that actions by a humanoid robot can induce distinct effects on human behaviors, during and after observation.

To move a hard table together, humans may coordinate by following the dominant partner's motion [1-4], but this strategy is unsuitable for a soft mattress where the perceived forces are small. How do partners readily coordinate in such differing interaction dynamics? To address this, we investigated how pairs tracked a target using flexion-extension of their wrists, which were coupled by a hard, medium or soft virtual elastic band. Tracking performance monotonically increased with a stiffer band for the worse partner, who had higher tracking error, at the cost of the skilled partner's muscular effort. This suggests that the worse partner followed the skilled one's lead, but simulations show that the results are better explained by a model where partners share movement goals through the forces, whilst the coupling dynamics determine the capacity of communicable information. This model elucidates the versatile mechanism by which humans can coordinate during both hard and soft physical interactions to ensure maximum performance with minimal effort.

Humans can skilfully use tools and interact with the environment by adapting their movement trajectory, contact force, and impedance. Motivated by the human versatility, we develop here a robot controller that concurrently adapts feedforward force, impedance, and reference trajectory when interacting with an unknown environment. In particular, the robot's reference trajectory is adapted to limit the interaction force and maintain it at a desired level, while feedforward force and impedance adaptation compensates for the interaction with the environment. An analysis of the interaction dynamics using Lyapunov theory yields the conditions for convergence of the closed-loop interaction mediated by this controller. Simulations exhibit adaptive properties similar to human motor adaptation. The implementation of this controller for typical interaction tasks including drilling, cutting, and haptic exploration shows that this controller can outperform conventional controllers in contact tooling.

Tool-held hitting tasks, like hammering a nail or striking a ball with a bat, require humans, and robots, to purposely collide and transfer momentum from their limbs to the environment. Due to the vibrational dynamics, every tool has a location where a hit is most efficient results in minimal tool vibrations, and consequently maximum energy transfer to the environment. In sports, this location is often referred to as the “sweet spot” of a bat, or racquet. Our recent neuroscience study suggests that humans optimize hits by using the jerk and torque felt at their hand. Motivated by this result, in this work we first analyze the vibrational dynamics of an end-effector-held bat to understand the signature projected by a sweet spot on the jerk and torque sensed at the end-effector. We then use this analysis to develop a controller for a robotic “baseball hitter”. The controller enables the robot-hitter to iteratively adjust its swing trajectory to ensure that the contact with the ball occurs at the sweet spot of the bat. We tested the controller on the DLR LWR III manipulator with three different bats. Like a human, our robot hitter is able to optimize the energy transfer, specifically maximize the ball velocity, during hits, by using its end effector position and torque sensors, and without any prior knowledge of the shape, size or material of the held bat.

This article presents a versatile controller that enables various contact tooling tasks with minimal prior knowledge of the tooled surface. The controller is derived from results of neuroscience studies that investigated the neural mechanisms utilized by humans to control and learn complex interactions with the environment. We demonstrate here the versatility of this controller in simulations of cutting, drilling and surface exploration tasks, which would normally require different control paradigms. We also present results on the exploration of an unknown surface with a 7-DOF manipulator, where the robot builds a 3D surface map of the surface profile and texture while applying constant force during motion. Our controller provides a unified control framework encompassing behaviors expected from the different specialized control paradigms like position control, force control and impedance control.

Most of today's robots have rigid structures and actuators requiring complex software control algorithms and sophisticated sensor systems in order to behave in a compliant and safe way adapted to contact with unknown environments and humans. By studying and constructing variable impedance actuators and their control, we contribute to the development of actuation units which can match the intrinsic safety, motion performance and energy efficiency of biological systems and in particular the human. As such, this may lead to a new generation of robots that can co-exist and co-operate with people and get closer to the human manipulation and locomotion performance than is possible with current robots.

This paper presents a novel human-like learning controller to interact with unknown environments. Strictly derived from the minimization of instability, motion error, and effort, the controller compensates for the disturbance in the environment in interaction tasks by adapting feedforward force and impedance. In contrast with conventional learning controllers, the new controller can deal with unstable situations that are typical of tool use and gradually acquire a desired stability margin. Simulations show that this controller is a good model of human motor adaptation. Robotic implementations further demonstrate its capabilities to optimally adapt interaction with dynamic environments and humans in joint torque controlled robots and variable impedance actuators, without requiring interaction force sensing.

Interaction of a robot with dynamic environments would require continuous adaptation of force and impedance, which is generally not available in current robot systems. In contrast, humans learn novel task dynamics with appropriate force and impedance through the concurrent minimization of error and energy, and exhibit the ability to modify movement trajectory to comply with obstacles and minimize forces. This article develops a similar automatic motor behavior for a robot and reports experiments with a one degree-of-freedom system. In a postural control task, the robot automatically adapts torque to counter a slow disturbance and shifts to increasing its stiffness when the disturbance increases in frequency. In the presence of rigid obstacles, it refrains from increasing force excessively, and relaxes gradually to follow the obstacle, but comes back to the desired state when the obstacle is removed. A trajectory tracking task demonstrates that the robot is able to adapt to different loads during motion. On introduction of a new load, it increases its stiffness to adapt to the load quickly, and then relaxes once the adaptation is complete. Furthermore, in the presence of an obstacle, the robot adjusts its trajectory to go around it.

This study examines effects of learning 3D micromanipulation in an unstable dynamic environment. A test group trained in an unstable divergent force field while a control group trained the movement in the null force field. The subjects in the test group increased the success rate, in contrast to the control group which had similar rate after training. The error and its standard deviation decreased in the test group but not in the control group. In summary, training in unstable dynamics enable subjects to become more accurate, in contrast to training using only visual feedback.

This paper describes a two-degrees-of-freedom haptic interface to investigate the brain mechanisms of human motor control, which is capable of safely and gently interacting with human arm motion during functional magnetic resonance imaging (fMRI). A hydrostatic transmission separates the interface into a master and an MR compatible slave system, allowing the placement of all interfering components outside the electromagnetic shield of the MR room. The transmission mirrors force and motion of the master actuators on the slave system placed close to the MR scanner. The parallel architecture takes advantage of the linear MR compatible actuators and allows human subjects to perform reaching movements comfortably in the small workspace limited by the dimensions of the MR scanner and the biomechanics of the arm. The kinematic structure of the slave interface was optimized with respect to the available space and types of movements to be investigated. Materials were chosen based on their MR compatibility, their stiffness and weight. The interaction force with the subject is measured over two optical force sensors, located close to the output of the interface. Two shielded optoelectronic encoders measure the extension of the slave hydraulic pistons. Detailed tests demonstrated the fMRI compatibility even during movement of the interface

We analyze the dynamics of an MR-compatible hydrostatic transmission designed to transfer power over distances of up to 10 m. In this system, a master actuates a passive slave connected by two hydrostatic lines in a cyclic arrangement. We derive a nonlinear model of this system and use it to analyze the system's behavior and the design parameters. The transmission acts as a low-pass filter with cut-off frequency decreasing for longer hoses. Even for a length of 10 m the cut-off frequency is about 20 Hz, resulting in a bandwidth that suffices for haptic interfaces interacting with human motion as well as for medical robots. A pragmatic control delivered free movements, position and velocity dependent force fields and trajectory control suitable to investigate how the brain controls movements in interaction with the environment. For short hose lengths (/spl les/1 m) the dynamics can be well approximated by a linear model, and the system is dynamic and stiff. The hydraulic transmission can produce force and motion in any orientation, enabling a more flexible design than other types of transmissions such as by cables.