When individuals are paralyzed from injury or damage to the brain, upper body movement and function can be compromised. While the use of body motions to interface with machines has shown to be an effective noninvasive strategy to provide movement assistance and to promote physical rehabilitation, learning to use such interfaces to control complex machines is not well understood. In a five session study, we demonstrate that a subset of an uninjured population is able to learn and improve their ability to use a high-dimensional Body-Machine Interface (BoMI), to control a robotic arm. We use a sensor net of four inertial measurement units, placed bilaterally on the upper body, and a BoMI with the capacity to directly control a robot in six dimensions. We consider whether the way in which the robot control space is mapped from human inputs has any impact on learning. Our results suggest that the space of robot control does play a role in the evolution of human learning: specifically, though robot control in joint space appears to be more intuitive initially, control in task space is found to have a greater capacity for longer-term improvement and learning. Our results further suggest that there is an inverse relationship between control dimension couplings and task performance.

In human-machine interfaces, decoder calibration is critical to enable an effective and seamless interaction with the machine. However, recalibration is often necessary as the decoder off-line predictive power does not generally imply ease-of-use, due to closed loop dynamics and user adaptation that cannot be accounted for during the calibration procedure. Here, we propose an adaptive interface that makes use of a non-linear autoencoder trained iteratively to perform online manifold identification and tracking, with the dual goal of reducing the need for interface recalibration and enhancing human-machine joint performance. Importantly, the proposed approach avoids interrupting the operation of the device and it neither relies on information about the state of the task, nor on the existence of a stable neural or movement manifold, allowing it to be applied in the earliest stages of interface operation, when the formation of new neural strategies is still on-going. In order to more directly test the performance of our algorithm, we defined the autoencoder latent space as the control space of a body-machine interface. After an initial offline parameter tuning, we evaluated the performance of the adaptive interface versus that of a static decoder in approximating the evolving low-dimensional manifold of users simultaneously learning to perform reaching movements within the latent space. Results show that the adaptive approach increased the representational efficiency of the interface decoder. Concurrently, it significantly improved users' task-related performance, indicating that the development of a more accurate internal model is encouraged by the online co-adaptation process.

A medical student learning to perform a laparoscopic procedure or a recently paralyzed user of a powered wheelchair must learn to operate machinery via interfaces that translate their actions into commands for an external device. Since the user's actions are selected from a number of alternatives that would result in the same effect in the control space of the external device, learning to use such interfaces involves dealing with redundancy. Subjects need to learn an externally chosen many-to-one map that transforms their actions into device commands. Mathematically, we describe this type of learning as a deterministic dynamical process, whose state is the evolving forward and inverse internal models of the interface. The forward model predicts the outcomes of actions, while the inverse model generates actions designed to attain desired outcomes. Both the mathematical analysis of the proposed model of learning dynamics and the learning performance observed in a group of subjects demonstrate a first-order exponential convergence of the learning process toward a particular state that depends only on the initial state of the inverse and forward models and on the sequence of targets supplied to the users. Noise is not only present but necessary for the convergence of learning through the minimization of the difference between actual and predicted outcomes.

The laws of physics establish the energetic efficiency of our movements. In some cases, like locomotion, the mechanics of the body dominate in determining the energetically optimal course of action. In other tasks, such as manipulation, energetic costs depend critically upon the variable properties of objects in the environment. Can the brain identify and follow energy-optimal motions when these motions require moving along unfamiliar trajectories? What feedback information is required for such optimal behavior to occur? To answer these questions, we asked participants to move their dominant hand between different positions while holding a virtual mechanical system with complex dynamics (a planar double pendulum). In this task, trajectories of minimum kinetic energy were along curvilinear paths. Our findings demonstrate that participants were capable of finding the energy-optimal paths, but only when provided with veridical visual and haptic information pertaining to the object, lacking which the trajectories were executed along rectilinear paths.

Stroke can lead to loss or impairment of somatosensory sensation (i.e. proprioception), that reduces functional control of limb movements. Here we examine the possibility of providing artificial feedback to make up for lost sensory information following stroke. However, it is not clear whether this kind of sensory substitution is even possible due to stroke-related loss of central processing pathways that subserve somatosensation. In this paper we address this issue in a small cohort of stroke survivors using a tracking task that emulates many activities of daily living. Artificial proprioceptive information was provided to the subjects in the form of vibrotactile cues. The goal was to assist participants in guiding their arm towards a moving target on the screen. Our experiment indicates reliable tracking accuracy under the effect of vibrotactile proprioceptive feedback, even in subjects with impaired natural proprioception. This result is promising and can create new directions in rehabilitation robotics with augmented somatosensory feedback.

In bilateral teleoperation, a human operator manipulates a remote environment through a pair of master and slave robots. The transparency quantifies the fidelity of the teleoperation system, and is typically defined as the ability to accurately display remote environment properties to the operator. We propose a novel multidimensional measure of transparency which takes into account the human operator and consists of three components: 1) perceptual transparency, which quantifies human perception of the remote environment, 2) local motor transparency, which quantifies how far is the movement of the human operator from ideal, and 3) remote motor transparency, which describes how far is the movement of the remote device from ideal. We suggest that for many practical applications, the goal of the transparency optimization is to maintain perceptual and remote motor transparency while sacrificing local motor transparency, and that it is plausible to take advantage of the gap between perception and action in the operators sensorimotor system. We prove analytically that for a teleoperation channel with a position and force scaling and a constant transmission delay, in a palpation and perception of stiffness task, it is possible to find gains that ensure perfect perceptual and remote motor transparency while maintaining stability. We also show that stability depends on the operator that maintain sufficient arm impedance relative to environment impedance and delay.

Progress in decoding neural signals has enabled the development of interfaces that translate cortical brain activities into commands for operating robotic arms and other devices. The electrical stimulation of sensory areas provides a means to create artificial sensory information about the state of a device. Taken together, neural activity recording and microstimulation techniques allow us to embed a portion of the central nervous system within a closed-loop system, whose behavior emerges from the combined dynamical properties of its neural and artificial components. In this study we asked if it is possible to concurrently regulate this bidirectional brain-machine interaction so as to shape a desired dynamical behavior of the combined system. To this end, we followed a well-known biological pathway. In vertebrates, the communications between brain and limb mechanics are mediated by the spinal cord, which combines brain instructions with sensory information and organizes coordinated patterns of muscle forces driving the limbs along dynamically stable trajectories. We report the creation and testing of the first neural interface that emulates this sensory-motor interaction. The interface organizes a bidirectional communication between sensory and motor areas of the brain of anaesthetized rats and an external dynamical object with programmable properties. The system includes (a) a motor interface decoding signals from a motor cortical area, and (b) a sensory interface encoding the state of the external object into electrical stimuli to a somatosensory area. The interactions between brain activities and the state of the external object generate a family of trajectories converging upon a selected equilibrium point from arbitrary starting locations. Thus, the bidirectional interface establishes the possibility to specify not only a particular movement trajectory but an entire family of motions, which includes the prescribed reactions to unexpected perturbations.

-“Knock, knock-who's there?” Here, we do not address this question but rather the underlying mechanism behind the perception of knocking impacts. When one knocks on a surface, her hand makes a forward-backward motion and, at the point of reversal, the knuckles collide with the rigid surface. How does one perceive the unity, or simultaneity, of the sensory events associated with this impact? Does this binding derive from a temporal estimate of simultaneity, or does the brain use some other mechanism? In this paper, we ask whether the tap and the reversal of the hand are perceived as happening together, since both took place at the same time or at a particular state of motion. The aim of this research is to find out whether a tactile event and the flow of proprioceptive information regarding the state of the arm are matched within the central nervous system according to time or state. We tested this experimentally with subjects who actively moved one arm, as well as subjects who were servoed by a robotic device. Our results suggest that time is the mechanism used for judging the unity of the modalities for both active and passive movements. Taken together, these results provide a useful cue for neuroscientists as to the structure and function of the perceptual and motor systems and essential engineering knowledge for the development of effective and realistic augmented reality systems with haptics for the telerobotics, telesurgery, and telepresence applications of the future.

Describes a new theoretical framework for robot-aided training of arm movements. This framework is based on recent studies of motor adaptation in human subjects and on general considerations about adaptive control of artificial and biological systems. The authors propose to take advantage of the adaptive processes through which subjects, when exposed to a perturbing field, develop an internal model of the field as a relation between experienced limb states and forces. The problem of teaching new movements is then reduced to the problem of designing force fields capable of inducing the desired movements as after-effects of the adaptation triggered by prolonged exposure to the fields. This approach is an alternative to more standard training methods based on the explicit specification of the desired movement to the learner. Unlike these methods, the adaptive process does not require explicit awareness of the desired movement as adaptation is uniquely concerned with restoring a preexisting kinematic pattern after a change in dynamical environment.

We have created a hybrid neuro-robotic system that establishes two-way communication between the brain of a lamprey and a small mobile robot. The purpose of this system is to offer a new paradigm for investigating the behavioral, computational, and neurobiological mechanisms of sensory-motor learning in a unified context. The mobile robot acts as an artificial body that delivers sensory information to the neural tissue and receives command signals from it. The sensory information encodes the intensity of light generated by a fixed source. The closed-loop interaction between brain and robot generates autonomous behaviors whose features are strictly related to the structure and operation of the neural preparation. We provide a detailed description of the hybrid system, and we present experimental findings on its performance. In particular, we found (a) that the hybrid system generates stable behaviors, (b) that different preparations display different but systematic responses to the presentation of an optical stimulus, and (c) that alteration of the sensory input leads to short- and long-term adaptive changes in the robot responses. The comparison of the behaviors generated by the lamprey's brain stem with the behaviors generated by network models of the same neural system provides us with a new tool for investigating the computational properties of synaptic plasticity.

Experiments were performed to reveal some of the computational properties of the human motor memory system. We show that as humans practice reaching movements while interacting with a novel mechanical environment, they learn an internal model of the inverse dynamics of that environment. Subjects show recall of this model at testing sessions 24 hours after the initial practice. The representation of the internal model in memory is such that there is interference when there is an attempt to learn a new inverse dynamics map immediately after an anticorrelated mapping was learned. We suggest that this interference is an indication that the same computational elements used to encode the first inverse dynamics map are being used to learn the second mapping. We predict that this leads to a forgetting of the initially learned skill.

One way to control the mechanical interactions between a manipulator and its environment is by specifying the manipulator's end-point impedance. According to this view, the function of a controller is to determine the force that the controlled system should apply in response to an externally imposed state, i.e., to determine an output force field. The possibility of combining the output force fields generated by a number of separate controllers in order to approximate an arbitrary output field is considered for a kinematically redundant manipulator. The end-point force fields of a simulated kinematically redundant manipulator operated by a set of spring-link actuators is numerically derived. The results indicate that the linear superposition of end-point fields may provide a quite good approximation of the actual passive behavior. This offers a simple way for solving the ill-posed problems associated with the specification of end-point impedance fields in a kinematically redundant system.

Problems arising when kinematically redundant manipulators are controlled using the Jacobian pseudoinverse are related to the nonintegrability of the standard pseudoinverse. The authors present a class of pseudoinverse which have the property of being integrable within any simply connected nonsingular region of the workspace. Integrability is obtained by deriving the equations which describe an externally imposed motion with the hypothesis that a compliance function is associated with each degree of freedom of the manipulator. The result is a weighted pseudoinverse containing a term which accounts for the nonlinear features of the forward kinematics. The relation of this integrable pseudoinverse to the standard Moore-Penrose and weighted pseudoinverses are discussed.>