Humans activate muscles to shape the mechanical interaction with their environment, but can they harness this control mechanism to best sense the environment? We investigated how participants adapt their muscle activation to visual and haptic information when tracking a randomly moving target with a robotic interface. The results exhibit a differentiated effect of these sensory modalities, where participants' muscle coactivation increases with the haptic noise and decreases with the visual noise, in apparent contradiction to previous results. These results can be explained when considering muscle spring-like mechanics, where stiffness increases with coactivation to regulate motion guidance. Increasing coactivation to more closely follow the motion plan favors accurate visual over haptic information, while decreasing it filters visual noise and relies more on accurate haptic information. We formulated this active sensing mechanism as the optimization of visuo-haptic information and effort. This optimal information and effort (OIE) model can explain the adaptation of muscle activity to unimodal and multimodal sensory information when interacting with fixed or dynamic environments, or with another human, and can be used to optimize human-robot interaction.

Physically connected humans have been shown to exploit the exchange of haptic forces and tactile information to improve their performance in joint action tasks. As human interactions are increasingly mediated through robots and networks it is important to understand the impact that network features such as lag and noise may have on human behaviour. In this paper, we investigated interaction with a human-like robot controller that provides similar haptic communication behaviour as human-human interaction and examined the influence and compensation mechanisms for delay and noise on haptic communication. The results of our experiments show that participants can perceive a difference between noise and delay, and make use of compensation mechanisms to preserve performance in both cases. However, while noise is compensated for by increasing co-contraction, delay compensation could not be explained by this strategy. Instead, computational modelling suggested that a distinct mechanism is used to compensate for the delay and yield an efficient haptic communication.

Extending the human field of view could enhance our ability to perceive our surroundings thereby improving user safety and enabling us to perform complex manipulation tasks in industrial assembly processes. We investigated the augmentation of environment perception through a study that systematically measured the performance and perception arising from the use of a third eye placed on the back of the user’s head in virtual reality. 28 participants were asked to conduct two goal-oriented tasks: One requiring the identification of targets that discretely appeared in a set of predefined locations; the other the catching of a continuously moving target. The results show that participants were able to incorporate the additional visual information in real time resulting in changes of their motion behaviour. These changes led to localised improvements in task performance for the discrete target task and more efficient motion for both tasks. Participants also showed a strong preference for performing the tasks with the third eye and perceived no accompanying increase in cognitive load.

Supernumerary robotic limbs can act as intelligent prostheses or augment the motion of healthy people to achieve actions which are not possible with only two natural hands. However, as trimanual control is not typical in everyday activities, it is still unknown how different training could influence its acquisition. We conducted an experimental study to evaluate the impact of different forms of trimanual action on training. Two groups of twelve subjects were each trained in virtual reality for five weeks using either a three independent goals task or one dependent goal task. The success of their training was then evaluated by comparing their task performance and motion characteristics between sessions. The results show that subjects dramatically improved their trimanual task performance as a result of training. However, while they showed improved motion efficiency and reduced workload for tasks with multiple independent goals with practice, no such improvement was observed when they trained with the one coordinated goal task.

Many teleoperation tasks require three or more tools working together, which need the cooperation of multiple operators. The effectiveness of such schemes may be limited by communication issues between individuals. Trimanipulation by a single operator using an artificial third arm controlled together with their natural arms may address this issue. Foot-controlled interfaces have previously shown the capability to be used for the continuous control of robot arms. However, the use of such interfaces for controlling a supernumerary robotic limb in coordination with the natural limbs is not well understood. In this paper, a teleoperation task imitating physically-coupled hands in a virtual reality scene was conducted with 14 subjects to evaluate human performance during trimanipulation. The participants were required to move three limbs together in a coordinated way mimicking three arms holding a shared physical object. It was found that after a short practice session, three-hand trimanipulation with a single subject’s hands and foot was still slower than dyad operation. However, they displayed similar performance in their success rate and higher motion efficiency than two people cooperating.