Humans activate muscles to shape the mechanical interaction with their environment, but can they harness this control mechanism to best sense the environment? We investigated how participants adapt their muscle activation to visual and haptic information when tracking a randomly moving target with a robotic interface. The results exhibit a differentiated effect of these sensory modalities, where participants' muscle coactivation increases with the haptic noise and decreases with the visual noise, in apparent contradiction to previous results. These results can be explained when considering muscle spring-like mechanics, where stiffness increases with coactivation to regulate motion guidance. Increasing coactivation to more closely follow the motion plan favors accurate visual over haptic information, while decreasing it filters visual noise and relies more on accurate haptic information. We formulated this active sensing mechanism as the optimization of visuo-haptic information and effort. This optimal information and effort (OIE) model can explain the adaptation of muscle activity to unimodal and multimodal sensory information when interacting with fixed or dynamic environments, or with another human, and can be used to optimize human-robot interaction.

The manipulation of instruments under a microscope suffers from physiological tremor and human errors, which are inevitable in long microsurgery interventions. Robotic systems developed in recent years for microsurgery are expensive and not flexible, as they cannot use standard instruments, and need the surgeon to modify their operative skills and strategies. In this paper, we introduce a modular robotic system for microsurgery enabling the surgeon to operate using conventional instruments. Our system was implemented using a commercial Kinova robot and a dedicated modular end-effector that uses standard microsurgery instruments. An initial teleoperation validation was carried out by eleven participants, who could successfully control the microsurgery tools to perform basic surgical movements. Furthermore, participants performed a simple anastomosis task with the robot and compared it to manual control. The results showed that robotic control is superior to manual control in simple surgical tasks and the converse in complex tasks. Participants preferred the proposed robotic system due to its user-friendliness and effort reduction.

The utilization of shared control in the realm of surgical robotics augments precision and safety by amalgamating human expertise with autonomous assistance. This paper proposes a user-centered shared control framework enabling a robot to learn from expert demonstration, predict operators’ intent and modulate control authority to provide natural assistance when needed. We employ deep inverse reinforcement learning (IRL) to enable the robot to learn path planning from expert demonstrations with fast convergence, subsequently enhancing the policy with a potential field method. The control authority is allocated seamlessly between the human operator and the autonomous agent based on the prediction of operators’ movement from an adaptive filter and fuzzy logic inference. The proposed method is executed using the da Vinci Research Kit (dVRK) robot in a simulation environment, and its effectiveness is assessed through user performance evaluation in a trajectory tracking task. Compared to direct control and simple shared control, the proposed shared control scheme exhibits superior tracking accuracy and trajectory smoothness under external disturbances. Subjective responses underscore users’ perception of the method’s efficacy in enhancing their performance.

Extending the human field of view could enhance our ability to perceive our surroundings thereby improving user safety and enabling us to perform complex manipulation tasks in industrial assembly processes. We investigated the augmentation of environment perception through a study that systematically measured the performance and perception arising from the use of a third eye placed on the back of the user’s head in virtual reality. 28 participants were asked to conduct two goal-oriented tasks: One requiring the identification of targets that discretely appeared in a set of predefined locations; the other the catching of a continuously moving target. The results show that participants were able to incorporate the additional visual information in real time resulting in changes of their motion behaviour. These changes led to localised improvements in task performance for the discrete target task and more efficient motion for both tasks. Participants also showed a strong preference for performing the tasks with the third eye and perceived no accompanying increase in cognitive load.