Physical Human-Machine Interaction plays a pivotal role in facilitating collaboration across various domains. When designing appropriate model-based controllers to assist a human in the interaction, the accuracy of the human model is crucial for the resulting overall behavior of the coupled system. When looking at state-of-the-art control approaches, most methods rely on a deterministic model or no model at all of the human behavior. This poses a gap to the current neuroscientific standard regarding human movement modeling, which uses stochastic optimal control models that include signal-dependent noise processes and therefore describe the human behavior much more accurate than the deterministic counterparts. To close this gap by including these stochastic human models in the control design, we introduce a novel design methodology resulting in a Human-Variability-Respecting Optimal Control that explicitly incorporates the human noise processes and their influence on the mean and variability behavior of a physically coupled human-machine system. Our approach results in an improved overall system performance, i.e. higher accuracy and lower variability in target point reaching, while allowing to shape the joint variability, for example to preserve human natural variability patterns.

In this paper, a method for a cooperative trajectory planning between a human and an automation is extended by a behavioral model of the human. This model can characterize the stubbornness of the human, which measures how strong the human adheres to his preferred trajectory. Accordingly, a static model is introduced indicating a link between the force in haptically coupled human-robot interactions and humans's stubbornness. The introduced stubbornness parameter enables an application-independent reaction of the automation for the cooperative trajectory planning. Simulation results in the context of human-machine cooperation in a care application show that the proposed behavioral model can quantitatively estimate the stubbornness of the interacting human, enabling a more targeted adaptation of the automation to the human behavior.

While robotic arms have been used in a vast range of application areas, so far no extensive reports on the utilization as human-machine interface exist. Compared to HMI devices from literature, the robotic arm used in this work (KUKA LBR iiwa 14 R820) features a relatively large workspace and is able to generate force and torque feedback that surpasses the capabilities of literature devices. We describe the setup allowing to use the robotic arm as HMI and analytically determine the optimal initial pose of it based on the manipulability measure of Yoshikawa. To demonstrate that the robotic arm is able to serve as HMI, we report on a comparative study with a state of the art haptic HMI featuring 20 participants. Additionally, two applications from the context of planetary exploration are presented: The first considers the teleoperation of the pan-tilt unit of a lightweight rover unit and illustrates how the large workspace of the HMI benefits the precision of the teleoperation compared to a setup with a smaller workspace. The second experiment showcases the use of the force feedback of the HMI to enable a cooperation between the operator and a supporting path-following automation in a shared control of a simulated ground robot. Both the study and the applications highlight the performance, precision and reliability of our proposed system.

Telemanipulation has become a promising technology that combines human intelligence with robotic capabilities to perform tasks remotely. However, it faces several challenges such as insufficient transparency, low immersion, and limited feedback to the human operator. Moreover, the high cost of haptic interfaces is a major limitation for the application of telemanipulation in various fields, including elder care, where our research is focused. To address these challenges, this paper proposes the usage of nonlinear model predictive control for telemanipulation using low-cost virtual reality controllers, including multiple control goals in the objective function. The framework utilizes models for human input prediction and task-related models of the robot and the environment. The proposed framework is validated on an UR5e robot arm in the scenario of handling liquid without spilling. Further extensions of the framework such as pouring assistance and collision avoidance can easily be included.

This article presents a systematic method for the design of a limited information shared control (LISC). LISC is used in applications where not all system states or references trajectories are measurable by the automation. Typical examples are partially human controlled systems, in which some subsystems are fully controlled by the automation, whereas others are controlled by a human. The proposed systematic design uses a novel class of games to model human–machine interaction: the near potential differential games (NPDG). We provide a necessary and sufficient condition for the existence of an NPDG and derive an algorithm for finding a NPDG, which completely describes a given differential game. The proposed design method is applied to the control of a large vehicle manipulator system, in which the manipulator is controlled by the human operator and the vehicle is fully automated. The suitability of the NPDG modeling differential games is verified in simulations leading to a faster and more accurate controller design compared with manual tuning. Furthermore, the overall design process is validated in a study with 16 test subjects indicating the applicability of the proposed concept in real applications.

This paper presents the validation and the comparative study of a shared control concept for a large vehicle manipulator (LVM). The state-of-the-art controlling a LVM is manual control: The operator controls the manipulator to carry out a specific task and keeps the vehicle on the road. Easing the work for the operator, an automatic lane-keeping of the vehicle can be taken into account: An automation of the vehicle which keeps it on its reference, but without taking into consideration of the manipulator’s specific task. However, the operator has his specific task with the manipulator, and therefore, such automation may not be satisfying. Therefore, this paper presents the validation and compares the Limited Information Shared Controller (LISC) proposed previously with the manual control mode. This step is crucial, showing the concept’s applicability and benefits compared to the state-of-the-art solution. Thus, the LISC is compared with a non-cooperative controller (NCC) and the manual mode on a real-time simulator with test subjects. It has a more realistic experimental setup than in other studies because there is no predefined manipulator reference. The study results indicate that the NCC can lead to undesired motions of the overall system because the test subjects cannot carry out their specific task. On the other hand, the proposed the LISC of the vehicle can reduce the working load while supporting the operator in carrying out the manipulator’s specific task.

Large vehicle-manipulators are systems consisting of a medium-sized heavy-duty vehicle and a hydraulic manipulator. They operate in an unstructured environment and are therefore not fully automated. However, semi-automation of such a system is possible, where the automation controls the vehicle and a human operator controls the manipulator. Since in the unstructured environment not all system trajectories are measurable for the automation, a so-called Limited Information Cooperative Shared-Controller (LICSC) has been proposed in previous work. However, the design of the LICSC is done through tuning which is sensitive to the operator controlling the manipulator. The first contribution of this work is to reduce this sensitivity. A novel personalized design of the LICSC is presented that provides control tailored to the human operator. Our proposed approach uses state-of-the-art design methods of a cooperative controller to obtain the LICSC. The second contribution is the experimental validation of the LICSC, which proves the importance of the personalization to the human operator and demonstrates the advantage of the method.

This paper presents a novel cooperative control algorithm for vehicle-manipulators (VMs) with a human operator. VMs usually operate in unstructured environments, which means that a full automation of the overall system, combing a vehicle and a robotic manipulator is currently very challenging. Therefore, human operator controlled VMs are state-of-the-art. With current developments in autonomous driving, the automation of the vehicle platform is within reach. A cooperative shared control between the autonomous platform and the human controlled manipulator can happen through the coupling motion between the vehicle platform and the manipulator. An autonomous vehicle platform can furthermore be used to support the human operator with the control of the manipulator. However, the future trajectory of the manipulator intended by the human operator is in general not known to the autonomous vehicle. The main question is thus how the autonomous vehicle should act in order to support the human controlled manipulator in following its unknown trajectory. To solve this problem, we propose an approach that characterizes the cooperation and the unknown errors with an algebraic equation. The novel approach is compared to cooperative control methods with known errors of the manipulator, based on the theory of differential games. The benefits of the proposed method are that no sensors for the environment perception and for the state measurements of the manipulator are necessary, which are demonstrated in simulations.