Benjamin Weyersa* , Barbara Frankb & Annette Klugeb a Human-Computer Interaction, University of Trier, Trier, Germanyb Department of Work and Organizational Psychology, Ruhr-University Bochum, Bochum, GermanyBenjamin Weyers is currently Assistant Professor at University of Trier and Head of the Human-Computer Interaction Group. Before, he was PostDoc at the Virtual Reality and Immersive Visualization group at RWTH Aachen University. He received his PhD in 2011 at the University of Duisburg-Essen and joined RWTH in 2013. He is interested in the development and research on interactive analysis methods for abstract and scientific data using immersive systems as well as the integration of VR and AR into the control of technical systems for the support of human users in semi-automated control scenarios. Additionally, he focuses on the development and use of formal methods for the description of interactive systems.Barbara Frank obtained her PhD in 2017 at the Department of Work, Organizational and Business Psychology at Ruhr University Bochum. She holds a Master of Science in Applied Cognitive and Media Science. Her research focused on various methods for complex cognitive skill retention and impact of individual factors on skill retention. Barbara Frank is currently working in the field of HR-IT at ThyssenKrupp, Essen.Annette Kluge is a Full Professor for Work, Organizational and Business Psychology at the Ruhr University Bochum, Germany. She obtained her Diploma in Work and Organizational Psychology at the Technical University Aachen and her doctorate in ergonomics and vocational training at the University of Kassel, Germany in 1994. Her expertise is in Human Factors and Ergonomics, training science, skill acquisition and retention, safety management and organizational learning from errors and organizational forgetting.CONTACT Benjamin Weyers [email protected] Human-Computer Interaction, University of Trier, Trier, Germany.Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/hihc.ABSTRACTThis paper presents a comprehensive formal engineering approach for the development of gaze guiding as an adaptive computer-based job aid to support semi-automated technical control tasks. Major requirement for this engineering method was to present a model-based approach, which on the one hand offers a coherent integration of various heterogeneous descriptions of the system, the standard operating procedures to be applied, and the used gaze guiding, and an algorithmic transformation into an executable form on the other. The latter is used as part of a runtime system or execution framework that integrates gaze guiding into an existing graphical user interface as an overlay. To demonstrate the suitability of the resulting model that enables gaze guiding, we present a use case for the modeling approach and we conducted a user study with 140 participants who worked on three different types of SOPs. The study showed the positive impact of the gaze guiding and its potential to suite as an adaptive computer-based job aid. The results confirmed findings from previous studies and showed decreased numbers of errors during the execution of the start-up procedure if gaze guiding was used.

State-of-the-art neural microprobes contain hundreds of electrodes within a single shaft. Due to hardware and wiring restrictions, it is usually only possible to measure a small subset of the available electrodes simultaneously. The selection of the best channels is typically performed offline either manually or automatically. However, having a fixed selection for long-term observation does not allow the system to react to changes in the neural activity, and may therefore lead to the loss of important information. In this paper, we formulate the process of autonomously selecting the best subset of electrodes as a combinatorial multi-armed bandit problem with non-stationary rewards, thus allowing the probe to adapt its selection policies online. In order to minimize exploratory actions of the probe, we furthermore take advantage of the existing dependencies between neighboring channels. Our approach is an adaptation of the discounted upper confidence bounds (D-UCB) algorithm, and identifies the electrodes providing the largest amount of non-redundant information. To the best of our knowledge, this is the first online approach for the problem of electrode selection. In extensive experiments, we demonstrate that our solution is not only able to converge towards an average optimal selection policy, but it is also able to react to changes in the neural activity or to damages of the recording electrodes.

Observing human activities can reveal a lot about the structure of the environment, the objects contained therein and also their functionality. This knowledge, in turn, can be useful for robots interacting with humans or for robots performing mobile manipulation tasks. In this paper, we present an approach to infer the geometric and functional structure of the environment and the position of certain relevant objects in it from human activity. We observe this activity using a full-body motion capture system consisting of a set of inertial measurement units. This is a hard problem since our data suit provides odometry estimates only, which severely drift over time. Therefore, we regard the objects inferred from the activities as landmarks in a graph-based simultaneous localization and mapping problem, which we optimize to obtain accurate estimates about the poses of the objects and the trajectory of the human. In extensive experiments, we demonstrate the effectiveness of the proposed method for the reconstruction of 3D representations. The resulting models not only contain a geometric but also a functional description of the environment and naturally provide a segmentation into individual objects.

Stroke and neurodegenerative diseases, among a range of other neurologic disorders, can cause chronic paralysis. Patients suffering from paralysis may remain unable to achieve even basic everyday tasks such as liquid intake. Currently, there is a great interest in developing robotic assistants controlled via brain-machine interfaces (BMIs) to restore the ability to perform such tasks. This paper describes an autonomous robotic assistant for liquid intake. The components of the system include autonomous online detection both of the cup to be grasped and of the mouth of the user. It plans motions of the robot arm under the constraints that the cup stays upright while moving towards the mouth and that the cup stays in direct contact with the user's mouth while the robot tilts it during the drinking phase. To achieve this, our system also includes a technique for online estimation of the location of the user's mouth even under partial occlusions by the cup or robot arm. We tested our system in a real environment and in a shared-control setting using frequency-specific modulations recorded by electroencephalography (EEG) from the brain of the user. Our experiments demonstrate that our BMI-controlled robotic system enables a reliable liquid intake. We believe that our approach can easily be extended to other useful tasks including food intake and object manipulation.

High-resolution microprobes are used to record single neuron activity in the brain. This technology is envisaged to be a central component for brain-controlled computers and robots. Current neural probes, however, allow for recording only a small number of the densely spaced electrodes simultaneously. Therefore, we address the problem of autonomously choosing, for a given number, the subset of electrodes with the corresponding size so as to extract as much information as possible. We first present an approach for predicting neural spikes across different channels of the probe. Our method employs nonparametric sparse Gaussian process regression to predict the signal of a channel given the signals recorded at neighboring sites. Second, we utilize the signal predictions for efficiently seeking for the subset of electrodes that minimizes the overall prediction error. In experiments carried out using real neural data, we demonstrate that our selection procedure provides highly accurate results. Furthermore, the solutions found in our experiments are close to the optimal solution.

The ability to plan their own motions and to reliably execute them is an important precondition for autonomous robots. In this paper, we consider the problem of planning the motion of a mobile manipulation robot in the presence of deformable objects. Our approach combines probabilistic roadmap planning with a physical deformation simulation system. Since the physical deformation simulation is computationally demanding, we use efficient Gaussian process regression to estimate the deformation cost for individual objects based on training examples. We generate the training data by employing a simulation system in a preprocessing step. Consequently, no simulations are needed during runtime. We implemented and tested our approach on a mobile manipulation robot. Our experiments show that the robot is able to accurately predict and thus consider the deformation cost its manipulator introduces to the environment during motion planning. Simultaneously, the computation time is substantially reduced compared to a system that employs physical simulations online.

In this paper, we consider the problem of determining the elasticity properties of deformable objects with a mobile manipulator equipped with a force sensorb. We learn the parameters by establishing a relation between the applied forces and the corresponding surface deformations. To determine the parameters, we minimize the difference between the observed surface of an object that is deformed by a real manipulator and the deformed surface obtained with a deformation simulator based on finite element methods. To establish the correspondences between the surfaces, our approach applies a 3D registration technique based on point-clouds which is used as the basis for comparing the results of the simulation system with the observations of the real deformations. As we demonstrate in real-world experiments, our system is able to estimate appropriate parameters that can be used to predict future deformations. This information can directly be incorporated into motion planning approaches that are designed for robots operating with deformable objects.

In this paper, we consider the problem of mobile robots navigating in environments with non-rigid objects. Whereas robots can plan their paths more effectively when they utilize the information about the deformability of objects, they also need to consider the influence of the interaction with the deformable objects on their measurements during the execution of their navigation task. In this paper, we present a probabilistic approach to identify the measurements influenced by the deformable objects. Based on a learned statistics about the influence of the deformable objects on the measurements, the robot is able to perform a sensor-based collision avoidance of unforeseen objects. We present experiments carried out with a real robot that illustrate the practicability of our approach.

The ability to reliably navigate through the environment is an important prerequisite for truly autonomous robots. In this paper, we consider the problem of path planning in environments with non-rigid obstacles such as curtains or plants. We present an approach that combines probabilistic roadmaps with a physical simulation of object deformations to determine a path that optimizes the trade-off between the deformation cost and the distance to be traveled. We describe how our approach utilizes Finite Element theory for calculating the deformation cost. Since the high computational requirements of the corresponding simulation prevent this method from being applicable online, we present an approximation that uses a preprocessing step to determine a deformation cost function for each object. This cost function allows us to estimate the deformation costs of arbitrary paths through the objects and is used to evaluate the trajectories generated by the roadmap planner online. We present experiments which demonstrate that the resulting algorithm plans nearly identical trajectories compared to the method that relies on computationally intense simulations. At the same time, our approach allows the robot to quickly calculate paths in environments with deformable objects.