Variable stiffness of a remote robot is crucial for a teleoperation system to deal with challenging tasks. External stiffness command interfaces have emerged as a promising solution to regulating the remote robot stiffness because of the benefits of their accuracy, ergonomics, and avoidance of the “coupling effect” that usually exists in muscle activity-based stiffness interfaces. However, the use of an external stiffness command interface requires good coordination between two limbs of an operator, which take care of the teleoperation task and the stiffness regulation task, respectively, at the same time, which is demanding for novice operators in dynamic situations necessitating agile and timely stiffness adjustments. In this paper, a new concept of Stiffness Regulation Co-pilot was proposed to facilitate the use of these interfaces. A co-pilot is a virtual agent that consists of a Stiffness Regulation Policy, which infers a reasonable stiffness regulation action from the task performance, and a feedback modality, which conveys the suggested stiffness regulation action to the operator. A preliminary user study was conducted to evaluate the efficacy of the co-pilot and the effect of different modalities of the co-pilot. The results showed that the cutaneous feedback or combined with another modality can potentially improve the task performance of the system and reduce the cognitive load of the operator compared to a teleoperation system without using the co-pilot.

In this paper, we propose a model-mediated tele-operation (MMT) system using an octree-based model (OBM) to spatially map the environment impedance for the emerging use cases in Tactile Internet. Different from the existing just-noticeable-difference (JND) based MMT, our method avoids continuous transmission of environment impedance. Moreover, it allows the local model to generate accurate force feedback and reduces the number of model updates. Furthermore, the OBM can be deployed with or without previous knowledge of the environment. An online estimation of the OBM is proposed using a JND and a rate-of-change threshold. An offline estimation method is also proposed when the geometry and impedance parameters of the remote environment are known. In addition, a point cloud-based force rendering algorithm is tailored to use the OBM, thereby allowing the generating of force feedback for complex environments. An experiment without human-in-the-loop was conducted, showing that for an online estimated OBM, the accuracy of the force feedback was improved by up to 44 percent while using less than half the number of model updates when compared to JND-based MMT. Another experiment with a human operator interacting with a virtual environment showed that using an offline estimated OBM improves the accuracy of the force feedback and is reliable against packet loss and short temporal breakdown of the communication link.

The COVID-19 pandemic forced many countries worldwide to take restrictive measures from securing borders and minimising movement, cancelling public gatherings, controlling citizens’ movement, and e...

Virtual reality (VR) training offers the capability to industrial workers to acquire skills and address complex tasks by immersing them in a safe and controlled virtual environment. Immersive VR (IVR) training is adopted in many diverse settings, yet little systematic work currently exists on how researchers have applied it for industrial skills training and if it holds the potential to be applied remotely. In this review, 78 representative studies were analysed to answer three key questions: Is IVR an effective training method for industrial skills training? How is research in this field applied? And how can we make IVR training more effective and applicable for remote training? We can testify that IVR is a promising training method with high effectiveness scores. However, our analysis has uncovered several gaps in the application of IVR training, like the lack of learning theories in the design process and limited metrics beyond time and scores. Additionally, our review also exposed unexplored but intriguing avenues of research, like the utilisation of biosensors for users’ data collection, haptics that increases realism and applications with remote training potential.

From paper-based and face-to-face workshops to electronic and digital simulated educational programs, vast amounts are spent to train workers to perform assembly-tasks, especially in manufacturing. This need for robust, cost-effective, adaptable learning environments supported the rise of Immersive Training. The phenomenon of utilizing the immersive capabilities of Augmented and Virtual Reality (AR/VR) to construct learning environments, i.e. Immersive Training, has flourished, mainly due to the rise and availability of commercial headsets and applications. In this paper, we describe and analyze two such scenarios, a VR lab experiment created to test the VR capabilities in training and a field study exploring the use of AR in a large manufacturing company. Our results provide a) guidelines on how to set up such experiments, b) indicate the temporal demand, and accuracy as significant improvements in VR and c) indicate the effort, physical demand and the time performance in AR.

The focus of this research is to depict the design process of a cost-effective, robust but user-friendly Virtual Immersive Educational (VIE) system. Thus, assist researchers, instructors and designers in identifying an effective method to design VIE systems. In this report, we describe our initial steps to design such a system in order to educate engineering students on the basic health and safety guidelines of safe interaction with a robotic arm. To do so, a set of 360° videos have been designed, developed and tested.

Using Virtual Reality (VR) to realise immersive training environments is not a new concept. However, investigating arousal in immersive environments is. By arousal, we denote a general physical and psychological activity that in the form of anxiety and stress for example, can affect trainees' performance. In this work, we describe the setup design for a two-phase explorative experiment linking arousal and performance, during training in a Virtual Reality (VR) environment. To do so we are using an appraised well-crafted VR puzzle game, questionnaires (i.e. NASA Task Load Index [3]), and sensors (skin conductance response / pulse). The experiment will involve participants from the public that will be trained in two predefined processes of variant difficulty.

Robot teams require planning and adaptive capabilities in order to perform cooperative manipulation tasks in dynamic or unstructured environments. Since these capabilities are inherent to humans, it is suitable to consider human-robot team teleoperation for cooperative manipulation where a single human collaborates with the robot team. In this paper, we present a subtask-based control approach which enables a simultaneous execution of two subtasks by the robot team, interacting with the object: trajectory tracking and formation preservation. Control inputs for both subtasks are provided by the human operator. The commands are projected onto the spaces of subtasks using a command mapping strategy. Analogously, measured interacting forces are projected onto the space of feedback signals, provided to the human via wearable fingertip haptic devices through a feedback mapping strategy. Experimental results validate the proposed approach.

This paper introduces a tactile rendering algorithm for wearable cutaneous devices that stimulate the skin through local contact surface modulation. The first step in the algorithm simulates contact between a skin model and virtual objects, and computes the contact surface to be rendered. The accuracy of this surface is maximized by simulating soft skin with its characteristic nonlinear behavior. The second step takes the desired contact surface as input, and computes the device configuration by solving an optimization problem, i.e., minimizing the deviation between the contact surface in the virtual environment and the contact surface rendered by the device. The method is implemented on a thimble-like wearable device.

The Kuka Control Toolbox (KCT) is a collection of MATLAB functions for motion control of KUKA robot manipulators, developed to offer an intuitive and high-level programming interface to the user. The toolbox, which is compatible with all 6 DOF small and low payload KUKA robots that use the Eth.RSIXML, runs on a remote computer connected with the KUKA controller via TCP/IP. KCT includes more than 30 functions, spanning operations such as forward and inverse kinematics computation, point-to-point joint and Cartesian control, trajectory generation, graphical display and diagnostics. The flexibility, ease of use and reliability of the toolbox is demonstrated through two applicative examples.

This paper presents some preliminary results on RemoTouch, a system allowing to perform experiences of remote touch. The system consists of an avatar equipped with an instrumented glove and a user wearing tactile displays allowing to feel the remote tactile interaction. The main features of RemoTouch are that it is a wearable system and that a human avatar is used to collect remote tactile interaction data. New paradigms of tactile communication can be designed around the RemoTouch system. Two simple experiences are reported to show the potential of the proposed remote touch architecture.