Contemporary robotic workplaces are increasingly complex, involving multiple robots, machines, and digital services. Their setup and adaptation demand collaboration across domains, yet domain experts with essential process knowledge are typically non-programmers. Prior research in end-user robot programming has simplified individual task specification, but little is known about how multiple users coordinate and maintain shared awareness when programming together.

The paper presents an approach for handheld augmented reality in constrained industrial environments, where it might be hard or even impossible to reach certain poses within a workspace. Therefore, a user might be unable to see or interact with some digital content in applications like visual robot programming, robotic program visualizations, or workspace annotation. To overcome this limitation, we propose a temporal switching to a non-immersive virtual reality that allows the user to see the virtual counterpart of the workspace from any angle and distance, where the viewpoint is controlled using a unique combination of on-screen controls complemented by the physical motion of the handheld device. Using such a combination, the user can position the virtual camera roughly to the desired pose using the on-screen controls and then continue working just as in augmented reality. To explore how people would use it and what the benefits would be over pure augmented reality, we chose a representative task of object alignment and conducted a study. The results revealed that mainly physical demands, which is often a limiting factor for handheld augmented reality, could be reduced and that the usability and utility of the approach are rated as high. In addition, suggestions for improving the user interface were proposed and discussed.

The paper presents an approach for handheld augmented reality in constrained industrial environments, where it might be hard or even impossible to reach certain poses within a workspace. Therefore, a user might not be able to see or interact with some digital content in applications like visual robot programming, robotic program visualizations, or workspace annotation. To overcome this limitation, we propose a temporal switching to a non-immersive virtual reality, enabling the user to see the workspace from any angle and distance. To explore how people would use it and what the benefits would be over pure augmented reality, we chose a representative task of object alignment and conducted a study. The results revealed that mainly physical demands, which is often a limiting factor for handheld augmented reality, could be reduced and that the usability and utility of the approach are rated as high. In the next iteration, we want to investigate other possibilities of controlling the viewpoint in the virtual environment, as the current approach has potential for improvements.

The problem of selecting virtual objects within augmented reality on handheld devices has been tackled multiple times. However, evaluations were carried out on purely synthetic tasks with uniformly placed homogeneous objects, often located on a plane and with none or low occlusions. This paper presents two novel approaches to indirect object selection dealing with highly occluded objects with large spatial distribution variability and heterogeneous size and appearance. The methods are de-signed to enable long-term usage with a tablet-like device. One method is based on a spatially anchored hierarchy menu, and the other utilizes a crosshair and a side menu that shows candidate objects according to a custom-developed metric. The proposed approaches are compared with direct touch in the context of spatial visual programming of collaborative robots problem, on a realistic workplace and a common robotic task. The preliminary evaluation indicates that the main benefit of the proposed indirect methods could be their higher precision and higher selection confidence for the user.

This paper proposes an intuitive approach for collaborative robot end-user programming using a combination of interactive spatial augmented reality (ISAR) and headmounted display (HMD). It aims to reduce user's workload and to let the user program the robot faster than in classical approaches (e.g. kinesthetic teaching). The proposed approach, where user is using a mixed-reality HMD - Microsoft HoloLens - and touch-enabled table with SAR projected interface as input devices, is compared to a baseline approach, where robot's arms and a touch-enabled table are used as input devices. Main advantages of the proposed approach are the possibility to program the collaborative workspace without the presence of the robot, its speed in comparison to the kinesthetic teaching and an ability to quickly visualize learned program instructions, in form of virtual objects, to enhance the users' orientation within those programs. The approach was evaluated on a set of 20 users using the within-subject experiment design. Evaluation consisted of two pick and place tasks, where users had to start from the scratch as well as to update the existing program. Based on the experiment results, the proposed approach is better in qualitative measures by 33.84% and by 28.46% in quantitative measures over the baseline approach for both tasks.

Nowadays, industrial robots are being programmed using proprietary tools developed by robot manufacturer. A skilled robot programmer is needed to create even as simple task as pick a well-known object and put it somewhere else. Contrary, in every-day life people are using end-user programming to make different electronic devices work in expected manner, without even noticing they are actually programming. We propose augmented reality-enabled end-user programming system allowing regular shop-floor workers to program industrial robotic tasks. The user interface prototype for this system was evaluated in the user study with 7 participants with respect to usability, mental workload and user experience.

This paper presents a novel approach to interaction between human workers and industrial collaborative robots. The proposed approach addresses problems introduced by existing solutions for robot programming. It aims to reduce the mental demands and attention switches by centering all interaction in a shared workspace, combining various modalities and enabling interaction with the system without any external devices. The concept allows simple programming in the form of setting program parameters using spatial augmented reality for visualization and a touch-enabled table and robotic arms as input devices. We evaluated the concept utilizing a user experience study with six participants (shop-floor workers). All participants were able to program the robot and to collaborate with it using the program they parametrized. The final goal is to create a distraction-free, usable and low-effort interface for effective human-robot collaboration, enabling any ordinary skilled worker to customize the robot's program to changes in production or to personal (e.g. ergonomic) needs.

This paper presents results of an exploratory study comparing various modalities employed in an industrial-like robot-human shared workplace. Experiments involved 39 participants who used a touch table, a touch display, hand gestures, a 6D pointing device, and a robot arm to show the robot how to assemble a simple product. To rule out a potential dependence of results on the number of misrecognized actions (resulting, e.g., from unreliable gesture recognition), a controlled amount of interaction errors was introduced. A Wizard-of-Oz setting with three user groups differing in the amount of simulated recognition errors helped us to show that hand gestures and 6D pointing are the fastest modalities that are also generally preferred by users for setting parameters of certain robot operations.