Embedding virtual humans in educational settings enables the transfer of the approved concepts of learning by observation and imitation of experts to extended reality scenarios. Whilst various presentation concepts of virtual humans for learning have been investigated in sports and rehabilitation, little is known regarding industrial use cases. In prior work on manual assembly, Lampen et al. [21] show that three-dimensional (3D) registered virtual humans can provide assistance as effective as state-of-the-art HMD-based AR approaches. We extend this work by conducting a comparative user study (N=30) to verify implementation costs of assistive behavior features and 3D registration. The results reveal that the basic concept of a 3D registered virtual human is limited and comparable to a two-dimensional screen aligned presentation. However, by incorporating additional assistive behaviors, the 3D assistance concept is enhanced and shows significant advantages in terms of cognitive savings and reduced errors. Thus, it can be concluded, that this presentation concept is valuable in situations where time is less crucial, e.g. in learning scenarios or during complex tasks.

For head mounted displays, like they are used in mixed reality applications, eye gaze seems to be a natural interaction modality. EyeMRTK provides building blocks for eye gaze interaction in virtual and augmented reality. Based on a hardware abstraction layer, it allows interaction researchers and developers to focus on their interaction concepts, while enabling them to evaluate their ideas on all supported systems. In addition to that, the toolkit provides a simulation layer for debugging purposes, which speeds up prototyping during development on the desktop.

Augmented Reality (AR) is a promising technology for assistance and training in work environments, as it can provide instructions and feedback contextualised. Not only, but especially impaired workers can benefit from this technology. While previous work mostly focused on using AR to assist or train specific predefined tasks, "general purpose" AR applications, that can be used to intuitively author new tasks at run-time, are widely missing.

Recently, the significant technological developments of head-mounted displays and tracking systems have boosted a widespread use of immersive virtual reality in the manufacturing industry. Regardless of the respective use-case, however, methods to ensure the validity of the human motion being performed in such virtual environments remain largely unaddressed. In the context of human locomotion, previous work present models quantifying behavioral differences between virtual reality and the real world. However, those findings are not used in control experiments to post-hoc counterbalance VR-induced performance modulations. Consequently, the prediction quality of previous analyses is not known. This paper bridges this gap, by testing such a behavior model in the context of an independent experiment (n = 10). The evaluation shows that a model derived from literature can indeed be used to post-hoc correct temporal disparities between locomotion in the real world and in virtual reality.

An important use-case of augmented reality-based assistance systems is supporting users in search tasks by guiding their attention towards the relevant targets. This has been shown to reduce search time and errors, such as wrongly picked items or false placements. The optimization of attention guiding techniques is thus one area of research in augmented assistance. In this paper, we address the problem of attention guiding in domains with occluded targets. We propose and evaluate a variant of a line-based approach and show that it improves upon two existing approaches in a newly designed evaluation scenario.

The localization of objects or locations in an environment is an essential task relevant for many work processes. An augmented reality (AR)-based assistance system may support users by guiding their attention towards the relevant targets. This will reduce the time needed for visual search and reduce errors, such as wrongly picked items or false placements. The design of proper attention guiding techniques is thus one area of research in augmented assistance. We developed a number of new attention guiding techniques and evaluated them in several experiments together with recent or classic existing techniques in varying picking scenarios. In our current work, we adapted a standardized assembly scenario to a more complex environment including occlusions to provide a scenario supporting reproducibility of our results. In the research demo, a number of visual and acoustic guiding techniques can be tested and combined. The demonstration is implemented on the Microsoft HoloLens.

With the rise of immersive visualization techniques, many domains within the manufacturing industry are increasingly validating production processes in virtual reality (VR). The validity of the results gathered in such simulations, however, is widely unknown—in particular, with regard to human locomotion behavior. To bridge this gap, this article presents an experiment analyzing the behavioral disparity between human locomotion being performed without any equipment and in immersive VR while wearing a head-mounted display (HMD). The presented study (n= 30) is split up in three sections and covers linear walking, non-linear walking, and obstacle avoidance. Special care has been given to design the experiment so that findings are generally valid and can be applied to a wide range of domains beyond the manufacturing industry. The findings provide novel insights into the effect of immersive VR on specific gait parameters. In total, a comprehensive sample of 18.09km is analyzed. The results reveal that the HMD had a medium effect (up to 13%) on walking velocity, on non-linear walking toward an oriented target, and on clearance distance. The overall differences are modeled using multiple regression models, thus allowing the general usage within various domains. Summarizing, it can be concluded that VR can be used to analyze and plan human locomotion; however, specific details may have to be adjusted to transfer findings to the real world.

Heat maps, or more generally, attention maps or saliency maps are an often used technique to visualize eye-tracking data. With heat maps qualitative information about visual processing can be easily visualized and communicated between experts and laymen. They are thus a versatile tool for many disciplines, in particular for usability engineering, and are often used to get a first overview about recorded eye-tracking data.

With the launch of ultra-portable systems, mobile eye tracking finally has the potential to become mainstream. While eye movements on their own can already be used to identify human activities, such as reading or walking, linking eye movements to objects in the environment provides even deeper insights into human cognitive processing.

While display quality and rendering for Head-Mounted-Displays (HMDs) has increased in quality and performance, the interaction capabilities with these devices are still very limited or relying on expensive technology. Current experiences offered for mobile HMDs often stick to dome-like looking around, automatic or gaze-triggered movement, or flying techniques. We developed an easy to use walking-in-place technique that does not require additional hardware to enable basic navigation, such as walking, running, or jumping, in virtual environments. Our approach is based on the analysis of data from the inertial unit embedded in mobile HMDs. In a first prototype realized for the Samsung Galaxy Gear VR we detect steps and jumps. A user study shows that users novice to virtual reality easily pick up the method. In comparison to a classic input device, using our walking-in-place technique study participants felt more present in the virtual environment and preferred our method for exploration of the virtual world.

Industrie 4.0 (English translation: Industry 4.0) stands for functional integration, dynamic reorganization, and resource efficiency. Technical advances in control and communication create infrastructures that handle more and more tasks automatically. As a result, the complexity of today's and future technical systems is hidden from the user. These advances, however, come with distinct challenges for user interface design. A central question is: how to empower users to understand, monitor, and control the automated processes of Industrie 4.0? Addressing these design challenges requires a full integration of user-centered design (UCD) processes into the development process. This paper discusses flexible but powerful methods for usability and user experience engineering in the context of Industrie 4.0.

Measuring visual attention has become an important tool during product development. Attention maps are important qualitative visualizations to communicate results within the team and to stakeholders. We have developed a GPU-accelerated approach which allows for real-time generation of attention maps for 3D models that can, e.g., be used for on-the-fly visualizations of visual attention distributions and for the generation of heat-map textures for offline high-quality renderings. The presented approach is unique in that it works with monocular and binocular data, respects the depth of focus, can handle moving objects and is ready to be used for selective rendering.

Virtual Reality (VR) has the potential to support motor learning in ways exceeding beyond the possibilities provided by real world environments. New feedback mechanisms can be implemented that support motor learning during the performance of the trainee and afterwards as a performance review. As a consequence, VR environments excel in controlled evaluations, which has been proven in many other application scenarios.

For validly analyzing human visual attention, it is often necessary to proceed from computer-based desktop set-ups to more natural real-world settings. However, the resulting loss of control has to be counterbalanced by increasing participant and/or item count. Together with the effort required to manually annotate the gaze-cursor videos recorded with mobile eye trackers, this renders many studies unfeasible.

For solving complex tasks cooperatively in close interaction with robots, they need to understand natural human communication. To achieve this, robots could benefit from a deeper understanding of the processes that humans use for successful communication. Such skills can be studied by investigating human face-to-face interactions in complex tasks. In our work the focus lies on shared-space interactions in a path planning task and thus 3D gaze directions and hand movements are of particular interest.

Kousidis S, Pfeiffer T, Schlangen D. MINT.tools: Tools and Adaptors Supporting Acquisition, Annotation and Analysis of Multimodal Corpora. In: Proceedings of Interspeech 2013. ISCA; 2013.

Knowledge about the point of regard is a major key for the analysis of visual attention in areas such as psycholinguistics, psychology, neurobiology, computer science and human factors. Eye tracking is thus an established methodology in these areas, e. g., for investigating search processes, human communication behavior, product design or human-computer interaction. As eye tracking is a process which depends heavily on technology, the progress of gaze use in these scientific areas is tied closely to the advancements of eye-tracking technology. It is thus not surprising that in the last decades, research was primarily based on 2D stimuli and rather static scenarios, regarding both content and observer.

In this paper, we argue that empirical research on genuine linguistic topics, such as on the production of multimodal utterances in the speaker and the interpretation of the multimodal signals in the interlocutor, can greatly benefit from the use of virtual reality technologies. Established methodologies for research on multimodal interactions, like the presentation of pre-recorded 2D videos of interaction partners as stimuli and the recording of interaction partners using multiple 2D video cameras have crucial shortcomings regarding ecological validity and the precision of measurements that can be achieved. In addition, these methodologies enforce restrictions on the researcher. The stimuli, for example, are not very interactive and thus not as close to natural interactions as ultimately desired. Also, the analysis of 2D video recordings requires intensive manual annotations, often frame-by-frame, which negatively affects the feasible number of interactions which can be included in a study. The technologies bundled under the term virtual reality offer exciting possibilities for the linguistic researcher: gestures can be tracked without being restricted to fixed perspectives, annotation can be done on large corpora (semi-)automatically and virtual characters can be used to produce specific linguistic stimuli in a repetitive but interactive fashion. Moreover, immersive 3D visualizations can be used to recreate a simulation of the recorded interactions by fusing the raw data with theoretic models to support an iterative data-driven development of linguistic theories. This paper discusses the potential of virtual reality technologies for linguistic research and provides examples for the application of the methodology.

Swiftness and robustness of natural communication is tied to the redundancy and complementarity found in our multimodal communication. Swiftness and robustness of human-computer interaction (HCI) is also a key to the success of a virtual reality (VR) environment. The interpretation of multimodal interaction signals has therefore been considered a high goal in VR research, e.g. following the visions of Bolt's put-that-there in 1980 [1]. It is our impression that research on user interfaces for VR systems has been focused primarily on finding and evaluating technical solutions and thus followed a technology-oriented approach to HCI. In this article, we argue to complement this by a human-oriented approach based on the observation of human-human interaction. The aim is to find models of human-human interaction that can be used to create user interfaces that feel natural. As the field of Linguistics is dedicated to the observation and modeling of human-human communication, it could be worthwhile to approach natural user interfaces from a linguistic perspective. We expect at least two benefits from following this approach. First, the human-oriented approach substantiates our understanding of natural human interactions. Second, it brings about a new perspective by taking the interaction capabilities of a human addressee into account, which are not often explicitly considered or compared with that of the system. As a consequence of following both approaches to create user interfaces, we expect more general models of human interaction to emerge.

The time course and the distribution of visual attention are powerful measures for the evaluation of the usability of products. Eye tracking is thus an established method for evaluating websites, software ergonomy or modern cockpits for cars or airplanes. In most cases, however, the point of regard is measured on 2D products. This article presents work that uses an approach to measure the point of regard in 3D to generate 3D Attention Volumes as a qualitative 3D visualization of the distribution of visual attention. This visualization can be used to evaluate the design of virtual products in an immersive 3D setting, similar as heatmaps are used to assess the design of websites.

Interaction in conversational interfaces strongly relies on the system's capability to interpret the user's references to objects via deictic expressions. Deictic gestures, especially pointing gestures, provide a powerful way of referring to objects and places, e.g., when communicating with an embodied conversational agent in a virtual reality environment. We highlight results drawn from a study on pointing and draw conclusions for the implementation of pointing-based conversational interactions in partly immersive virtual reality.

Recently videoconferencing has been extended from human face-to-face communication to human machine interaction with Virtual Environments [2]. Relying on established videoconferencing protocol standards this thin client solution does not require specialized 3D soft- or hardware and scales well to multimedia enabled mobile devices. This brings a whole range of new applications to the mobile platform. To facilitate our research in mobile interaction the Open Source project P@CE has been started to bring a full-featured videoconferencing client to the Pocket PC platform.