Creating Mixed Reality applications poses distinct challenges for development and design. One of the challenges is designing Mixed Reality application-specific experiences in the wild. In this paper, we present a structured reflection approach to revisit projects from the past. In applying this structured reflection to the data collected during a nine-month industrial project, we unveiled the Virtuality-Reality Clash. To generate a sufficient data corpus, we structurally analysed git commits, tickets, emails, handwritten notes, and weekly snapshots of the 3D designs. The clash could be narrowed down in our data corpus to the situations in which we were fusing the real environment with the virtual content. Finally, we could find five design patterns for MR experience. With these patterns, we aim to help developers and designers of MR applications identify situations where Virtuality and Reality clash and propose approaches to address them.

Rowing has great potential in Virtual Reality (VR) exergames as it requires physical effort and uses physical motion to map the locomotion in a virtual space. However, rowing in VR is currently restricted to locomotion along one axis, leaving 2D and 3D locomotion out of the scope. To facilitate rowing-based locomotion, we implemented three steering techniques based on head, hands, and feet movements for 2D and 3D VR environments. To investigate these methods, we conducted a controlled experiment (N = 24) to assess the user performance, experience and VR sickness. We found that head steering leads to fast and precise steering in 2D and 3D, and hand steering is the most realistic. Feet steering had the largest performance difference between 2D and 3D but comparable precision to hands in 2D. Lastly, head steering is the least mentally demanding, and all methods had comparable VR sickness.

Currently, VR technology is increasingly being used in applications to enable immersive yet controlled research settings. One such area of research is expertise assessment, where novel technological approaches to collecting process data, specifically eye tracking, in combination with explainable models, can provide insights into assessing and training novices, as well as fostering expertise development. We present a machine learning approach to predict teacher expertise by leveraging data from an off-the-shelf VR device collected in a VirATec study. By fusing eye-tracking and controller-tracking data, teachers’ recognition and handling of disruptive events in the classroom are taken into account or considered. Three classification models were compared, including SVM, Random Forest, and LightGBM, with Random Forest achieving the best ROC-AUC score of 0.768 in predicting teacher expertise. The SHAP approach to model interpretation revealed informative features (e.g., fixations on identified disruptive students) for distinguishing teacher expertise. Our study serves as a pioneering effort in assessing teacher expertise using eye tracking within an interactive virtual setting, paving the way for future research and advancements in the field.

As programming is typically a static activity in front of a screen, we perform an initial exploration around the capabilities of block-based programming in the immersive space using Virtual Reality (VR) to make an early charting on how programming could involve moving the programmer’s body. We created a block-based programming interface in a VR space called BlocklyVR based on the existing Blockly programming environment. To investigate programmer performance and experience in BlocklyVR, we conducted a controlled lab experiment (N = 20) with eight programming tasks that covered mathematical operations, if-statements, and function creation. Our initial exploration contributes by classifying movement types made by BlocklyVR programmers and reflecting on how these movements are related to the programming tasks. Additionally, our data suggests that participant performance in BlocklyVR was not affected compared to the 2D Blockly, even if participants were physically moving in VR space. We also found that the virtual reality sickness was marginal. Lastly, we identified four types of interaction that can potentially be employed by VR designers and developers aiming to convert a static task, like programming at a desk, into a “mobile” immersive experience.

Virtual reality applications offer the promise to immerse end users in a synthetic environment where several actions could be observed, simulated, and reproduced, before transferring them to reality, which makes them particularly appropriate for training. Yet, when the training requires complex handling of information, the tasks become cognitively intensive, and developing adequate applications becomes challenging. To address this challenge, we define a method for developing head-mounted-display-based virtual reality applications for modular training tasks, composed of a training model with parameters, a step-wise approach for supporting this development, and a software framework enacting the application of this approach. Authoring such applications is expected to become more flexible and provide personalization facilities. To evaluate the impact of this method, we define a case study concerning an application for training school teachers who deal with a variety of situations in a classroom for an experiment involving N= 7 participants for a set of tasks. Pre-study and post-study acceptances reveal the impact of the software framework and a workload evaluation is conducted using the NASA TLX questionnaire.

Scoliosis is a complex spinal disorder that mostly affects adolescents and younger population. To avoid and reduce harmful radiation techniques in adolescent idiopathic scoliosis investigation and follow-ups, a novel non-invasive approach is highly recommended and needed in clinical practice. In this paper ARScoliosis application to diagnose, visualize and document spinal condition and particularly adolescent idiopathic scoliosis in real time is presented. This ARScoliosis application is developed in Unity 3D, a game engine, and allows non-invasive quantification of posture and spinal deformity using recently available non-ionizing wide-angle depth sensor Microsoft Azure Kinect DK as a diagnostic medium. Parametric and scalable 3D model of the spine is registered to the internal spinal alignment extracted from the anatomical landmarks and joints in real time and visible over the specific patient. It is envisioned that ARScoliosis will support monitoring of scoliosis after applied therapy and include more diagnostic parameters, after final testing and validation against standard X-Ray imaging modalities.

Augmented Reality and Gamification are displaying beneficial effects to enhance user experience and performance in many domains.They are widespread across many areas like education, industrial training, marketing, and services.However, the idea of combining the two approaches for an innovative training instrument is fairly new, especially in assembly training.Moreover, learning about the effects of gamification on human, user engagement, in particular, is a complicated subject.There have been several efforts toward this direction, yet the overall situation is still nascent.In this work, we present a gamified augmented reality training for an industrial task and investigate user engagement effect while training with the gamified and the nongamified system.The result shows that people perform better and engage to a greater degree in the gamified design.

The adoption of virtual reality in various industrial sectors other than gaming is spreading by the day. Many people are still sceptical regarding the potentials and advantages of virtual reality because this has not been made obvious enough to convince them. We investigated how virtual reality affects the concentration, involvement and enjoyment of users during evaluation sessions. Eighty four participants drove on a virtual automated driving simulator with and without the Oculus rift CV1. The experiment showed a statistically significant result for all variables that virtual reality enables better concentration, better involvement and enjoyment when compared with conventional displays.

The integration of new technologies in existing work tasks requires the integration of the users in all steps of the development process. In this work the human-centred design process according to the ISO 9241-210 is applied to fulfil the necessary participation of users. The Head Worn Display Glass of Google Inc. prototypically is used to improve the periodical calibration of the driver assistance system testing bay of the AUDI A8 production line. Therefore the users where observed and interviewed during the performance of the calibration and furthermore highly integrated in the iterative design process of the user interface, the human computer interaction and the program flow. At the end a final prototype could be tested.

Based on the analysis of existing HMI development processes in the automotive domain, a reference process for software engineering has been developed. This process was used to develop a domain data model and a model-based specification language in order to establish a common exchange format based on consistent domain knowledge. This approach provides a common ground for collaboration to OEMs, suppliers, and software tool developers. It also facilitates traceability from requirements and conceptual elements to concrete, specified elements using the model-based specification language.

Automotive development has been dominated by the constraints of driving. However, natural relations to the more general area of Intelligent User Interfaces exist. Previous research in related fields therefore should be adopted and included. The aim of the 2011 MIAA workshop is to foster discussion between experts in otherwise unrelated fields of research. For example, public interfaces use crossmodal referencing in order to circumvent restrictions by the users current focus on a limited communication channel. Our aim is to raise awareness for this approach, concluding that crossmodal references in the car are helpful to bridge the gap between information inside the car and the environment. Another focus topic of the workshop is eco-friendly driving. Although universally regarded as a necessity, it remains an open question how to encourage drivers to drive ecologically. We discuss for example awards for eco-friendly driving by making it competitive and game-like.

Smart labels enable novel item-centric ways of communication along a product's lifecycle. With respect to the open nature of a scenario ranging from manufacturer to consumer, the expectations to such "smart items" are diverse. In this article, we discuss the perspective of potential users and system architects on selected aspects of so-called digital product memories. Our report aligns information we acquired from 332 visitors of a complex prototype system presented at a public IT fair, an internal questionnaire where 14 members of the project SemProM reported about their vision of this concept from an application and technology point of view, and the ongoing implementation works within that project. Our observations address to a large extent research issues related to smart items in general and thus do not only provide an insight to the research development process in this particular project, but also can be exploited for design decisions in other projects in this field.

Industrial production and supply chains face an increased demand for mass-customization and tightening regulations for the traceability of goods, leading to higher requirements concerning flexibility, adaptability and transparency of processes. Technologies for the ¿Internet of Things¿ such as smart products and semantic representations pave their way into future factories and supply chains to fulfill these challenging market demands. In this paper a backend-independent approach for information exchange in open-loop processes based on so-called digital object memories (DOM) is presented. By storing order-related data via smart labels on the item, relevant life cycle information is attached to the product itself. This way, information handover via several stages of the value chain with potentially different stakeholders including manufacturer, distributor, retailer, and end customer has been realized. To summarize first best-practice experiences regarding memory structure and content, a prototype implementation based on a scenario of processing dietary supplements in an adaptive process is illustrated.

Model Driven Development (MDD) is an important para-digm in Software Engineering. In MDD, applications are specified systematically using abstract, platform independent models. The models are then transformed into executable code for different platforms and target devices. Model-driven techniques become ever more prominent in any kind of application, such as multimedia and Web, ubiquitous and automotive applications.