Cross Reality (CR) is a new emerging field based on the current developments in Mixed Reality hardware, especially supported by the broad market penetration of video-based see-through Head-Mounted Displays. It refers to applications that span across different stages (real, Augmented Reality, Augmented Virtuality, Virtual Reality) of the reality-virtuality continuum, where users are interconnected between different stages and/or are able to transition between these stages. This publication follows the concept of other grand challenges publications and reflects the discussion of various researchers invested in CR. After an initial discussion at the 1stJoint Workshop on Cross Reality at IEEE ISMAR 2023, six topic groups have been identified, leading to 22 challenges, which were discussed in groups over the period of multiple months. The discussion of these challenges should act as a road map for future research in the area of CR.

The paper summarizes the contributions to the AVI 2024 workshop on Prototyping and Developing Real-World Applications of Extended Reality.

Recent Head-Mounted Displays enable users to perceive the real environment using a video-based see-through mode and the fully virtual environment within a single display. Leveraging these advancements, we present a generic concept to seamlessly transition between the real and virtual environment, with the goal of supporting users in engaging with and disengaging from any real environment into Virtual Reality. This transition process uses a digital replica of the real environment and incorporates various stages of Milgram’s Reality-Virtuality Continuum, along with visual transitions that facilitate gradual navigation between them. We implemented the overall transition concept and four object-based transition techniques. The overall transition concept and four techniques were evaluated in a qualitative user study, focusing on user experience, the use of the replica and visual coherence.

Abstract User evaluation is a common and useful tool for systematically generating knowledge and validating novel approaches in the domain of Immersive Analytics. Since this research domain centres around users, user evaluation is of extraordinary relevance. Additionally, Immersive Analytics is an interdisciplinary field of research where different communities bring in their own methodologies. It is vital to investigate and synchronise these different approaches with the long‐term goal to reach a shared evaluation framework. While there have been several studies focusing on Immersive Analytics as a whole or on certain aspects of the domain, this is the first systematic review of the state of evaluation methodology in Immersive Analytics. The main objective of this systematic literature review is to illustrate methodologies and research areas that are still underrepresented in user studies by identifying current practice in user evaluation in the domain of Immersive Analytics in coherence with the PRISMA protocol. (see https://www.acm.org/publications/class-2012 )

Collaboration is a key aspect of immersive visual data analysis. Due to its inherent benefit of seeing co-located collaborators, augmented reality is often useful in such collaborative scenarios. However, to enable the augmentation of the real environment, there are different types of technology available. While there are constant developments in specific devices, each of these device types provide different premises for collaborative visual data analysis. In our work we combine handheld, optical see-through and video see-through displays to explore and understand the impact of these different device types in collaborative immersive analytics. We conducted a mixed-methods collaborative user study where groups of three performed a shared data analysis task in augmented reality with each user working on a different device, to explore differences in collaborative behaviour, user experience and usage patterns. Both quantitative and qualitative data revealed differences in user experience and usage patterns. For collaboration, the different display types influenced how well participants could participate in the collaborative data analysis, nevertheless, there was no measurable effect in verbal communication.

Safety-critical interactive spaces for supervision and time-critical control tasks are usually characterized by many small displays and physical controls, typically found in control rooms or automotive, railway, and aviation cockpits. Using Virtual Reality (VR) simulations instead of a physical system can significantly reduce the training costs of these interactive spaces without risking real-world accidents or occupying expensive physical simulators. However, the user's physical interactions and feedback methods must be technologically mediated. Therefore, we conducted a within-subjects study with 24 participants and compared performance, task load, and simulator sickness during training of authentic aircraft cockpit manipulation tasks. The participants were asked to perform these tasks inside a VR flight simulator (VRFS) for three feedback methods (acoustic, haptic, and acoustic+haptic) and inside a physical flight simulator (PFS) of a commercial airplane cockpit. The study revealed a partial equivalence of VRFS and PFS, control-specific differences input elements, irrelevance of rudimentary vibrotactile feedback, slower movements in VR, as well as a preference for PFS.

Cross-Virtuality applications enabling users to move between different stages of Milgram’s reality-virtuality continuum are a rapidly growing field of research. Modern video see-through head-mounted displays allow users to switch between augmented and virtual reality without removing the headset. This enables for the first time a fluent transition between augmented and virtual reality. Based on insights from literature and preliminary experiments we designed and implemented four transitions: Fade, SimpleCut, TeleportBeam and Portal. These techniques were expected to represent the best suitable concepts for transitioning seamlessly between augmented and virtual reality. After incorporating results from a pre-study, the transition techniques were evaluated in a qualitative user study regarding user experience, simulator sickness, continuity and applicability. Participants were able to freely move between both realities during the study in an immersive analytics scenario for logistics data. In the user study, users preferred Fade in a workplace setting due to its efficiency and simplicity when transitioning frequently between realities. The Portal technique was deemed visually exciting and suitable for infrequent transitions between realities that differ greatly.

Abstract Cross‐virtuality analytics (XVA) is a novel field of research within immersive analytics and visual analytics. A broad range of heterogeneous devices across the reality–virtuality continuum, along with respective visual metaphors and analysis techniques, are currently becoming available. The goal of XVA is to enable visual analytics that use transitional and collaborative interfaces to seamlessly integrate different devices and support multiple users. In this work, we take a closer look at XVA and analyse the existing body of work for an overview of its current state. We classify the related literature regarding ways of establishing cross‐virtuality by interconnecting different stages in the reality–virtuality continuum, as well as techniques for transitioning and collaborating between the different stages. We provide insights into visualization and interaction techniques employed in current XVA systems. We report on ways of evaluating such systems, and analyse the domains where such systems are becoming available. Finally, we discuss open challenges in XVA, giving directions for future research.

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.

We propose using virtual reality (VR) as a design tool for sketching and simulating spatially-aware interactive spaces. Using VR, designers can quickly experience their envisioned spaces and interactions by simulating technologies such as motion tracking, multiple networked devices, or unusual form factors such as spherical touchscreens or bezel-less display tiles. Design ideas can be rapidly iterated without restrictions by the number, size, or shape and availability of devices or sensors in the lab. To understand the potentials and challenges of designing in VR, we conducted a user study with 12 interaction designers. As their tool, they used a custom-built virtual design environment with finger tracking and physics simulations for natural interactions with virtual devices and objects. Our study identified the designers' experience of space in relation to their own bodies and playful design explorations as key opportunities. Key challenges were the complexities of building a usable yet versatile VR-based "World Editor".

Textures are widely used in modern computer graphics. Their size, however, is often a limiting factor. Considering the widespread adaptation of mobile virtual and augmented reality applications, efficient storage of textures has become an important factor.

In this work we describe the steps required to render scientific data for real-time immersive displays. First, the input data needs to be taken into account for further processing. As a second step, the data needs to be reduced without losing significant information in order to enable real-time interaction. Therefore, we describe different reduction techniques and discuss advantages and disadvantages of those. As a third step, the data is integrated in a framework that is able to render the information on different display technologies. We elaborate the general workflow on a geoscientific dataset that illustrates the temperature distribution of the Earth's mantle.

Presentations and discussions between architects and clients during the early stages of design usually involve sketches, paper and models, with digital information in the form of simulations and analyses used to assess variants and underpin arguments. Laypeople, however, are not used to reading plans or models and find it difficult to relate digital representations to the real world. Immersive environments represent an alternative approach but are laborious and costly to produce, particularly in the early design phases where information and ideas are still vague. Our project shows, how linking analogue design tools and digital VR representation has given rise to a new interactive presentation platform that bridges the gap between analogue design methods and digital architectural presentation. The prototypical platform creates a direct connection between a physical volumetric model and interactive digital content using a large-format multi-touch table as a work surface combined with real-time 3D scanning. Coupling the 3D data from the scanned model with the 3D digital environment model makes it possible to compute design relevant simulations and analyses. These are displayed in real-time on the working model to help architects assess and substantiate their design decisions. Combining this with a 5-sided projection installation based on the concepts Carolina Cruz Neiras CAVE Automatic Virtual Environment (CAVE)1offers an entirely new means of presentation and interaction. The design (physical working model), the surroundings (GIS data) and the simulations and analyses are presented stereoscopically in real-time in the virtual environment. While the architect can work as usual, the observer is presented with an entirely new mode of viewing. Different ideas and scenarios can be tried out spontaneously and new ideas can be developed and viewed directly in three dimensions. The client is involved more directly in the process and can contribute own ideas and changes, and then see these in user-centred stereoscopic 3D. By varying system parameters, the model can be walked through at life size.

We present the conceptual design and implementation of the Collaborative Design Platform Protocol (CDPP), a communication protocol that offers the synchronisation of virtual worlds between two mixed reality peers. The CDPP is applied to connect the Collaborative Design Platform (CDP), a design tool which supports the architectural design process in an early stage, with the immersive Cave Automatic Virtual Environment (CAVE) display, where the design is visualised in life-size. This creates a prototype which enables a cost-efficient and easily comprehensible presentation of the early-staged design, and thus significantly simplifies the incorporation of laypeople in the early design process. By this means, the creative capabilities of laymen are exploited to a greater extent.

ShoeSoleSense is a proof of concept, novel body worn interface - an insole that enables location independent hands-free interaction through the feet. Forgoing hand or finger interaction is especially beneficial when the user is engaged in real world tasks. In virtual environments as moving through safety training applications is often conducted via finger input, which is not very suitable. To enable a more intuitive interaction, alternative control concepts utilize gesture control, which is usually tracked by statically installed cameras in CAVE-like-installations. Since tracking coverage is limited, problems may also occur. The introduced prototype provides a novel control concept for virtual reality as well as real life applications. Demonstrated functions include movement control in a virtual reality installation such as moving straight, turning and jumping. Furthermore the prototype provides additional feedback by heating up the feet and vibrating in dedicated areas on the surface of the insole.

Future teleconferencing may enhance communication between remote people by supporting non-verbal communication within an unconstrained space where people can move around and share the manipulation of artefacts. By linking walk-in displays with a Collaborative Virtual Environment (CVE) platform we are able to physically situate a distributed team in a spatially organised social and information context. We have found this to demonstrate unprecedented naturalness in the use of space and body during non-verbal communication and interaction with objects.However, relatively little is known about how people interact through this technology, especially while sharing the manipulation of objects. We observed people engaged in such a task while geographically separated across national boundaries. Our analysis is organised into collaborative scenarios, that each requires a distinct balance of social human communication with consistent shared manipulation of objects.Observational results suggest that walk-in displays do not suffer from some of the important drawbacks of other displays. Previous trials have shown that supporting natural non-verbal communication, along with responsive and consistent shared object manipulation, is hard to achieve. To better understand this problem, we take a close look at how the scenario impacts on the characteristics of event traffic. We conclude by suggesting how various strategies might reduce the consistency problem for particular scenarios.