Cross Reality (CR) is a new emerging field based on the current developments in Mixed Reality hardware, especially supported by the broad market penetration of video-based see-through Head-Mounted Displays. It refers to applications that span across different stages (real, Augmented Reality, Augmented Virtuality, Virtual Reality) of the reality-virtuality continuum, where users are interconnected between different stages and/or are able to transition between these stages. This publication follows the concept of other grand challenges publications and reflects the discussion of various researchers invested in CR. After an initial discussion at the 1stJoint Workshop on Cross Reality at IEEE ISMAR 2023, six topic groups have been identified, leading to 22 challenges, which were discussed in groups over the period of multiple months. The discussion of these challenges should act as a road map for future research in the area of CR.

Proficient training in Diagnostic Imaging (DI) is crucial for veterinary students. However, the training process faces challenges, including limited access to animals and strict radiation safety regulations. Can Mixed Reality (MR) address these issues? Two co-located collaborative MR teaching systems were implemented to simulate the horse DI procedure with real-time X-ray simulation. One system integrated actual X-ray equipment, while the other was entirely virtual. A user-centered comparative study was conducted, assessing students’ performance and opinions to explore the differences and the necessity of physical versus virtual equipment. The performance of 22 veterinary students and four DI experts was evaluated, focusing on the quality of the readiographs and the time taken to obtain the X-ray images. Students initially underperformed compared to experts using physical equipment but matched expert performance by the final image, while they performed similarly with virtual equipment. Experts took X-rays faster than students in both cases. Both systems offered comparable user experiences, though most preferred virtual equipment for its usability and presentation. Physical equipment remained essential for realistic simulation. The results show that the MR system, combined with real-time X-ray image synthesis, has great potential to overcome obstacles in veterinary DI training and serve as a support tool for both training and assessment.

Real-time rendering applications leverage heterogeneous computing to optimize performance. However, software development across multiple devices presents challenges, including data layout inconsistencies, synchronization issues, resource management complexities, and architectural disparities. Additionally, the creation of such systems requires verbose and unsafe programming models. Recent developments in domain-specific and unified shading languages aim to mitigate these issues. Yet, current programming models primarily address data layout consistency, neglecting other persistent challenges.In this paper, we introduce RenderKernel, a programming model designed to simplify the development of real-time rendering systems. Recognizing the need for a high-level approach, RenderKernel addresses the specific challenges of real-time rendering, enabling development on heterogeneous systems as if they were homogeneous. This model allows for early detection and prevention of errors due to system heterogeneity at compile-time. Furthermore, RenderKernel enables the use of common programming patterns from homogeneous environments, freeing developers from the complexities of underlying heterogeneous systems. Developers can focus on coding unique application features, thereby enhancing productivity and reducing the cognitive load associated with real-time rendering system development.

Augmented Reality (AR), as a novel data visualization tool, is advantageous in revealing spatial data patterns and data-context associations. Accordingly, recent research has identified AR data visualization as a promising approach to increasing decision-making efficiency and effectiveness. As a result, AR has been applied in various decision support systems to enhance knowledge conveying and comprehension, in which the different data-reality associations have been constructed to aid decision-making. However, how these AR visualization strategies can enhance different decision support datasets has not been reviewed thoroughly. Especially given the rise of Big Data in the modern world, this support is critical to decision-making in the coming years. Using AR to embed the decision support data and explanation data into the end user’s physical surroundings and focal contexts avoids isolating the human decision-maker from the relevant data. Integrating the decision-maker’s contexts and the DSS support in AR is a difficult challenge. This paper outlines the current state of the Art through a literature review in allowing AR data visualization to support decision-making. To facilitate the publication classification and analysis, the paper proposes one taxonomy to classify different AR data visualization based on the semantic associations between the AR data and physical context. Based on this taxonomy and a decision support system taxonomy, 37 publications have been classified and analyzed from multiple aspects. One of the contributions of this literature review is a resulting AR visualization taxonomy that can be applied to decision support systems. Along with this novel tool, the paper discusses the current state of Art in this field and indicates possible future challenges and directions that AR Data Visualization will bring to support decision-making.

As part of the clinical training during their degree course, veterinary students learn how to safely obtain radiographs in horses. However, this can sometimes be challenging due to ethical considerations related to the use of live animals and the fact that it is strictly dependent on the caseload of the veterinary teaching hospital. This networked setup allows the lecturer to guide students in equine radiographic techniques without requiring actual horses or an X-ray machine. A study involving veterinary students showed promising results regarding the effectiveness of using MR to teach radiation safety while performing radiographic techniques on horses. In addition to performance metrics, we employed questionnaires, including MREQ, VRSQ, and UEQ, to collect demographic data and participant feedback. Participants praised the system’s pedagogical effectiveness and overall user experience. The immersive MR experience created a sense of presence and co-presence, underscoring the potential for broader applications of co-located MR in radiology and other areas.

The concept of merging gamification and mixed reality (MR) can create a motivational combination for better student performance and learning outcomes. Therefore, this work-in-progress study aims to design a methodology that can create an educational application to improve student engagement, motivation, and their learning outcomes with the advantage of MR technology features. This study presents a gamified MR application approach using an enhanced attention, relevance, confidence, and satisfaction (ARCS) model with gamification called “ARCS +G” to combine instructional designs with motivational designs. Furthermore, it enables educators and developers to design and develop engaging applications for their audiences.

New technologies are becoming an essential part of education. Advances in these technologies are changing traditional learning processes, thus affecting students’ motivation and strongly influencing their learning outcomes. Mixed reality (MR) is one of the emerging technologies that may affect students’ learning motivation, with the potential to improve their performance and enhance learning outcomes. Nevertheless, it remains unclear how MR affects students’ motivation to learn. Thus, the aim of this doctoral colloquium is to propose how research may be conducted to evaluate the efficacy and potential of MR technology for motivating dentistry students in higher education. In this research, we design and develop a gamified educational MR application by combining the (Analysis, Design, Development, Implementation, and Evaluation) ADDIE instructional design framework with the (Attention, Relevance, Confidence, and Satisfaction) ARCS model for motivational design that is integrated with gamification components, called the ARCS+G model. The MR application is to be used by dentistry students at Umm Al-Qura University in a fundamental implantology course to train them on pre-operative planning using dental cone-beam computed tomography (CBCT). In this setting, we will investigate students’ pretest and posttest learning motivation with the Instructional Materials Motivation Survey (IMMS).

Virtual Reality (VR) technology has been shown to achieve remarkable results in multiple fields. Due to the nature of the immersive medium of Virtual Reality it logically follows that it can be used as a high-quality educational tool as it offers potentially a higher bandwidth than other mediums such as text, pictures and videos. This short paper illustrates the development of a climate change educational awareness application for virtual reality to simulate virtual scenes of local scenery and sea level rising until 2100 using prediction data. The paper also reports on the current in progress work of porting the system to Augmented Reality (AR) and future work to evaluate the system.

Light Field Displays (LFD) offer the potential for a true window into a virtual world without any form of headset. However, the vast majority of LFD only provide motion parallax along the x-axis (horizontal) due to its domination of human 3D perception. The additional motion parallax along y and the z-axis are rarely or never achieved and let alone evaluated. This paper proposed the first approach that provided the real full-motion parallax covering the z-axis. Moreover, this generic approach enables on-demand y and z-axis motion parallax on any off-the-shelf LFD. A prototype was created according to the proposed approach, and with the use of it, an experiment with two tasks was carried out among 24 participants. This pioneering study explored the effect of the motion parallax along the additional axes. Subjective and objective metrics were collected to measure participants’ 3D perception on four viewing conditions (LFD with motion parallax along the x-axis, x-and-y-axis, x-and-z-axis, and x-y-and-z-axis). The study manifested three main findings: 1) The additional y-axis and z-axis motion parallax increased viewers’ engagement with the LFD. 2) LFD with full-motion parallax provided the optimal user experience. 3) The additional motion parallax along with the y-axis increased viewers’ user experience more than the z-axis.

Due to the global pandemic, heavily practical based teaching in veterinary education has been hugely impacted. Even before the crisis, an innovative way to provide an assessment that could simulate the examination of a cadaver has been a long-standing dream for veterinary schools and faculty. We designed and implemented a summative assessment system using touchless interaction head-mounted display (HMD) Virtual Reality (VR), and presented a user study that was an initial exploration of the proposed system designed to replace traditional spatial multiple choices questions (MCQs) for veterinary students. The performance relating to the score and time spent of 24 veterinary students was measured in online MCQ and VR MCQ. Questionnaires were sent out to gather demographic data and opinions. Manipulation time in the VR environment was tracked. The results showed that the time spent was significantly shorter using VR as an assessment tool compared to online MCQ. Participants revealed that the VR assessment outperformed the traditional method in terms of satisfaction, distinctiveness, concentration, and nervousness levels. However, they underperformed in the VR examination significantly. The summative assessment in VR has the potential to further support veterinary education, and the usage of HMDs and touchless interaction will boost the process. This evaluation illustrates the gap between students’ perception of VR examination and their MCQ performance that will require future research to untangle if this technology is to be adopted in the anatomy classroom.

Augmented Reality (AR) has shown significant potential in supporting in-situ decision-making in various application areas. Many prior works have shown how AR can visualize the decision support data in various contexts. However, prior research about AR-based decision support systems rarely explored how the explanations were visualized. Providing context-aware explanations within AR-based recommendation systems may help users instantly understand the recommendations they have been given. Therefore, this paper presents the world-first user study exploring AR explanation visualization designs. Three feature importance analysis visualizations that apply different color-coding and size-scaling strategies were designed to explain the recommendations provided by a context-aware AR shopping assistant system. Twenty-four participants were recruited to evaluate these three explanations in a shopping scenario. The results revealed novel findings that could help guide the appropriate utilization of descriptive parameters when designing AR explanation artifacts. The results also show the potential of providing intuitive visualization to explain recommendations in AR.

The reliability of European Remote Sensing 5 (ERA5) satellite-based air temperature data is under investigation in this letter. To evaluate this, the ERA5 data will be compared with land-based data obtained from weather stations on the global historical climatology network (GHCN). Two climate regions are taken into consideration, temperate and tropical. Five years worth of data is collected and compared through box plots, regression models, and statistical metrics. The results show that the satellite temperature performs better in the temperate region than the tropical region. This suggests that the time of year and climate region have an impact on the accuracy of the satellite data as milder temperatures produce better approximations.

During the last decade, there has been a surge in research studies exploring the adoption of Augmented Reality (AR) in educational settings. Within these multiple research studies, AR's capability to extend the teaching and learning environment with augmented 3D learning objects with enhanced interactive capabilities have been demonstrated. This new technology has not been widely adopted in the mainstream but with the recent unprecedented circumstances of COVID-19, there has been an increasing societal willingness to adopt these technologies. AR has been a desirable technology due to its inherent touchless nature which facilitates social distancing at this time but AR applications crucially offer so much more. They can provide interactive functionality through augmentation of the teaching and learning environment within an immersive user experience including 3D interactions with learning objects, gestures, hand interaction, tangible and multi-modal interaction. This paper presents the results of a review of touchless interaction studies in educational applications and proposes the implementation of real-time touchless hand interaction within kinesthetic learning and utilization of machine learning agents. The architecture of two AR applications with real-time hand interaction and machine learning agents are demonstrated within this paper enabling engaged kinesthetic learning as an alternative learning interface.

Augmented Reality and Virtual Reality become increasingly popular in scientific visualization especially for education where they can support collaborative scientific visualization experiences in the classroom. However, the inherent limitations of head-mounted AR and VR tools are stemming the popularization of these existing content sharing tools. Instead of sharing 3D educational content between AR/VR headsets, this paper proposes a novel prototype Mixed rEaliTy shAring pLatform (METAL) to allow for 3D educational content to be shared between a Microsoft HoloLens 2 and multiple Looking Glass displays which are a type of Light Field (Multi-view Autostereoscopic) display. This platform allows one teacher to use a HoloLens to manipulate and share different 3D contents with multiple student groups via the network, thus each student group can observe the synchronized 3D educational content with autostereoscopic experiences. Therefore, this proposed prototype enables a low-cost one-to-multiple 3D content sharing experience that allows the intuitive 3D model interaction and seamless communication between the students and the teacher.

Augmented Reality (AR) learning has successfully been proven to support new learning pedagogies. The pedagogical approach of Kinesthetic Learning or "Learning by Doing" using AR combined with virtual touchless hand interaction and machine learning agents has not been explored in great detail. Fundamentally this exploration of touchless technologies and learning intelligence has taken on new importance in the new post-COVID world which created a need for more interactive and personalized learning solutions. This research presented the implementation of touchless hand interaction and machine learning agents in an AR learning application for teaching the PC assembly in the resource constraint environment.

During the last decade, there has been an increase in the development of training systems based on Virtual Reality (VR) and Augmented Reality (AR). This paper describes work in progress on exploring the challenges in the creation of anatomy simulators for professionals such as Doctors and Veterinarians. Given the current limitations of workstation powered VR headsets which are primarily their lack of mobility both for a given machine and the Head Mounted Display (HMD) which is commonly tethered. This lack of mobility limits the efficiency of communications between the lecturers and students. This work explores using AR-based smartphone version and mobile VR HMD as an alternative. This work-in-progress paper describes the comparison between the mobile platform AR and mobile immersive headset VR in different aspects of Veterinary Education.

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of radiology. Radiology departments have begun exploring the use of these technologies to help with radiology education and training. The purpose of this article is to address how skills have been developed in the gaming world and how these can be adopted for radiology education. Radiology is rapidly evolving with the use of AI for more effective diagnostic and prognostic clinical assessment. Advances in computing technology have enabled widespread availability of simulated reality technologies, including virtual reality (VR) and augmented reality (AR). This work in progress paper describes VR and AR technologies as novel means to communicate and have potential for supplementing radiology training; communicating with colleagues, referring clinicians, and patients; and aiding in interventional radiology procedures.

As a promising alternative to VR head-mounted displays, current autostereoscopic displays and light field displays require high GPU consumption for multiple-view rendering and do not display different scenes for multiple viewers with motion parallax. Building upon prior work demonstrating how GPU utilization can be reduced by only rendering visible views, we propose an innovative approach to only render the visible views of different scenes towards multiple viewers’ eyes. Moreover, we found that the number of visible views decreases as the viewing distances increase. Thus, a dynamic approach can be taken to adjust the number of rendered views according to viewers distance from the display. This approach can be easily adapted to the off-the-shelf light field displays to display different 3D scenes for at least two viewers according to their head positions with reduced GPU costs.

VR remote learning is an environment-friendly approach for anatomy learning compared with the paper-based, physical model or prepared specimens. However, VR devices can potentially be costly pieces of equipment, and inexperienced users can experience unwanted symptoms like motion sickness. To solve these problems, an asymmetrical system has been developed to connect an experienced head-mounted-display user (lecturer) with light field display users (students) through the Internet. The scenes of lecturer’s end and students’ end are adjusted to match the corresponding displays technologies.

The future of surgical training is changing. One of the bottlenecks in the training of surgeons is the availability of either real patients while being supervised or cadavers. Both these resources can be in limited supply. Surgical trainers have been a common solution to this problem but can be expensive thus limiting their use in the developing world. This paper will examine how using Augmented Reality (AR), Virtual Reality (VR) and inexpensive 3D printing can act as a replacement to expensive training devices for sole purposes. For the work described in this paper, the EndoVascular Aneurysm Repair (EVAR) operation will be examined to explore whether medical training for young surgeons can be improved with the integration of VR/AR. EVAR operation is an innovative, less invasive procedure used to treat problems affecting the blood vessels, such as an aneurysm, which is a swelling or “ballooning” of the blood vessel. The surgery involves making a small incision near each hip to access the blood vessels. The paper will outline an approach as a proof of concept, in order to create a mixed reality training tool that through the use of 3D printing can give tactile feedback to the trainee during training.

We increasingly live in a world where sensors have become truly ubiquitous in nature. Many of these sensors are an integral part of devices such as smartphones, which contain sufficient sensors to allow for their use as Augmented Reality (AR) devices. This AR experience is limited by the precision and functionality of an individual device's sensors and the its capacity to process the sensor data into a useable form. This paper discuss the current work on a mobile version of the SIXTH middleware which allows for creation of Sensor Web enabled AR applications. This paper discusses current work on mobile SIXTH, which involves the creation of a sensor web between different Android and non-Android devices. This has led to several small demonstrators which are discussed in this work in progress paper. Future work on the project will be outline the aims of the project to allow for the integration of additional devices so as to explore new abilities such as leveraging additional proprieties of those devices.

With the increasing availability of sensors within smartphones and within the world at large, a question arises about how this sensor data can be leveraged by Augmented Reality (AR) devices. AR devices have traditionally been limited by the capability of a given device's unique set of sensors. Connecting sensors from multiple devices using a Sensor Web could address this problem. Through leveraging this SensorWeb existing AR environments could be improved and new scenarios made possible, with devices that previously could not have being used as part of an AR environment. This paper proposes the use of SIXTH: a middleware designed to generate a Sensor Web, which allows a device to leverage heterogeneous external sensors within its environment to help facilitate the creation of richer AR experiences. This paper will present a worst case scenario, in which the device chosen will be a see-through, Android-based Head Mounted Display that has no access to sensors. This device is transformed into an AR device through the creation of a Sensor Web allowing it to sense its environment facilitated through the use of SIXTH.

Networked robotic applications enable robots to operate in distant, hazardous, or otherwise inaccessible environments, such as search and rescue, surveillance, and exploration applications.

Addressing the obesity epidemic that plagues many societies remains an outstanding public health issue. One innovative approach to addressing this problem is Exergaming. A combination of "Exercise" and "Gaming", the objective is to motivate people participate in exercise regimes, usually in their home environment. In this paper a more holistic interpretation of this exercise paradigm is proposed. Freegaming augments Exergaming in a number of key dimensions but especially through the promotion of games in outdoor mobile contexts and within a social environment. The design and implementation of a platform for Freegaming is described and illustrated through the description of a sample game.