Medical imaging plays a crucial role in diagnosis and surgical planning, but accurately merging segmented 3D models with volumetric medical data remains a significant challenge. Traditional surface-based 3D models provide a clear structural representation but may introduce segmentation inaccuracies, whereas Direct Volume Rendering (DVR) maintains all image details but lacks explicit surface definition. We present a novel hybrid rendering algorithm that seamlessly integrates Per-Pixel Linked Lists for accurate transparency sorting of 3D meshes with DVR techniques to ensure a smooth and precise blending of both data types. This approach captures the advantages of both methods—preserving the structural clarity of surface models while maintaining the depth and density information from the medical image. Our algorithm operates in real time and is compatible with virtual reality, enhancing spatial perception through stereoscopic visualisation. By allowing direct interaction with both segmented models and volumetric data, this technique reduces reliance on radiologists’ reports and improves surgical planning. Performance evaluations confirm real-time rendering, and an expert-based usability study further supports the method’s relevance for advanced medical visualisation applications.

Motion platforms are integral to enhancing user immersion in vehicle simulators by reproducing vehicle accelerations through virtual reality. A key challenge in this domain is ensuring that users perceive critical motion cues accurately. Due to the physical limitations of motion platforms, Motion Cueing algorithms (MCA) often fail to transmit specific acceleration signals that are important for the purpose of the simulator (e.g., training), leading to a loss in motion fidelity. This gap, widely acknowledged but insufficiently addressed in prior research, directly impacts the user’s perception of realism. To address this issue, we propose a heuristic-based enhancement to the classical washout algorithm (CWA) that prioritizes the retention of these critical signals while maintaining system stability. To evaluate the effectiveness of this approach, an experimental study was conducted with 40 participants who tested the system with and without the proposed heuristics. Participants provided real-time feedback and completed post-test questionnaires. The results indicate that the heuristic-enhanced CWA significantly improves users’ ability to perceive signals that were previously lost, thus bridging the identified research gap and advancing the state of motion simulation.

Abstract Augmented Reality (AR) involves the combination of synthetic and real stimuli, not being restricted to visual cues. For the inclusion of computer-generated sound in AR environments, it is often assumed that the distance attenuation model is the most intuitive and useful system for all users, regardless of the characteristics of the environment. This model reduces the gain of the sound sources as a function of the distance between the source and the listener. In this paper, we propose a different attenuation model not only based on distance, but also considering the listener orientation, so the user could listen more clearly the objects that they are looking at, instead of other near objects that could be out of their field of view and interest. We call this a directional attenuation model. To test the model, we developed an AR application that involves visual and sound stimuli to compare the traditional model versus the new one, by considering two different tasks in two AR scenarios in which sound plays an important role. A total of 38 persons participated in the experiments. The results show that the proposed model provides better workload for the two tasks, requiring less time and effort, allowing users to explore the AR environment more easily and intuitively. This demonstrates that this alternative model has the potential to be more efficient for certain applications.

Robotic-Assisted Surgery (RAS) is beginning to unlock its potential. However, despite the latest advances in RAS, the steep learning curve of RAS devices remains a problem. A common teaching resource in surgery is the use of videos of previous procedures, which in RAS are almost always stereoscopic. It is important to be able to add virtual annotations onto these videos so that certain elements of the surgical process are tracked and highlighted during the teaching session. Including virtual annotations in stereoscopic videos turns them into Mixed Reality (MR) experiences, in which tissues, tools and procedures are better observed. However, an MR-based annotation of objects requires tracking and some kind of depth estimation. For this reason, this paper proposes a real-time hybrid tracking–matching method for performing virtual annotations on RAS videos. The proposed method is hybrid because it combines tracking and stereo matching, avoiding the need to calculate the real depth of the pixels. The method was tested with six different state-of-the-art trackers and assessed with videos of a sigmoidectomy of a sigma neoplasia, performed with a Da Vinci® X surgical system. Objective assessment metrics are proposed, presented and calculated for the different solutions. The results show that the method can successfully annotate RAS videos in real-time. Of all the trackers tested for the presented method, the CSRT (Channel and Spatial Reliability Tracking) tracker seems to be the most reliable and robust in terms of tracking capabilities. In addition, in the absence of an absolute ground truth, an assessment with a domain expert using a novel continuous-rating method with an Oculus Quest 2 Virtual Reality device was performed, showing that the depth perception of the virtual annotations is good, despite the fact that no absolute depth values are calculated.

Research on alternative ways to provide anatomy learning and training has increased over the past few years, especially since the COVID-19 pandemic. Virtual reality (VR) and augmented reality (AR) represent two promising alternatives in this regard. For this reason, in this work, we analyze the suitability of applying VR and AR for anatomy training, comparing an optical-based AR setup and a semi-immersive setup based on a VR table, using the same anatomy training software and the same interaction system. The AR-based setup uses a Magic Leap One, whereas the VR table is configured through the use of stereoscopic TV displays and a motion-capture system. This experiment builds on a previous one (Vergel et al., 2020) on which we have improved the AR-based setup and increased the complexity of one of the two tasks. The goal of this new experiment is to confirm whether the changes made in the setups modify the previous conclusions. Our hypothesis is that the improved AR-based setup will be more suitable, for anatomy training, than the VR-based setup. For this reason, we conducted an experimental research with 45 participants, comparing the use of an anatomy training software. Objective and subjective data were collected. The results show that the AR-based setup is the preferred choice. The differences in measurable performance were small but also favorable to the AR setup. In addition, participants provided better subjective ratings for the AR-based setup, confirming our initial hypothesis. Nevertheless, both setups offer a similar overall performance and provide excellent results in the subjective measures, with both systems approaching the highest possible values.

Abstract It is generally accepted that the use of natural interaction provides a positive impact in Virtual Reality (VR) applications. Therefore, it is important to understand what is the best way to integrate and visualize this feature in VR. For this reason, this paper presents a comparative study of the integration of natural hand interaction in two immersive VR systems: a Cave Audio Visual Experience (CAVE) system –where users’ real hands are visible– and a non-see-through Head-Mounted Display (HMD) system –where only a virtual representation of the hands is possible–. In order to test the suitability of using this type of interaction in a CAVE and compare it to an HMD, we raise six research questions related to task performance, usability and perception differences regarding natural hand interaction with these two systems. To answer these questions, we designed an experiment where users have to complete a pick-and-place task with virtual balls and a text-typing task with virtual keyboards. In both systems, the same tracking technology, based on a Leap Motion device, was used. To the best of our knowledge this is the first academic work addressing a comparison of this type. Objective and subjective data were collected during the experiments. The results show that the HMD has a performance, preference and usability advantage over the CAVE with respect to the integration of natural hand interaction. Nevertheless, the results also show that the CAVE system can be, as well, successfully used in combination with an optical hand tracking device.

Abstract Human-computer interaction based on hand gesture tracking is not uncommon in Augmented Reality. In fact, the most recent optical Augmented Reality devices include this type of natural interaction. However, due to hardware and system limitations, these devices, more often than not, settle for semi-natural interaction techniques, which may not always be appropriate for some of the tasks needed in Augmented Reality applications. For this reason, we compare two different optical Augmented Reality setups equipped with hand tracking. The first one is based on a Microsoft HoloLens (released in 2016) and the other one is based on a Magic Leap One (released more than two years later). Both devices offer similar solutions for the visualization and registration problems but differ in the hand tracking approach, since the former uses a metaphoric hand-gesture tracking and the latter relies on an isomorphic approach. We raise seven research questions regarding these two setups, which we answer after performing two task-based experiments using virtual elements, of different sizes, that are moved using natural hand interaction. The questions deal with the accuracy and performance achieved with these setups and also with user preference, recommendation and perceived usefulness. For this purpose, we collect both subjective and objective data about the completion of these tasks. Our initial hypothesis was that there would be differences, in favor of the isomorphic and newer setup, in the use of hand interaction. However, the results surprisingly show that there are very small objective differences between these setups, and the isomorphic approach is not significantly better in terms of accuracy and mistakes, although it allows a faster completion of one of the tasks. In addition, no remarkable statistically significant differences can be found between the two setups in the subjective datasets gathered through a specific questionnaire. We also analyze the opinions of the participants in terms of usefulness, preference and recommendation. The results show that, although the Magic Leap-based system gets more support, the differences are not statistically significant.

Motion-based simulators are used for a variety of applications, such as research, education, entertainment and training. In fact, motion cues are required to achieve the highest regulatory certifications in training vehicle simulators. Nonetheless, the reproduction of self-motion cues presents technological and economic limitations that are not present in the generation of audiovisual cues. For this reason, the generated motion does not generally match the expected one. Therefore, it is necessary to define means to assess the suitability/fidelity of the generated motion cues. After more than 50 years of motion-based vehicle simulation, no mechanism has been universally accepted as the standard solution for the evaluation of motion cues. This paper reviews the mechanisms for obtaining measures of motion fidelity, focusing on those based on objective methods. Since the design of the Objective Motion Cueing Test in 2006, researchers have shown a renewed interest in identifying objective methods to evaluate motion cueing, as the number of works following this approach in recent years reveals. Objective motion fidelity systems allow also performing automatic tuning of MCA by means of optimization techniques, which addresses one of the other main problems of these algorithms. Nevertheless, a universally accepted objective method to assess perceptual motion fidelity in vehicle simulators has not been proposed yet. For this reason, this review work frames and classifies the existing methods. In addition, the authors propose a series of features that an ideal evaluation method for assessing perceptual motion fidelity should include and provide future research guidelines for this complex topic.

Anatomy training with real cadavers poses many practical problems for which new training and educational solutions have been developed making use of technologies based on real-time 3-D graphics. Although virtual reality (VR) and augmented reality (AR) have been previously used in the medical field, it is not easy to select the right 3-D technology or setup for each particular problem. For this reason, this article presents a comprehensive comparative study with 82 participants between two different 3-D interactive setups: an optical-based AR setup, implemented with a Microsoft HoloLens device, and a semi-immersive setup based on a VR Table. Both setups are tested using an anatomy training software application. Our primary hypothesis is that there would be statistically significant differences between the use of the AR application and the use of the VR Table. Our secondary hypothesis is that user preference and recommendation for the VR setup would be higher than for the HoloLens-based system. After completing two different tasks with both setups, the participants filled two questionnaires about the use of the anatomy training application. Three objective measures are also recorded (time, number of movements, and a score). The results of the experiments show that more than two-thirds of the users prefer, recommend, and find more useful the VR setup. The results also show that there are statistically significant differences in the use of both systems in favor of the VR Table.

In recent years Virtual Reality technologies have enabled astronomers to recreate and explore three dimensional structures of the Universe for scientific purposes. Mars, due to its scientific interest,has been the focal point of numerous research projects using these technologies, however, none of these virtual reality tools have been developed specifically for entertainment purposes.The focus of this paper is to present MarsVR, as an entertainment research project that educates people on the topography and orography of the planet Mars from the perspective of popular science. Some projects have been designed MarsVR for entertainment purposes and include the latest advances in 3D real time applications. However, these applications have underestimated the relevant data necessary for simulating the planet Mars as an interactive virtual environment.