Radiologists are trained professionals who use medical images to obtain clinically relevant information. However, little is known about visual search patterns and strategies radiologists employ during medical image analysis. Thus, there is a current need for guidelines to specify optimal visual search routines commonly used by radiologists. Identifying these features could improve radiologist training and assist radiologists in their work. Our study found that during the moments in which radiologists view chest X-ray images in silence before verbalizing the analysis, they exhibit unique search patterns regardless of the type of disease depicted. Our findings suggest that radiologists’ search behaviors can be identified at this stage. However, when radiologists verbally interpret the X-rays, the gaze patterns appear noisy and arbitrary. Current deep-learning approaches train their systems using this noisy and arbitrary gaze data. This may explain why previous research still needs to show the superiority of deep-learning models that use eye tracking for disease classification. Our paper investigates these patterns and attempts to uncover the eye-gaze configurations during the different analysis phases.

course Share on Approaches and Challenges in XR and AI for Health Applications Authors: Joaquim Jorge INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , João Pereira INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , Anderson Maciel INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , Catarina Moreira Technical University Sydney Technical University SydneyView Profile , Regis Kopper UNC Greensboro UNC GreensboroView Profile Authors Info & Claims SA '23: SIGGRAPH Asia 2023 CoursesDecember 2023Article No.: 2Pages 1–147https://doi.org/10.1145/3610538.3614640Published:06 December 2023Publication History 0citation52DownloadsMetricsTotal Citations0Total Downloads52Last 12 Months52Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

High-speed broadband networks will enable public-safety organizations to handle critical situations that go beyond the common voice communication channel. In this paper, we present a user-centered approach that makes the deployment and adoption of next-generation user interfaces reflect the first responders' needs, requirements, and contexts of use. It is composed of four phases where we elicit requirements, iteratively prototype user interfaces, and evaluate our designs. In this process, public-safety organizations are always engaged, contributing through their feedback and evaluation. We will use immersive Virtual Reality to simulate the user interface designs. Within the virtual environment, it is possible to prototype several concepts before committing to a definitive interface. The solutions proposed will be instrumental for the adoption of next-generation user interfaces by the public safety community.

Mnemonics is a powerful tool to assist memorization of a large amount of complex information. Amongst the different techniques, memory palace, also known as the method of Loci, presents significant results in enhancing memory performance for massive lists of numbers, objects and even texts. Substantial research has been conducted to determine both the validity of applying this complex technique in education, as well as that of applying VR to ease the process of learning this technique. Experiments have shown auspicious results suggesting that a virtual environment helps with spatial memory enhancements. However, previous studies have not investigated factors such as training participants over several stages of learning or including interactivity in the virtual environment. The objective of this pilot project is to assess such effects systematically and suggest several guidelines for improving future studies on virtual memory palace.

Visually-induced motion sickness (VIMS), also known as cyber-sickness, is a major challenge for wide-spread Virtual Reality (VR) adoption. VIMS can be reduced in different ways, for example by using high-quality tracking systems and reducing the user's field of view. However, there are no universal solutions for all situations, and a wide variety of techniques are needed in order for developers to choose the most appropriate options depending on their needs. One way to reduce VIMS is through the use of rest frames-portions of the virtual environment that remain fixed in relation to the real world and do not move as the user virtually moves. We report the results of two multi-day within-subjects studies with 44 subjects who used virtual travel to navigate the environment. In the first study, we investigated the influence of static rest frames with fixed opacity on user comfort. For the second study, we present an enhanced version of rest frames that we call dynamic rest frames, where the opacity of the rest frame changes in response to visually perceived motion as users virtually traversed the virtual environment. Results show that a virtual environment with a static or dynamic rest frame allowed users to travel through more waypoints before stopping due to discomfort compared to a virtual environment without a rest frame. Further, a virtual environment with a static rest frame was also found to result in more real-time reported comfort than when there was no rest frame.

Immersive virtual reality (VR) systems offer flexible control of an interactive environment, along with precise position and orientation tracking of realistic movements. Immersive VR can also be used in conjunction with neurophysiological monitoring techniques, such as electroencephalography (EEG), to record neural activity as users perform complex tasks. As such, the fusion of VR, kinematic tracking, and EEG offers a powerful testbed for naturalistic neuroscience research. In this study, we combine these elements to investigate the cognitive and neural mechanisms that underlie motor skill learning during a multi-day simulated marksmanship training regimen conducted with 20 participants. On each of 3 days, participants performed 8 blocks of 60 trials in which a simulated clay pigeon was launched from behind a trap house. Participants attempted to shoot the moving target with a firearm game controller, receiving immediate positional feedback and running scores after each shot. Over the course of the 3 days that individuals practiced this protocol, shot accuracy and precision improved significantly while reaction times got significantly faster. Furthermore, results demonstrate that more negative EEG amplitudes produced over the visual cortices correlate with better shooting performance measured by accuracy, reaction times, and response times, indicating that early visual system plasticity underlies behavioral learning in this task. These findings point towards a naturalistic neuroscience approach that can be used to identify neural markers of marksmanship performance.

In recent years, digital technology has become ubiquitous in the museum. They have changed the ways museums document, preserve and present cultural heritage [3]. Now, we are exploring if there are some ways that could provide more historical context to a displayed object and make an exhibition more immersive. Therefore, we did a project called “The Virtual Reality Touch Museum” and used an experiment to test if such museum performs better on “Presence” and learning achievements. As the results show, our VR Touch Museum was outstanding in “presence” but more research is necessary to verify how effective it is for learning.

This demo presents a fully immersive and interactive virtual environment (VE) - the ArcheoVR, which represents Itapeva Rocky Shelter, a prehistoric archeological site in Brazil. W workflow started with a real world data capture - laser scanners, drones and photogrammetry. Captured information was transformed into a carefully designed realistic 3D scene and interactive features that allows users to experience the virtual archeological site in real-time. The main objective of this VR model is to allow the general public to feel and explore an otherwise restricted and ephemeral site and to assess prototype tools intended for future digital archaeological exploration.

This poster presents the conceptual process of developing Itapeva 3D, a Virtual Reality (VR) archeology experience. It describes the technical spectrum of cyber-archeology process applied to the creation of a fully immersive and interactive virtual environment (VE), which represents Itapeva Rocky Shelter, a prehistoric archeological site in Brazil. The workflow starts with a real world data capture - laser scanners, drones and photogrammetry, continues with the transposition of the captured information into a 3D surface model capable of real-time rendering to head-mounted displays (HMDs), and ends with the design of interactive features allowing users to experience the virtual archeological site. The main objective of this VR model is to make plausible to general public to feel what it means to explore an otherwise restricted and ephemeral place. As final thoughts it is reported on preliminary results from an initial user observation.

The emotional response a person has to a living space is predominantly affected by light, color and texture as space-making elements. In order to verify whether this phenomenon could be replicated in a simulated environment, we conducted a user study in a six-sided projected immersive display that utilized equivalent design attributes of brightness, color and texture in order to assess to which extent the emotional response in a simulated environment is affected by the same parameters affecting real environments. Since emotional response depends upon the context, we evaluated the emotional responses of two groups of users: inactive (passive) and active (performing a typical daily activity). The results from the perceptual study generated data from which design principles for a virtual living space are articulated. Such a space, as an alternative to expensive built dwellings, could potentially support new, minimalist lifestyles of occupants, defined as the neo-nomads, aligned with their work experience in the digital domain through the generation of emotional experiences of spaces. Data from the experiments confirmed the hypothesis that perceivable emotional aspects of real-world spaces could be successfully generated through simulation of design attributes in the virtual space. The subjective response to the virtual space was consistent with corresponding responses from real-world color and brightness emotional perception. Our data could serve the virtual reality (VR) community in its attempt to conceive of further applications of virtual spaces for well-defined activities.

We propose Specimen Box, an interaction technique that allows world-fixed display (such as CAVEs) users to naturally hold a plausible physical object while manipulating virtual content inside it. This virtual content is rendered based on the tracked position of the box. Specimen Box provides the weight and tactile feel of an actual object and does not occlude rendered objects in the scene. The end result is that the user sees the virtual content as if it exists inside the clear physical box. We conducted a user study which involved a cognitively loaded inspection task requiring extensive manipulation of the box. We compared Specimen Box to Grab-and-Twirl, a naturalistic bimanual manipulation technique that closely mimics the mechanics of our proposed technique. Results show that performance was significantly faster with Specimen Box. Further, performance of the control technique was positively affected by experience with Specimen Box.

In this work we present the methodological approach applied to develop a fully immersive and interactive virtual environment that simulates an archaeological site located in Sao Paulo (Brazil). To create a realistic 3D space, which would be relevant for research through a cyber-archeology exploration, laser scanners and photometry were used for collecting 3D point clouds data from the physical. In consequence, the digital data acquired from these apparatus generated a huge density of point clouds, requiring a many gigabytes computer storage and a research work on design to compact all the information in an user friendly interactive virtual model, but realistic to archaeologists. Also to provide an immersive feeling when exploring the virtual reality we decided to allow the user to navigate through the scene using control devices (keyboard, mouse and joysticks) and a head-mounted display (Oculus Rift) to visualize the aesthetical and spatial elements of the archaeological site as if she/he was really in that place (forms, scales, proportions, perspective, textures, illumination, shadows). In resume, through a sophisticated digital simulation environment, which regards the playful of an electronic game in first-person field of vision, we created a telepresence sense to the user, as well providing archeologists a landscape (and objects) exploration through a non-destructive way.

Here we explore a visual display technique for low frame rate virtual environments called low persistence (LP). This involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system.

This paper describes the use of 3D sound as a wayfinding tool in virtual environments. We present the design of a study and a pilot evaluation of the use of spatial sound as a wayfinding cue on navigation in virtual environments. A pilot study with four participants offered insights suggesting that auditory cues may have the potential to help users perform wayfinding tasks in a virtual mall environment faster and more accurately than in an environment without auditory cues.

In virtual reality applications, there is an aim to provide real time graphics which run at high refresh rates. However, there are many situations in which this is not possible due to simulation or rendering issues. When running at low frame rates, several aspects of the user experience are affected. For example, each frame is displayed for an extended period of time, causing a high persistence image artifact. The effect of this artifact is that movement may lose continuity, and the image jumps from one frame to another. In this paper, we discuss our initial exploration of the effects of high persistence frames caused by low refresh rates and compare it to high frame rates and to a technique we developed to mitigate the effects of low frame rates. In this technique, the low frame rate simulation images are displayed with low persistence by blanking out the display during the extra time such image would be displayed. In order to isolate the visual effects, we constructed a simulator for low and high persistence displays that does not affect input latency. A controlled user study comparing the three conditions for the tasks of 3D selection and navigation was conducted. Results indicate that the low persistence display technique may not negatively impact user experience or performance as compared to the high persistence case. Directions for future work on the use of low persistence displays for low frame rate situations are discussed.

Virtual reality training systems are commonly used in a variety of domains, and it is important to understand how the realism of a training simulation influences training effectiveness. We conducted a controlled experiment to test the effects of display and scenario properties on training effectiveness for a visual scanning task in a simulated urban environment. The experiment varied the levels of field of view and visual complexity during a training phase and then evaluated scanning performance with the simulator's highest levels of fidelity and scene complexity. To assess scanning performance, we measured target detection and adherence to a prescribed strategy. The results show that both field of view and visual complexity significantly affected target detection during training; higher field of view led to better performance and higher visual complexity worsened performance. Additionally, adherence to the prescribed visual scanning strategy during assessment was best when the level of visual complexity during training matched that of the assessment conditions, providing evidence that similar visual complexity was important for learning the technique. The results also demonstrate that task performance during training was not always a sufficient measure of mastery of an instructed technique. That is, if learning a prescribed strategy or skill is the goal of a training exercise, performance in a simulation may not be an appropriate indicator of effectiveness outside of training-evaluation in a more realistic setting may be necessary.

Water Cherenkov neutrino detectors, like the existing Super-Kamiokande ("Super-K") and the planned Hyper-Kamiokande ("Hyper-K") detectors, are used to study neutrino particle physics. These detectors consist of large vessels holding many thousands of tons of water, as well as tens of thousands of photomultiplier tubes. Such detectors produce large and multi-layered datasets well-suited to immersive visualization and interaction. We have developed a novel virtual reality (VR) application called Neutrino-KAVE which functions as an visualization and data interaction application. Neutrino-KAVE displays the collocation of photon sensors and their color-coded data within a to-scale representation of the Super-K or Hyper-K detector, and provides a new visualization technique for neutrino interaction patterns. Neutrino-KAVE also provides both a mechanism for modifying aspects of the presented data set, and a user interface for system control of this multifaceted application. In this paper, we describe in detail our implementation and design choices. We also report on its use cases, initial reception, and future development.

It has been shown that virtual reality (VR) can be used to train mine workers for safety in critical situations [4]. The National Institute for Occupational Safety and Health (NIOSH) has a virtual reality (VR) laboratory on its Pittsburgh campus. Currently, input devices for the system are an Xbox 360 game pad and an air mouse. Due to the high cost and added complexity of most 3D tracking systems, we wanted to first test to see if the mine safety application could benefit from an upgrade to a 6-DOF tracking system. Thus, we conducted a pilot study at Duke University's six-sided CAVE-type system, and collected performance and questionnaire data for three tasks (selection, navigation, and maneuvering) and three devices (gamepad, air mouse, 6-DOF wand). Results indicate that the wand allows users to complete tasks faster and is preferred by users. However, in certain situations its use led to more errors.

In some virtual reality (VR) systems, OpenGL intercept methods are used to capture and render a desktop application's OpenGL calls within an immersive display. These systems often suffer from lower frame rates due to network bandwidth limitations, implementation of the intercept routine, and in some cases, the intercepted application's frame rate. To mitigate these issues and to enhance intercept-based systems in other ways, we present intercept tags, which are OpenGL geometries that are interpreted instead of rendered. We have identified and developed several uses for intercept tags, including hand-off interactions, display techniques, and visual enhancements. To demonstrate the value of intercept tags, we conducted a user study to compare a simple virtual hand technique implemented with and without intercept tags. Our results show that intercept tags significantly improve user performance and experience.

Spatial judgments are important for many real-world tasks in engineering and scientific visualization. While existing research provides evidence that higher levels of display and interaction fidelity in virtual reality systems offer advantages for spatial understanding, few investigations have focused on small-scale spatial judgments or employed experimental tasks similar to those used in real-world applications. After an earlier study that considered a broad analysis of various spatial understanding tasks, we present the results of a follow-up study focusing on small-scale spatial judgments. In this research, we independently controlled field of regard, stereoscopy, and head-tracked rendering to study their effects on the performance of a task involving precise spatial inspections of complex 3D structures. Measuring time and errors, we asked participants to distinguish between structural gaps and intersections between components of 3D models designed to be similar to real underground cave systems. The overall results suggest that the addition of the higher fidelity system features support performance improvements in making small-scale spatial judgments. Through analyses of the effects of individual system components, the experiment shows that participants made significantly fewer errors with either an increased field of regard or with the addition of head-tracked rendering. The results also indicate that participants performed significantly faster when the system provided the combination of stereo and head-tracked rendering.

Stressful interpersonal experiences can be difficult to prepare for. Virtual humans may be leveraged to allow learners to safely gain exposure to stressful interpersonal experiences. In this paper we present a between-subjects study exploring how the presence of a virtual human affected learners while practicing a stressful interpersonal experience. Twenty-six fourth-year medical students practiced performing a prostate exam on a prostate exam simulator. Participants in the experimental condition examined a simulator augmented with a virtual human. Other participants examined a standard unaugmented simulator. Participants reactions were assessed using self-reported, behavioral, and physiological metrics. Participants who examined the virtual human experienced significantly more stress, measured via skin conductance. Participants stress was correlated with previous experience performing real prostate exams; participants who had performed more real prostate exams were more likely to experience stress while examining the virtual human. Participants who examined the virtual human showed signs of greater engagement; non-stressed participants performed better prostate exams while stressed participants treated the virtual human more realistically. Results indicated that stress evoked by virtual humans is linked to similar previous real-world stressful experiences, implying that learners real-world experience must be taken into account when using virtual humans to prepare them for stressful interpersonal experiences.

The concepts of immersion and presence focus on the environment in a virtual environment. We instead focus on embodied conversational agents (ECAs). ECAs occupy the virtual environment as interactive partners. We propose that the ECA analogues of immersion and presence are physicality and social presence. We performed a study to determine the effect of an ECA's physicality on social presence and eliciting realistic behavior from the user. The results showed that increasing physicality can elicit realistic behavior and increase social presence but there was also an interaction effect with plausibility.

The design of virtual environments for applications that have several levels of scale has not been deeply addressed. In particular, navigation in such environments is a significant problem. This paper describes the design and evaluation of two navigation techniques for multiscale virtual environments (MSVEs). Issues such as spatial orientation and understanding were addressed in the design process of the navigation techniques. The evaluation of the techniques was done with two experimental and two control groups. The results show that the techniques we designed were significantly better than the control conditions with respect to the time for task completion and accuracy.