Augmented reality has shown promise as an assistive tool for mass casualty incident (MCI) triage to facilitate cognitive support during live response and to furnish immersive virtual MCI training. It has been shown that women and people of color experience less effective triaging in emergency room settings, yielding diminished healthcare outcomes. In the more chaotic and austere MCI event, triage efforts further determine which patients are left to expire and which get life-saving interventions before they arrive at the hospital. This work offers an exploratory within subjects investigation with actual first responders (N = 13) to examine preliminary patterns about whether diminished triage efficacy based on patient demographics can be captured during an augmented-reality virtual training simulation (1V) and/or within a 'real' patient training simulation (supported with AR triage interfaces) with patient actors (2R). While both conditions used a Hololens2, the 1V condition displayed AR virtual patients, whereas the 'patient actor' condition had real people scattered in a physical space needing triaged. We conduct ANOVAS and construct linear models to evaluate our data. Our exploratory analysis finds slight potential impacts of patient demographics in triage efficacy within the 'real' simulation, and more severe impacts in the virtual simulation. We further discuss implications of this work and explore underlying cognitive factors.

Fast, accurate spatial-temporal cognition is vital for situational awareness and mission success in indoor emergencies. Complex layouts and occlusions hinder first responders' ability to judge distances, scales, and task completion times. This paper explores the enhancement of these cognitive abilities through the application of Augmented Reality (AR) devices. Utilizing mixed reality and reality capturing technologies, a digital twin model of a three-level indoor structure was built based on LiDAR scans of Turlington Hall at the University of Florida. A mixed reality environment was then constructed using the Unity engine and Microsoft HoloLens2. In a user study (N = 30), we systematically compared three visualization conditions (2D Map, 3D Map, and Egocentric Map) across spatial and temporal estimation tasks. Non-parametric statistical analysis with correction for multiple comparisons revealed a task-dependent pattern. The Egocentric Map showed the lowest error for ETA estimation (MAPE = 17.2% vs. 23.3% for 2D maps), suggesting a trend toward improved temporal cognition, though this pairwise difference did not reach statistical significance after correction. Conversely, for distance-between-objects tasks, 2D and 3D maps yielded significantly lower error than Egocentric views (MAPE = 24.3% and 17.7% vs. 36.8%, $p_{adj} < 0.01$padj<0.01). These findings reveal a task-dependent trade-off, indicating that no single visualization mode is universally optimal: AR overlays may support action-oriented temporal planning, while exocentric views remain superior for precise metric spatial judgments in emergency response.

In this paper, we detail a novel method for calculating torque levels generated by users wearing a head-mounted display (HMD). This method considers not only the movement from a given user but also the weight characteristics of the specific configured headset. In this way, we can calculate a user's expended effort during a specific time frame to better understand the effort expended, as well as the risk of fatigue or injury, while using augmented and virtual reality HMDs. To illustrate the applicability of the method, we apply it in an initial user study analyzing torque and force during a basic target selection task and determined that (1) both force and torque decrease significantly over time as participants tire, and (2) the lighter HMD incurred lower levels of force and torque throughout the various trials. This research substantially furthers the practice of human factors and usability in augmented and virtual reality and fills a need within the community by adding to the extant literature an easy to use and well-documented method that can determine the effect of the total HMD system (HMD hardware, HMD interface, and task elements) on end users over both long- and short-duration tasks.

Our research explored the use of Whiskers, Wedge, and Compass augmented reality (AR) visualization techniques (VTs) in an outdoor field-based user study conducted with Search and Rescue (SAR) subject matter experts (SMEs). The goal of this study was to understand what hazard-based out-of-view information SAR responders need and how SAR SMEs prefer information to be presented. We investigated why certain design elements and preattentive properties of these visualization techniques are most preferred for use during a SAR mission. A key aim was to gain insight on what AR design elements can help guide SAR responders away from dangerous out-of-view hazards, while still allowing them to efficiently search for victims. We derived a set of AR VT design guidelines, specifically applicable to dynamic, potentially dangerous environments, through semi-structured interviews with SMEs and thematic analysis of data. We additionally identified how SAR SMEs envision these AR VTs fitting in with the current SAR ecosystem.

Recent advancements in augmented reality (AR) hardware, software and application capabilities have introduced exciting benefits and advantages, especially in industrial and occupational fields. However, many new challenges have arisen with the increasing use of AR in real work settings, such as increased visual capture and attention demanded by graphics presented within a relatively small field of view (FOV) (as compared to users’ natural FOV). This systematic review paper addresses how to effectively use and design AR visualization techniques to cue objects located outside a user’s FOV. We posit that new visualization techniques are needed to cue out-of-view objects while maintaining user attention and minimizing distraction from the user’s primary task. A significant amount of research has been done to examine effective visualizations for guiding attention in AR, specifically how to encode direction and distance of out-of-view objects. Our review compares the performance associated with existing techniques, as well as what characteristics have been implemented and studied. In this work, we also present design guidelines derived from our analysis and synthesis to understand what visualization technique characteristics may be effective. Our final recommendations describe the value of reference lines, feedback to users, location indicators, best practices to encode direction and distance, and the use of subtle cues.

Augmented Reality (AR) training is a cost-effective and safe alternative to traditional instructional methods. However, training novices in basic mid-air AR interactions remains challenging. To address this, we aimed to: (a) develop a robust metric to evaluate user performance across different AR interaction techniques and develop adaptation models to predict additional training requirements; (b) evaluate the adaptation models using a neuroergonomics approach. We conduct a two-phase study during which, novice participants perform simple AR interactions: poking and raycasting. In Phase-I, twenty-seven participants’ data is used to identify a bi-variate performance metric based on median completion ime and consistency. Unsupervised models are trained using this metric to classify participants as low/high performers. In Phase-II, we evaluate the models on twenty-one new participants and analyze the differences in performance, neural activity and heart-rate variability between low/high performers. Our study showcases the effectiveness of our models and further discusses the potential of integrating neuroergonomics for advanced AR-based training applications.

Physical QWERTY keyboards are the current standard for performing precision text-entry with extended reality devices. Ideally, there would exist a comparable, self-contained solution that works anywhere, without requiring external keyboards. Unfortunately, when physical keyboards are recreated virtually, we currently lose critical haptic feedback information from the sense of touch, which impedes typing. In this paper, we introduce the MusiKeys Technique, which uses auditory feedback in virtual reality to communicate missing haptic feedback information typists normally receive when using a physical keyboard. To examine this concept, we conducted a user study with 24 participants which encompassed four mid-air virtual keyboards augmented with increasing amounts of feedback information, along with a fifth physical keyboard for reference. Results suggest that providing clicking feedback on key-press and key-release improves typing performance compared to not providing auditory feedback, which is consistent with the literature. We also found that audio can serve as a substitute for information contained in haptic feedback, in that users can accurately perceive the presented information. However, under our specific study conditions, this awareness of the feedback information did not yield significant differences in typing performance. Our results suggest this kind of feedback replacement can be perceived by users but needs more research to tune and improve the specific techniques.

During cycling activities, cyclists often monitor a variety of information such as heart rate, distance, and navigation using a bike-mounted phone or cyclocomputer. In many cases, cyclists also ride on sidewalks or paths that contain pedestrians and other obstructions such as potholes, so monitoring information on a bike-mounted interface can slow the cyclist down or cause accidents and injury. In this article, we present HazARdSnap, an augmented reality-based information delivery approach that improves the ease of access to cycling information and at the same time preserves the user's awareness of hazards. To do so, we implemented real-time outdoor hazard detection using a combination of computer vision and motion and position data from a head mounted display (HMD). We then developed an algorithm that snaps information to detected hazards when they are also viewed so that users can simultaneously view both rendered virtual cycling information and the real-world cues such as depth, position, time to hazard, and speed that are needed to assess and avoid hazards. Results from a study with 24 participants that made use of real-world cycling and virtual hazards showed that both HazARdSnap and forward-fixed augmented reality (AR) user interfaces (UIs) can effectively help cyclists access virtual information without having to look down, which resulted in fewer collisions (51% and 43% reduction compared to baseline, respectively) with virtual hazards.

Augmented Reality (AR) and exergames have been trending areas of interest in healthcare spaces for rehabilitation and exercise. This work reviews 25 papers featuring AR rehabilitative/exercise games and paints a picture of the literature landscape. The included results span twelve years, with the oldest paper published in 2010 and the most recent work in 2022. More specifically, this work contributes a bank of representative ARREGs and a synthesis of measurement strategies for player perceptions of Augmented Reality Rehabilitative and Exercise Game (ARREG) experiences, the elements that comprise the exergame experience, the intended use cases of ARREGs, whether participants are actually representative users, the utilized devices and AR modalities, the measures used to capture rehabilitative success, and the measures used to capture participant perceptions. Informed by the literature body, our most significant contribution is nine considerations for future ARREG development.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the journal papers from the 22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2023), which will be held as a hybrid conference between October 16 and 20, 2023 in Sydney, Australia. ISMAR continues the over twenty-year long tradition of IWAR, ISMR, and ISAR, and is the premier conference for Mixed and Augmented Reality in the world.

In this work we present ARTTS: a head-worn Augmented Reality (AR) Triage Tool Suite containing an initial sorting tool, virtual assessment tool, and virtual triage tag to assist emergency responders in mass casualty incidents. The initial sorting tool can prompt novice responders through first-wave tasks to aid recalibration from shock to triage. The virtual assessment tool provides novice responders, potentially confused by the chaos, with a walkthrough of the SALT triage flowchart. Finally, current emergency medical triage processes leverage static paper tags susceptible to loss or illegible damage. ARTTS’ virtual triage tags are dynamic, can be updated through responder interaction, and employ user interface emergent features based on individual patient conditions. This paper describes ARTTS’ capabilities, as well as the applied user-centered design process including review of existing triage material, subject-matter expert interview transcripts, wireframing, application of usability and user-centered design principles, as well as iterative usability subject-matter expert assessments and design walkthroughs. The ARTTS user experience aims to enhance, not upend, existing triage processes. Finally, this paper provides a usability evaluation comparing ARTTS’ virtual triage tag to a physical paper triage tag. Our tag achieved requisite System Usability Scale (SUS) scores and showed negligible differences to the paper triage tag on usability and mental workload.

In optical see-through augmented reality (AR), information is often distributed between real and virtual contexts, and often appears at different distances from the user. To integrate information, users must repeatedly switch context and change focal distance. If the user's task is conducted under time pressure, they may attempt to integrate information while their eye is still changing focal distance, a phenomenon we term transient focal blur. Previously, Gabbard, Mehra, and Swan (2018) examined these issues, using a text-based visual search task on a one-eye optical see-through AR display. This paper reports an experiment that partially replicates and extends this task on a custom-built AR Haploscope. The experiment examined the effects of context switching, focal switching distance, binocular and monocular viewing, and transient focal blur on task performance and eye fatigue. Context switching increased eye fatigue but did not decrease performance. Increasing focal switching distance increased eye fatigue and decreased performance. Monocular viewing also increased eye fatigue and decreased performance. The transient focal blur effect resulted in additional performance decrements, and is an addition to knowledge about AR user interface design issues.

Augmented reality (AR) offers new ways to visualize information on-the-go. As noted in related work, AR graphics presented via optical see-through AR displays are particularly prone to color blending, whereby intended graphic colors may be perceptually altered by real-world backgrounds, ultimately degrading usability. This work adds to this body of knowledge by presenting a methodology for assessing AR interface color robustness, as quantitatively measured via shifts in the CIE color space, and qualitatively assessed in terms of users' perceived color name. We conducted a human factors study where twelve participants examined eight AR colors atop three real-world backgrounds as viewed through an in-vehicle AR head-up display (HUD); a type of optical see-through display used to project driving-related information atop the forward-looking road scene. Participants completed visual search tasks, matched the perceived AR HUD color against the WCS color palette, and verbally named the perceived color. We present analysis that suggests blue, green, and yellow AR colors are relatively robust, while red and brown are not, and discuss the impact of chromaticity shift and dispersion on outdoor AR interface design. While this work presents a case study in transportation, the methodology is applicable to a wide range of AR displays in many application domains and settings.

Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

In this special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the journal papers from the 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2022), which will be held as a hybrid conference between October 17 and 21, 2022 in Singapore. ISMAR continues the over twenty year long tradition of IWAR, ISMR, and ISAR, and is the premier conference for Mixed and Augmented Reality in the world.

As virtual reality (VR) technology becomes cheaper, higher-quality, and more widely available, it is seeing increasing use in a variety of applications including cultural heritage, real estate, and architecture. A common goal for all these applications is a compelling virtual recreation of a real place. Despite this, there has been very little research into how users perceive and experience such replicated spaces. This article reports the results from a series of three user studies investigating this topic. Results include that the scale of the room and large objects in it are most important for users to perceive the room as real and that non-physical behaviors such as objects floating in air are readily noticeable and have a negative effect even when the errors are small in scale.

Eye-tracking techniques have the potential for estimating driver awareness of road hazards. However, traditional eye-movement measures based on static areas of interest may not capture the unique characteristics of driver eyeglance behavior and challenge the real-time application of the technology on the road. This article proposes a novel method to operationalize driver eye-movement data analysis based on moving objects of interest. A human-subject experiment conducted in a driving simulator demonstrated the potential of the proposed method. Correlation and regression analyses between indirect (i.e., eye-tracking) and direct measures of driver awareness identified some promising variables that feature both spatial and temporal aspects of driver eye-glance behavior relative to objects of interest. Results also suggest that eye-glance behavior might be a promising but insufficient predictor of driver awareness. This work is a preliminary step toward real-time, on-road estimation of driver awareness of road hazards. The proposed method could be further combined with computer-vision techniques such as object recognition to fully automate eye-movement data processing as well as machine learning approaches to improve the accuracy of driver awareness estimation.

Interaction research has been initially focusing on partially and conditionally automated vehicles. Augmented Reality (AR) may provide a promising way to enhance drivers' experience when using autonomous driving (AD) systems. This study sought to gain insights on drivers' subjective assessment of a simulated driving automation system with AR-based support. A driving simulator study was conducted and participants' rating of the AD system in terms of information imparting, nervousness and trust was collected. Cumulative Link Models (CLMs) were developed to investigate the impacts of AR cues, traffic density and intersection complexity on drivers' attitudes towards the presented AD system. Random effects were incorporated in the CLMs to account for the heterogeneity among participants. Results indicated that AR graphic cues could significantly improve drivers' experience by providing advice for their decision-making and mitigating their anxiety and stress. However, the magnitude of AR's effect was impacted by traffic conditions (i.e. diminished at more complex intersections). The study also revealed a strong correlation between self-rated trust and takeover frequency, suggesting takeover and other driving behavior need to be further examined in future studies.

Full windshield displays (WSDs) have the potential to present imagery across the windshield. Current knowledge on display location has not investigated translucent displays at high eccentricities from the driver's forward view. A simulator study (n=26) was conducted aiming to, (a) investigate the effects of Head-Up Display (HUD) location across the entire windshield on driving performance, and (b) better understand how the visual demand for a complex HUD imagery differs from that for a Head-Down Display (HDD). Lane-keeping was poorer when HUD imagery was furthest from the driver (and for the HDD compared to the HUD). Equally, counts of "unacceptable" driving behaviour were greater for displays furthest from the driver's forward view. Furthermore, drivers preferred HUD imagery that was closer to them. The results indicate that HUD evaluations should account for image location, because of how driver gaze location can impact lateral driving performance.

Recent AR research efforts have explored the use of virtual environments to test augmented reality (AR) user interfaces. However, it is yet to be seen what effects the visual fidelity of such virtual environments may have on AR interface assessment, and specifically to what degree assessment results observed in a virtual world would apply to the real world. Automotive AR head-up (HUD) interfaces provide a meaningful application area to examine this problem, especially given that immersive, 3D-graphics-based driving simulators are established tools to examine in-vehicle interfaces safely before testing in real vehicles. In this work, we present an argument that adequately assessing AR interfaces requires a suite of different measures, and that such measures should be considered when debating the appropriateness of virtual environments for AR interface assessment. We present a case study that examines how an AR interface presented via HUD effects driver performance and behavior in different virtual and real environments. Twelve participants completed the study measuring driver task performance, eye gaze behavior and situational awareness during AR guided navigation in low-and high-fidelity virtual simulation, and an on-road environment. Our results suggest that the visual fidelity of the environmental in which an AR interface is assessed, could impact some measures of effectiveness. Discussion is guided by a proposed initial assessment classification for AR user interfaces that may serve to guide future discussions on AR interface evaluation, as well as the suitability of virtual environments for AR assessment.

Automotive manufactures are rapidly developing more advanced in-vehicle systems that seek to provide a driver with more active safety and information in real-time, in particular human machine interfaces (HMIs) using mixed or augmented reality (AR) graphical elements. However, it is difficult to properly test novel AR interfaces in the same way as traditional HMIs via on-road testing. Instead, simulation could likely offer a safer and more financially viable alternative for testing AR HMIs, inconsistent simulation quality may confound HMI research depending on the visual fidelity of each simulation environment. We investigated how visual fidelity in a virtual environment impacts the quality of resulting driver behavior, visual attention, and situational awareness when using the system. We designed two large-scale immersive virtual environments; a “low” graphic fidelity driving simulation representing most current research simulation testbeds and a “high” graphic fidelity environment created in Unreal Engine that represents state of the art graphical presentation. We conducted a user study with 24 participants who navigated a route in a virtual urban environment via direction of AR graphical cues while also monitoring the road scene for pedestrian hazards, and recorded their driving performance, gaze patterns, and subjective feedback via situational awareness survey (SART). Our results show drivers change both their driving and visual behavior depending upon the visual fidelity presented in the virtual scene. We further demonstrate the value of using multi-tiered analysis techniques to more finely examine driver performance and visual attention.

Technical information presentation is evolving from static contents presented on paper or via digital publishing to real-time context-aware contents displayed via virtual and augmented reality devices. We present a Context-Aware Technical Information Management system (CATIM), that dynamically manages (1) what information as well as (2) how information is presented in an augmented reality interface. CATIM acquires context data about activity, operator, and environment, and then based on these data, proposes a dynamic augmented reality output tailored to the current context. The system was successfully implemented and preliminarily evaluated in a case study regarding the maintenance of a hydraulic valve.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present theTVCGpapers from the 18th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2019), held October 14–18 in Beijing, China. ISMAR continues the over 20-year long tradition of IWAR, ISMR, and ISAR, and is undoubtedly the premier conference for mixed and augmented reality in the world.

In augmented reality (AR) environments, information is often distributed between real world and virtual contexts, and often appears at different distances from the user. Therefore, to integrate the information, users must repeatedly switch context and refocus the eyes. To focus at different distances, the user's eyes must accommodate, which when done repeatedly can cause eyestrain and degrade task performance. An experiment was conducted that examined switching context and focal distance between a real and an AR environment, using a text-based visual search task and a monocular optical see-through AR display. Both context switching and focal distance switching resulted in significantly reduced performance. In addition, repeatedly performing the task caused visual fatigue to steadily increase. Performance was particularly poor for virtual text presented at optical infinity, and for target letters that participants tried to read before their eyes had completely accommodated to a new focal distance. The results show that context switching and focal distance switching are important AR user interface design issues.

Augmented reality (AR) head-up displays (HUD) have previously been explored as a potential information delivery system for drivers. In driving scenarios, correct perception of virtual object distance assists with effective use of AR HUDs in safety-critical applications (e.g., collision warnings). AR volumetric displays purportedly offer increased accuracy of distance perception through consistent presentation of oculomotor cues such as vergence and accommodation over traditional displays. For this paper, we investigated volumetric AR displays as a mean of enhancing perception of virtual objects registered to the real world, specifically in terms of distance perception as a result of binocular cues. We designed and ran an experiment where participants controlled and placed virtual objects next to real-world counterparts at ranges between 7-12 meters. We found that the volumetric AR display outperformed a traditional fixed focal plane AR HUD when considering distances 5 meters greater than the traditional display's focal depth.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present theTVCGpapers from the 17th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), held October 16–20 in Munich, Germany. ISMAR continues the 20-year long tradition of IWAR, ISMR, and ISAR, and is undoubtedly the premier conference for mixed and augmented reality in the world.

This article investigates the effects of visual warning presentation methods on human performance in augmented reality (AR) driving. An experimental user study was conducted in a parking lot where participants drove a test vehicle while braking for any cross traffic with assistance from AR visual warnings presented on a monoscopic and volumetric head-up display (HUD). Results showed that monoscopic displays can be as effective as volumetric displays for human performance in AR braking tasks. The experiment also demonstrated the benefits of conformal graphics, which are tightly integrated into the real world, such as their ability to guide drivers' attention and their positive consequences on driver behavior and performance. These findings suggest that conformal graphics presented via monoscopic HUDs can enhance driver performance by leveraging the effectiveness of monocular depth cues. The proposed approaches and methods can be used and further developed by future researchers and practitioners to better understand driver performance in AR as well as inform usability evaluation of future automotive AR applications.

The automotive industry is rapidly developing new in-vehicle technologies that can provide drivers with information to aid awareness and promote quicker response times. Particularly, vehicles with augmented reality (AR) graphics delivered via head-up displays (HUDs) are nearing mainstream commercial feasibility and will be widely implemented over the next decade. Though AR graphics have been shown to provide tangible benefits to drivers in scenarios like forward collision warnings and navigation, they also create many new perceptual and sensory issues for drivers. For some time now, designers have focused on increasing the realism and quality of virtual graphics delivered via HUDs, and recently have begun testing more advanced 3D HUD systems that deliver volumetric spatial information to drivers. However, the realization of volumetric graphics adds further complexity to the design and delivery of AR cues, and moreover, parameters in this new design space must be clearly and operationally defined and explored. In this work, we present two user studies that examine how driver performance and visual attention are affected when using fixed and animated AR HUD interface design approaches in driving scenarios that require top-down and bottom-up cognitive processing. Results demonstrate that animated design approaches can produce some driving gains (e.g., in goal-directed navigation tasks) but often come at the cost of response time and distance. Our discussion yields AR HUD design recommendations and challenges some of the existing assumptions of world-fixed conformal graphic approaches to design.

In-vehicle support systems have the potential to reduce the risk of pedestrian collisions and promote gains in braking performance and visual attention when scanning for threats on the road. This study investigated changes in driver behavior in pedestrian collision scenarios with increasing urgency while using varying levels of pedestrian alert system (PAS) support in a medium fidelity driving simulator. During pedestrian collision scenarios, we assessed drivers' eye gaze behavior, braking performance, and acceptance ratings across three levels of PAS and four levels of increasing urgency, defined as time to collision (TTC). Results suggest that both audio- and visually-based PAS do not produce gains in the localization of pedestrians, but can nevertheless improve drivers' braking performance in events where pedestrians may pose a threat. Our results further suggest that drivers exhibit both innate and direct confidence in visually-based PAS support, despite no concurrent gains in visual scanning performance.

Technological advances in Head-Up Displays (HUDs) have renewed vehicle manufacturer interest. Recent research links time with eyes off of the road to increased chance of accidents, a problem that could be diminished when using HUDs. Before HUDs can be safely integrated, there must be proven methods of assessing these displays. NHTSA established guidelines for assessing in-vehicle tasks; however, these have not been examined with HUDs, differ from traditional Head-Down Displays (HDDs). This study followed most of the NHTSA guidelines for the Eye Glance Test for a visual text search task while tracking driving and secondary task performance measures. The HUD performed worse on the NHTSA eye glance test than the HDD did; however, the driving performance measures were superior when driving with the HUD. There were no significant differences in the secondary task performance between the two displays. Therefore, the NHTSA standard may not adequately assess HUDs in vehicles.

The overall purpose of our research is to identify unique benefits and challenges in gamifying personalized augmented reality experiences in K -- 12 education for minority students. Gamification is applying game-like elements and principles in a non-gaming environment with the desired outcome of increasing engagement, motivation and learning. Augmented reality (AR) technologies project virtual objects onto the real world. Personalized AR potentially creates new levels of engagement for students while simultaneously accounting for their skill level, strengths, challenges and personal as well as cultural needs to increase learning. Using gamification in augmented or virtual environments is not a new phenomenon for researchers, however, the model presented in this work will leverage gamified AR experiences while incorporating cultural relevance to bring a new perspective for keeping minority students engaged and increasing learning outcomes.

In this work, we present a depth-based solution to multi-level menus for selection and manipulation of virtual objects using freehand gestures. Navigation between and through menus is performed using three gesture states that utilize X, Y translations of the finger with boundary crossing. Although presented in a single context, this menu structure can be applied to a myriad of domains requiring several levels of menu data, and serves to supplement existing and emerging menu design for augmented, virtual, and mixed-reality applications.

This study explored the effects that augmented reality graphics have on a drivers' distance estimation/depth perception in an augmented side view mirror. The study was conducted both inside a simulator and outside in a parking lot. Sixteen participants partook in the study, 8 in the simulator and 8 in the test vehicle outside. Distance judgments were compared across four side view mirror conditions both simulator and outdoor scenarios. Results show some opportunities for AR to improve depth judgments, but further analysis is necessary.

Ecological interface design (EID) has the opportunity to complement current approaches for augmented reality (AR) interface design by considering human-environment interaction and leveraging the inherent benefit of AR interfaces: conformal graphics. This work applies EID to design a novel interface for pedestrian collision warning for an automotive AR head-up display (HUD). Our initial usability evaluation shows potential benefits of incorporating EID into AR interface design.

This study investigated the differences between head-up display (HUD) and head-down display (HDD) systems in vehicles with the ultimate goal of increasing driver performance and safety. Our initial focus was on comparing drivers' visual task performance (and associated driving behavior) while using a HUD and HDD. In a medium-fidelity driving simulator, 16 experienced drivers performed two types of tasks, a structured (text) and a semi-structured (grid) visual search task while driving. They first completed a baseline drive using a driving simulator with no secondary tasks followed by four drives under various conditions: HUD-Text, HUD-Grid, HDD-Text, and HDD-Grid. The study returned mixed results. HUDs generally contributed to more secondary task errors, but overall, users still preferred to use HUDs over HDDs. The response time to complete tasks was unevenly affected by display type. While there was no difference between response times in the grid tasks (HDD vs. HUD), the HUD was associated with significantly faster task performance as compared to the text tasks. These results suggest that HUDs may offer advantages to drivers in some task scenarios but more research is needed to understand under what conditions they do not add value.

Recent research indicates that users consistently underestimate depth judgments to Augmented Reality (AR) graphics when viewed through optical see-through displays. However, to our knowledge, little work has examined how AR graphics may affect depth judgments of real world objects that have been overlaid or annotated with AR graphics. This study explores whether AR graphics have directional effects on users' depth perception of real-world objects. Participants were shown an initial target location and were then asked to indicate when a moving target was perceived to be at the previously specified target location. Each participant experienced three different display conditions: no AR visual display (control), a conformal AR graphic overlaid on the pedestrian via head-up display (HUD), and the same graphic presented on a tablet physically located on the pedestrian. The judged distance from the correct location was recorded, and after each trial, participants' confidence in determining the correct distance was captured. Across all conditions, participants underestimated the distance of the physical object. Greater variability was observed in the accuracy of distance judgments under the AR HUD condition relative to the other two display conditions. In addition, participant confidence levels were significantly lower in the AR HUD condition.

Recent research indicates that users consistently underestimate depth judgments to Augmented Reality (AR) graphics when viewed through optical see-through displays. However, to our knowledge, little work has examined how AR graphics may affect depth judgments of real world objects that have been overlaid or annotated with AR graphics. This study begins a preliminary analysis whether AR graphics have directional effects on users' depth perception of real-world objects, as might be experienced in vehicle driving scenarios (e.g., as viewed via an optical see-through head-up display or HUD). Twenty-four participants were asked to judge the depth of a physical pedestrian proxy figure moving towards them at a constant rate of 1 meter/second. Participants were shown an initial target location that varied in distance from 11 to 20 m and were then asked to press a button to indicate when the moving target was perceived to be at the previously specified target location. Each participant experienced three different display conditions: no AR visual display (control), a conformal AR graphic overlaid on the pedestrian via a HUD, and the same graphic presented on a tablet physically located on the pedestrian. Participants completed 10 trials (one for each target distance between 11 and 20 inclusive) per display condition for a total of 30 trials per participant. The judged distance from the correct location was recorded, and after each trial, participants' confidence in determining the correct distance was captured. Across all conditions, participants underestimated the distance of the physical object consistent with existing literature. Greater variability was observed in the accuracy of distance judgments under the AR HUD condition relative to the other two display conditions. In addition, participant confidence levels were considerably lower in the AR HUD condition.

This tutorial is for researchers and engineers, working in the field of Virtual and Augmented Reality, who wish to conduct user-based experiments and/or evaluations for assessing usability. We propose a full-day tutorial presenting both quantitative and qualitative approaches to conducting human-subject experiments. It will cover (1) the basic principles of experimental design and analysis, with an emphasis on human-subject experiments in AR (Swan), and (2) qualitative studies (e.g., formative evaluation methods) for assessing and improving AR user interfaces and user interaction along with lessons learned from conducting many user-based studies (Gabbard). Swan, Gabbard, and other co-presenters have taught pre-cursor versions of this tutorial 11 previous times at IEEE Virtual Reality, IEEE Visualization, and ISMAR. This tutorial was most recently given at ISMAR 2012, where we included updated examples from our research and further expanded upon qualitative approaches for assessing usability and lessons learned from conducting studies. We both have current, active AR human-subject research projects, and if this tutorial is accepted to be presented at VR 2014, we will discuss some of these projects as case studies.

As the automotive industry moves toward the car of the future, technology companies are developing cutting-edge systems, in vehicle and out, that aim to make driving safer, more pleasant, and more convenient. While we are already seeing some successful video-based augmented reality (AR) auxiliary displays (e.g., center-mounted backup aid systems), the application opportunities of optical see-through AR as presented on a drivers' windshield are yet to be fully tapped; nor are the visual perceptual and attention challenges fully understood. As we race to field AR applications in transportation, we should first consider the perceptual and distraction issues that are known in both the AR and transportation communities, with a focus on the unique and intersecting aspects for driving applications. This paper describes the some opportunities and driver challenges associated with AR applications in the automotive domain. We first present a basic research space to assist in these inquiries, which delineates head-mounted from heads-up and center-mounted displays; video from optical see-through displays; and world-fixed from screen-fixed AR graphics. We then address benefits of AR related to primary, secondary, and tertiary driver tasks as well as driver perception and cognition challenges inherent in automotive AR systems.

Crash warning systems are designed to help avoid vehicle accidents by notifying drivers of potential hazards. In typical crash warning systems, primary warning information is provided through visual, audible and/or haptic cues. In general, the use of crash warning systems results in safer driving. However, driver vehicle interfaces that employ visual warning elements, such as text messages appearing on the center console display and blindspot detection icons in side view mirrors, may take drivers' eyes off the road momentarily, and may lead to divided attention, distracted driving and increased crash risks. To address this, we propose an augmented reality (AR) head-up display interface for crash warning systems that displays visual cues on the drivers' view of the road, with the ultimate goal of increasing driver awareness and safety. In this paper, we describe a simulator-based comparative user study to begin understanding the effect of AR interface design features on driver performance, mental workload, and preferences. Our results support the hypothesis that head-up AR display interfaces for crash warning systems have potential safety benefits and a high likelihood of driver acceptance.

Over the past decades, augmented reality (AR) has seen vast improvements in graphic displays and mobile tracking capabilities. To our knowledge, handheld AR researcher in text legibility has mostly focused on algorithms for overlaying screen-aligned annotations; no one has yet employed a series of user-based studies to systematically investigate text legibility in handheld AR applications. In this preliminary work, we survey current approaches to text depictions in AR applications and the literature. We have found no single consistent form of displaying text, but see some basic trends emerging. This initial work lays the foundation for examining different text styles in various environments in order to create guidelines for effective text drawing approaches. By assessing current strategies for rendering text labels in handheld AR, we can catalog drawing styles for use in follow-on user-based studies.

When viewing augmented reality (AR) through an optical see-through head-mounted display (oHMD), the colors of AR elements become washed out as the luminance of the background increases. In addition, the colors of AR elements can shift and change depending on the color of the background that the AR elements are seen against. Both of these phenomena result from the way the color of an AR element blends with the color of the real-world background that the AR element is seen against. Although the fact that this color blending occurs is generally known, to date it has not been systematically studied or quantified. In this work, we sought to quantify this color blending by building an apparatus that allows precise and repeatable color measurements, and then conducting an empirical engineering study.

This tutorial is for researchers and engineers, working in the field of Virtual Reality (VR) and Augmented Reality (AR), who wish to conduct user-based experiments with a specific aim of promoting both traditional quantitative human-subject experiments and qualitative methods for assessing usability. This tutorial is for a full-day tutorial presenting both quantitative and qualitative approaches to conducting human-subject experiments. It covers (1) the basic principles of experimental design and analysis, with an emphasis on human-subject experiments in VR/AR; (2) qualitative studies (e.g., formative evaluation methods) for assessing and improving VR/AR user interfaces and user interaction along with lessons learned from conducting many user-based studies; and (3) a “journalistic approach” to measuring human performance that organizes the activity around questions such as “Who? What? When? Where? How? and Why?”.

MOTIVATION: Infectious disease research is generating an increasing amount of disparate data on pathogenic systems. There is a growing need for resources that effectively integrate, analyze, deliver and visualize these data, both to improve our understanding of infectious diseases and to facilitate the development of strategies for disease control and prevention. RESULTS: We have developed Disease View, an online host-pathogen resource that enables infectious disease-centric access, analysis and visualization of host-pathogen interactions. In this resource, we associate infectious diseases with corresponding pathogens, provide information on pathogens, pathogen virulence genes and the genetic and chemical evidences for the human genes that are associated with the diseases. We also deliver the relationships between pathogens, genes and diseases in an interactive graph and provide the geolocation reports of associated diseases around the globe in real time. Unlike many other resources, we have applied an iterative, user-centered design process to the entire resource development, including data acquisition, analysis and visualization. AVAILABILITY AND IMPLEMENTATION: Freely available at http://www.patricbrc.org; all major web browsers supported. CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

It is well-documented that natural lighting conditions and real-world backgrounds affect the usability of optical see-through augmented reality (AR) displays in outdoor environments. In many cases, outdoor environmental conditions can dramatically alter users' color perception of user interface elements, by for example, washing out text or icon colors. As a result, users' semantic interpretation of interface elements can be compromised, rendering interface designs useless or counter-productive — an especially critical problem in application domains where color encoding is critical, such as military or medical visualization. In this paper, we present our experiences designing and constructing an optical AR testbed that emulates outdoor lighting conditions and allows us to measure the combined color of real-world backgrounds and virtual colors as projected through an optical see-through display. We present a formalization of color blending in AR, which supports further research on perceived color in AR displays. We describe an engineering study where we measure the color of light that reaches an optical see-through display user's eye under systematically varied virtual and real-world conditions. Our results further quantify the effect of lighting and background color on the color of virtual graphics, and specifically quantify how virtual colors change based on different real-world backgrounds.

A major challenge, and thus opportunity, in the field of human-computer interaction and specifically usability engineering is designing effective user interfaces for emerging technologies that have no established design guidelines or interaction metaphors or introduce completely new ways for users to perceive and interact with technology and the world around them. Clearly, augmented reality is one such emerging technology. We propose a usability engineering approach that employs user-based studies to inform design, by iteratively inserting a series of user-based studies into a traditional usability engineering lifecycle to better inform initial user interface designs. We present an exemplar user-based study conducted to gain insight into how users perceive text in outdoor augmented reality settings and to derive implications for design in outdoor augmented reality. We also describe lessons learned from our experiences conducting user-based studies as part of the design process.

A challenge in presenting augmenting information in outdoor augmented reality (AR) settings lies in the broad range of uncontrollable environmental conditions that may be present, specifically large-scale fluctuations in natural lighting and wide variations in likely backgrounds or objects in the scene. In this paper, we present a active AR testbed that samples the user's field of view, and collects outdoor illuminance values at the participant's position. The main contribution presented herein is a user-based study (conducted using the testbed) that examined the effects on user performance of four outdoor background textures, four text colors, three text drawing styles, and two text drawing style algorithms for a text identification task using an optical, see-through AR system. We report significant effects for all these variables, and discuss design guidelines and ideas for future work

A fundamental problem in optical, see-through augmented reality (AR) is characterizing how it affects the perception of spatial layout and depth. This problem is important because AR system developers need to both place graphics in arbitrary spatial relationships with real-world objects, and to know that users will perceive them in the same relationships. Furthermore, AR makes possible enhanced perceptual techniques that have no real-world equivalent, such as x-ray vision, where AR users are supposed to perceive graphics as being located behind opaque surfaces. This paper reviews and discusses techniques for measuring egocentric depth judgments in both virtual and augmented environments. It then describes a perceptual matching task and experimental design for measuring egocentric AR depth judgments at medium- and far-field distances of 5 to 45 meters. The experiment studied the effect of field of view, the x-ray vision condition, multiple distances, and practice on the task. The paper relates some of the findings to the well-known problem of depth underestimation in virtual environments, and further reports evidence for a switch in bias, from underestimating to overestimating the distance of AR-presented graphics, at 23 meters. It also gives a quantification of how much more difficult the x-ray vision condition makes the task, and then concludes with ideas for improving the experimental methodology.

A challenge in presenting augmenting information in outdoor augmented reality (AR) settings lies in the broad range of uncontrollable environmental conditions that may be present, specifically large-scale fluctuations in natural lighting and wide variations in likely backgrounds or objects in the scene. In this paper, we present a user-based study which examined the effects of outdoor background textures, changing outdoor illuminance values, and text drawing styles on user performance of a text identification task with an optical, see-through augmented reality system. We report significant effects for all of these variables, and discuss design guidelines and ideas for future work.

Provides an abstract of the tutorial presentation and a brief professional biography of the presenter. The complete presentation was not made available for publication as part of the conference proceedings.

A useful function of augmented reality (AR) systems is their ability to visualize occluded infrastructure directly in a user's view of the environment. This is especially important for our application context, which utilizes mobile AR for navigation and other operations in an urban environment. A key problem in the AR field is how to best depict occluded objects in such a way that the viewer can correctly infer the depth relationships between different physical and virtual objects. Showing a single occluded object with no depth context presents an ambiguous picture to the user. But showing all occluded objects in the environments leads to the "Superman's X-ray vision" problem, in which the user sees too much information to make sense of the depth relationships of objects. Our efforts differ qualitatively from previous work in AR occlusion, because our application domain involves far-field occluded objects, which are tens of meters distant from the user. Previous work has focused on near-field occluded objects, which are within or just beyond arm's reach, and which use different perceptual cues. We designed and evaluated a number of sets of display attributes. We then conducted a user study to determine which representations best express occlusion relationships among far-field objects. We identify a drawing style and opacity settings that enable the user to accurately interpret three layers of occluded objects, even in the absence of perspective constraints.

We present a comparative study of user performance with tasks involving navigation, visual search, and geometric manipulation, in a map-based battlefield visualization virtual environment (VE). Specifically, our experiment compared user performance of the same task across four different VE platforms: desktop, cave, workbench, and wall. Independent variables were platform type, stereopsis (stereo, mono), movement control mode (rate, position), and frame of reference (egocentric, exocentric). Overall results showed that users performed tasks fastest using the desktop and slowest using the workbench. Other results are detailed in the article. Notable is that we designed our task in an application context, with tasking much closer to how users would actually use a real-world battlefield visualization system. This is very uncommon for comparative studies, which are usually designed with abstract tasks to minimize variance. This is, we believe, one of the first and most complex studies to comparatively examine, in an application context, this many key variables affecting VE user interface design.

The ever-increasing power of computers and hardware rendering systems has, to date, primarily motivated the creation of visually rich and perceptually realistic virtual environment (VE) applications. Comparatively very little effort has been expended on the user interaction components of VEs. As a result, VE user interfaces are often poorly designed and are rarely evaluated with users. Although usability engineering is a newly emerging facet of VE development, user-centered design and usability evaluation in VEs as a practice still lags far behind what is needed. This paper presents a structured, iterative approach for the user-centered design and evaluation of VE user interaction. This approach consists of the iterative use of expert heuristic evaluation, followed by formative usability evaluation, followed by summative evaluation. We describe our application of this approach to a real-world VE for battlefield visualization, describe the resulting series of design iterations, and present evidence that this approach provides a cost-effective strategy for assessing and iteratively improving user interaction design in VEs. This paper is among the first to report applying an iterative, structured, user-centered design and evaluation approach to VE user interaction design.