The development of eXtended Reality (XR) technologies has led to an increase in the use of virtual spaces to help support collaboration. However, collaboration between physical and virtual environments is often considered solely with respect to how successfully the XR technology has been used to integrate virtual and physical worlds. In contrast, this study explored how participants collaborated within a Blended Reality (BR) system, with the overall aim of identifying the benefits and challenges associated with BR systems during group collaboration. Analysis of interview and video observations highlighted various beneficial affordances and considerations associated with different elements of the BR system. There were benefits associated with the physical, virtual and blended space as well as considerations relating to the task design and system design. Further work should seek to determine how to design BR spaces to better suit the collaborative context.

Augmented reality (AR) offers new ways to visualize information on-the-go. As noted in related work, AR graphics presented via optical see-through AR displays are particularly prone to color blending, whereby intended graphic colors may be perceptually altered by real-world backgrounds, ultimately degrading usability. This work adds to this body of knowledge by presenting a methodology for assessing AR interface color robustness, as quantitatively measured via shifts in the CIE color space, and qualitatively assessed in terms of users' perceived color name. We conducted a human factors study where twelve participants examined eight AR colors atop three real-world backgrounds as viewed through an in-vehicle AR head-up display (HUD); a type of optical see-through display used to project driving-related information atop the forward-looking road scene. Participants completed visual search tasks, matched the perceived AR HUD color against the WCS color palette, and verbally named the perceived color. We present analysis that suggests blue, green, and yellow AR colors are relatively robust, while red and brown are not, and discuss the impact of chromaticity shift and dispersion on outdoor AR interface design. While this work presents a case study in transportation, the methodology is applicable to a wide range of AR displays in many application domains and settings.

In a longitudinal study, 49 drivers undertook a commute-style journey, with part of the route supporting level-3 automation, over five consecutive days. Bespoke HMIs were provided to keep drivers in-the-loop during automation, and help them regain situational-awareness (SA) during handovers, in a 2×2 between-subjects design. Drivers demonstrated high levels of trust from the outset, delegating control to the vehicle (when available) and directing attention to their own activities/devices. Ratings of trust and technology acceptance increased during the week -- even following an unexpected, emergency handover on day four -- with the highest ratings recorded on day five. High levels of lateral instability were observed immediately following takeovers, although improvements were noted during the week and following the provision of SA-enhancing hand-over advice. Results demonstrate benefits associated with novel HMI designs to keep drivers in-the-loop and improve takeover performance, as well as the necessity of multiple exposures during the evaluation of future, immersive technologies.

Recognising that one of the aims of conversation is to build, maintain and strengthen positive relationships with others, the study explores whether passengers in an autonomous vehicle display similar behaviour during transactions with an on-board conversational agent-interface; moreover, whether related attributes (e.g. trust) transcend to the vehicle itself. Employing a counterbalanced, within-subjects design, thirty-four participants were transported in a self-driving pod using an expansive testing arena. Participants undertook three journeys with an anthropomorphic agent-interlocutor (via Wizard-of-Oz), a voice-command interface, or a traditional touch-surface; each delivered equivalent task-related information. Results show that the agent-interlocutor was the most preferred interface, attracting the highest ratings of trust, and significantly enhancing the pleasure and sense of control over the journey experience, despite the inclusion of 'trust challenges' as part of the design. The findings can help support the design and development of in-vehicle agent-based voice interfaces to enhance trust and user experience in autonomous cars.

Employing a 2x2 within-subjects design, forty-eight experienced drivers (28 male, 20 female) undertook repeated button selection and 'slider-bar' manipulation tasks, to compare a traditional touchscreen with a virtual mid-air gesture interface in a driving simulator. Both interfaces were tested with and without haptic feedback generated using ultrasound. Results show that combining gestures with mid-air haptic feedback was particularly promising, reducing the number of long glances and mean off-road glance time associated with the in-vehicle tasks. For slider-bar tasks in particular, gestures-with-haptics was also associated with the shortest interaction times, highest number of correct responses and least 'overshoots', and was favoured by participants. In contrast, for button-selection tasks, the touchscreen was most popular, enabling the highest accuracy and quickest responses, particularly when combined with haptic feedback to guide interactions, although this also increased visual demand. The study shows clear potential for gestures with mid-air ultrasonic haptic feedback in the automotive domain.

Touchscreen human–machine interfaces (HMIs) are commonly employed as the primary control interface and touch-point of vehicles. However, there has been very little theoretical work to model the demand associated with such devices in the automotive domain. Instead, touchscreen HMIs intended for deployment within vehicles tend to undergo time-consuming and expensive empirical testing and user trials, typically requiring fully functioning prototypes, test rigs, and extensive experimental protocols. While such testing is invaluable and must remain within the normal design/development cycle, there are clear benefits, both fiscal and practical, to the theoretical modeling of human performance. We describe the development of a preliminary model of human performance that makes a priori predictions of the visual demand (total glance time, number of glances, and mean glance duration) elicited by in-vehicle touchscreen HMI designs, when used concurrently with driving. The model incorporates information theoretic components based on Hick–Hyman Law decision/search time and Fitts’ Law pointing time and considers anticipation afforded by structuring and repeated exposure to an interface. Encouraging validation results, obtained by applying the model to a real-world prototype touchscreen HMI, suggest that it may provide an effective design and evaluation tool, capable of making valuable predictions regarding the limits of visual demand/performance associated with in-vehicle HMIs, much earlier in the design cycle than traditional design evaluation techniques. Further validation work is required to explore the behavior associated with more complex tasks requiring multiple screen interactions, as well as other HMI design elements and interaction techniques. Results are discussed in the context of facilitating the design of in-vehicle touchscreen HMI to minimize visual demand.

Unencumbered by the need to provide permanent manual control, future autonomous vehicles may be absent of traditional driving elements, such as a steering wheel and foot-pedals. While this provides additional space/comfort for drivers, an alternative device/s, that enables quick and effective manual control recovery, may still be required. Twenty participants, comprising an equal number of 'Drivers' and 'Non-Drivers', undertook two drives utilising a medium-fidelity simulator in which the original steering wheel and pedal-set were removed/deactivated. Participants controlled the vehicle using a centrally-mounted joystick in 2 scenarios: following an unexpected take-over request and during an extended period of manual driving. Results show that both groups rated the technology positively and were generally capable of manoeuvring the vehicle. Nevertheless, the groups differed in the way that performance was effected. For example, significant differences were evident in visual behaviour, resulting in 50% (n=5) of Non-Drivers colliding with another vehicle following transfer of control.

In-vehicle support systems have the potential to reduce the risk of pedestrian collisions and promote gains in braking performance and visual attention when scanning for threats on the road. This study investigated changes in driver behavior in pedestrian collision scenarios with increasing urgency while using varying levels of pedestrian alert system (PAS) support in a medium fidelity driving simulator. During pedestrian collision scenarios, we assessed drivers' eye gaze behavior, braking performance, and acceptance ratings across three levels of PAS and four levels of increasing urgency, defined as time to collision (TTC). Results suggest that both audio- and visually-based PAS do not produce gains in the localization of pedestrians, but can nevertheless improve drivers' braking performance in events where pedestrians may pose a threat. Our results further suggest that drivers exhibit both innate and direct confidence in visually-based PAS support, despite no concurrent gains in visual scanning performance.

Given the proliferation of digital assistants in everyday mobile technology, it appears inevitable that next generation vehicles will be embodied by similar agents, offering engaging, natural language interactions. However, speech can be cognitively captivating. It is therefore important to understand the demand that such interfaces may place on drivers. Twenty-five participants undertook four drives (counterbalanced), in a medium-fidelity driving simulator: 1. Interacting with a state-of-the-art digital driving assistant ('DDA') (presented using Wizard-of-Oz); 2. Engaged in a hands-free mobile phone conversation; 3. Undertaking the delayed-digit recall ('2-back') task and 4. With no secondary task (baseline). Physiological arousal, subjective workload assessment, tactile detection task (TDT) and driving performance measures consistently revealed the '2-back' drive as the most cognitively demanding (highest workload, poorest TDT performance). Mobile phone and DDA conditions were largely equivalent, attracting low/medium cognitive workload. Findings are discussed in the context of designing in-vehicle natural language interfaces to mitigate cognitive demand.

Touchscreen Human-Machine Interfaces (HMIs) inherently demand some visual attention. By employing a secondary device, to work in unison with a touchscreen, some of this demand may be alleviated. In a medium-fidelity driving simulator, twenty-four drivers completed four typical in-vehicle tasks, utilising each of four devices -- touchscreen, rotary controller, steering wheel controls and touchpad (counterbalanced). Participants were then able to combine devices during a final 'free-choice' drive. Visual behaviour, driving/task performance and subjective ratings (workload, emotional response, preferences), indicated that in isolation the touchscreen was the most preferred/least demanding to use. In contrast, the touchpad was least preferred/most demanding, whereas the rotary controller and steering wheel controls were largely comparable across most measures. When provided with 'free-choice', the rotary controller and steering wheel controls presented as the most popular candidates, although this was task-dependent. Further work is required to explore these devices in greater depth and during extended periods of testing.

This study explored the effects that augmented reality graphics have on a drivers' distance estimation/depth perception in an augmented side view mirror. The study was conducted both inside a simulator and outside in a parking lot. Sixteen participants partook in the study, 8 in the simulator and 8 in the test vehicle outside. Distance judgments were compared across four side view mirror conditions both simulator and outdoor scenarios. Results show some opportunities for AR to improve depth judgments, but further analysis is necessary.

Augmenting navigation systems with landmarks has been posited as a method of improving the effectiveness of the technology and enhancing drivers’ engagement with the environment. However, good navigational landmarks are both laborious to collect and difficult to define. This research aimed to devise a game concept which could be played by passengers in cars, and would collect useful landmark data as a by-product. The paper describes how a virtual graffiti tagging game concept was created and tested during on-road trials with 38 participants. The data collected in the road trials were then validated using a survey, in which 100 respondents assessed the quality of the landmarks collected and their potential for reuse in navigation applications. Players of the game displayed a consensus in choosing where to place their graffiti tags, with over 30% of players selecting the same object to tag in 10 of the 12 locations. Furthermore, significant correlation was found between how highly landmarks were rated in the survey and how frequently they were tagged during the game. The research provides evidence that using crowdsourcing games to collect landmarks does not require large numbers of people, or extensive coverage of an area, to produce suitable candidate landmarks for navigation.

Using landmark-based navigation can greatly improve drivers' route-finding performance. Previous research in this area has tended to focus on the inclusion of text or icon-based landmark information utilising dashboard-mounted displays. In contrast, we present landmark-based navigation information using a Head-Up Display (HUD). A major issue with using landmarks for navigation is their inherent variability in quality, with many 'poor' candidates that are not easily identifiable or communicable. A proposed solution to improve the usefulness and utility of such landmarks is to highlight/enhance them using augmented reality (AR). Twenty participants undertook four drives in a driving simulator utilising an AR navigation system presented on a HUD. Participants were provided navigational instructions presented as either conventional distance-to-turn information, on-road arrows or augmented landmark information (arrow highlighting or box enclosing landmark adjacent to the required turning). Participants demonstrated significant performance improvements while using the AR landmark 'box' presentations compared to conventional distance-to-turn information, with response times and success rates enhanced by 43.1% and 26.2%, respectively. Moreover, drivers reported a significant reduction in workload when using the AR landmark 'boxes'. We conclude that there are significant benefits to providing navigational information utilising AR presentations of landmark information on a HUD.

Finger-touch based interactions with capacitive touchscreen devices in cars are becoming increasingly common. As such, it is critical to understand the basic human factors of target acquisition (pointing/touching) in this context. We describe a simulator study that aims to build Fitts' Law relationships for predicting the visual demands (mean glance duration and total glance time) associated with finger-touch pointing tasks as a function of target size, location and design. The observed data show strong linear relationships between all visual demand measures and Fitts' index of difficulty, indicating that finger-touch pointing tasks conform well with the Fitts' Law model under conditions of divided attention. The derived equations are discussed in the context of designing in-vehicle touchscreen interfaces for minimal visual demand.

Driver distraction is a major contributor to road traffic accidents. Consequently, NHTSA have published detailed guidelines intended to discourage the introduction of excessively distracting electronic devices in vehicles. However, concerns have been raised regarding the theoretical basis and ecological validity of the NHTSA Eye Glance Testing Using a Driving Simulator (EGDS) testing protocol. In particular, it has been noted that the driving scenario is over-simplified and acceptance is predicated on absolute measures determined from a poorly defined and out-dated reference task ('radio-tuning'). Using a medium-fidelity driving simulator, a new prototype touchscreen interface was evaluated in full compliance with the NHTSA EGDS test protocol. As a comparison, the same interface was then tested using a well-established ('standard') driving scenario. Results show that the different scenarios elicited significantly different visual behaviour with drivers placing more emphasis (extended TEORT and MGD) on secondary task performance during the NHTSA scenario. Furthermore, driving performance measures indicated higher levels of 'presence' associated with the 'standard' environment. Conclusions are drawn regarding the effect of primary task demand on secondary task performance and NHTSA-compliance, and the validity of the proposed test scenario and reference task.

Occlusion is a popular technique used to evaluate the visual demand associated with secondary tasks/devices in a driving context offering a low cost, highly accessible alternative to driving simulators and on-road studies. Several standardised occlusion test protocols have been published, most notably by NHTSA and ISO. These differ significantly in terms of how many participants are deemed to be sufficient in order to elicit statistically representative behaviour, and thus impose different 'costs' on incumbent organisations. A NHTSA-compliant study investigated three navigation-related tasks using a smartphone app (APP) and portable navigation device (PND). As a comparison, 1000 iterations of 10 participant ISO sample groups were extracted from the NHTSA 24-participant cohort and analysed in accordance with the same measures. Results obtained using all 10-participant ISO groups were consistent with the NHTSA findings, indicating that both standards would support the same conclusions regarding relative performance. Applying NHTSA's acceptance criteria, it is evident that, in most cases, recommendations based on the smaller ISO samples would be the same as those obtained from the full NHTSA cohort. However, only 95.5% conformance was observed for task 3 using the PND, suggesting a small risk that different conclusions could be drawn for this particular device/task if a smaller number of participants were used. Given the lower inherent costs associated with the ISO protocol, we thus conclude that this may be better suited for relative/formative assessment, whereas the NHTSA protocol may be more appropriate for summative evaluations.

Touch screens are increasingly used within modern vehicles, providing the potential for a range of gestures to facilitate interaction under divided attention conditions. This paper describes a study aiming to understand how drivers naturally make swipe gestures in a vehicle context when compared with a stationary setting. Twenty experienced drivers were requested to undertake a swipe gesture on a touch screen in a manner they felt was appropriate to execute a wide range of activate/deactivate, increase/decrease and next/previous tasks. All participants undertook the tasks when either driving within a right-hand drive, medium-fidelity simulator or whilst sitting stationary. Consensus emerged in the direction of swipes made for a relatively small number of increase/decrease and next/previous tasks, particularly related to playing music. The physical action of a swipe made in different directions was found to affect the length and speed of the gesture. Finally, swipes were typically made more slowly in the driving situation, reflecting the reduced resources available in this context and/or the handedness of the participants. Conclusions are drawn regarding the future design of swipe gestures for interacting with in-vehicle touch screens.