Visually searching for objects is an everyday task. In many contexts, people must visually search for multiple objects at the same time while avoiding distractor objects, such as triage during a mass casualty incident. While many prior augmented reality (AR) and virtual reality (VR) studies have investigated cues to aid in visual search tasks, few have investigated cues in contexts involving multiple targets and distractors with a full 360° effective field of regard (EFOR). Individually, multiple targets, distractors, and a full 360° EFOR each add complexity to visual search; when combined, they compound the difficulty even further. In this paper, we present such a study that compares three common types of visual cues (2D Wedge, 3D Arrow, and Gaze Line) to a baseline condition with no cueing for a 360° visual search task. Our results reinforce the importance of providing some type of cue, with the Gaze Line design being particularly beneficial. We discuss the potential implications of these findings for designing cues specifically for such complex visual search tasks.

This study examined discretionary automation – the choice to engage optional, automated support. In a dynamic decision-making task, after manual and aided trials, participants chose whether to engage a machine learning decision aid. The aid improved performance but participants frequently disused automation and lagged in compliance. Experiment 2 forced greater evidence accumulation, improving compliance, automation use, and performance. Experiment 3 required an initial judgment coupled with subsequent evidence accumulation. This reduced automation use but retained higher levels of compliance and performance. Across experiments, trust in automation and self-confidence influenced use decisions, but task difficulty had little impact. Implications for human-automation systems are discussed.

Despite knowing exactly what an object looks like, searching for it in a person's visual field is a time-consuming and error-prone experience. In Augmented Reality systems, new algorithms are proposed to speed up search time and reduce human errors. However, these algorithms might not always provide 100% accurate visual cues, which might affect users' perceived reliability of the algorithm and, thus, search performance. Here, we examined the detrimental effects of automation bias caused by imperfect cues presented in the Augmented Reality head-mounted display using the YOLOv5 machine learning model. 53 participants in the two groups received either 100% accurate visual cues or 88.9% accurate visual cues. Their performance was compared with the control condition, which did not include any additional cues. The results show how cueing may increase performance and shorten search times. The results also showed that performance with imperfect automation was much worse than perfect automation and that, consistent with automation bias, participants were frequently enticed by incorrect cues.

While the visual search for targets in a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies by helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective consequences following automation errors. A total of 56 participants searched a three-dimensional environment containing 48 objects in a room, in order to locate a target object that was viewed prior to each trial. They searched either unaided or assisted by one of the three HMD types of cues: an arrow pointing to the target, a plan-view minimap highlighting the target, and a constantly visible icon depicting the appearance of the target object. The cue was incorrect in 17% of the trials for one group of participants and 100% correct for the second group. Through both analysis and modeling of both search speed and accuracy, the results indicated that the arrow and minimap cues depicting location information were more effective than the icon cue depicting visual appearance, both overall, and when the cue was correct. However, there was a tradeoff on the infrequent occasions when the cue erred. The most effective AR-based cue led to a greater automation bias in which the cue was more often blindly followed without careful examination of the raw images. The results speak to the benefits of AR and the need to examine potential costs when AR-conveyed information may be incorrect because of imperfectly reliable systems.

With changing technology and improving understanding of human neural mechanisms, the point is rapidly approaching where human-computer interaction could integrate information based on real-time cognitive state detection to adapt input to the user. This will result in a novel closed-loop system around a human operator. The shift to interactions around a closed-loop has the potential to produce fundamental changes in system performance of even well-understood open-loop scenarios. This article presents modeling based on engineering control systems theory that offers insight into such closed-loop systems. The model shows how dynamic instability can result from introducing feedback within a system and provides some methods that can be applied to remove such instability and optimize performance. The authors also examine the robustness of the closed-loop system to (parametric) variations in the (model of the) human operator. The use of such models allows for a systematic approach to analysis. This opens the door to many issues for future research, including system efficiency, design and optimization, as well as suitability of systems to variations across both operators and tasks. Some of the implications of such models for the future operation of human-computer systems are discussed, especially with a view to future work.