Visually searching for objects is an everyday task. In many contexts, people must visually search for multiple objects at the same time while avoiding distractor objects, such as triage during a mass casualty incident. While many prior augmented reality (AR) and virtual reality (VR) studies have investigated cues to aid in visual search tasks, few have investigated cues in contexts involving multiple targets and distractors with a full 360° effective field of regard (EFOR). Individually, multiple targets, distractors, and a full 360° EFOR each add complexity to visual search; when combined, they compound the difficulty even further. In this paper, we present such a study that compares three common types of visual cues (2D Wedge, 3D Arrow, and Gaze Line) to a baseline condition with no cueing for a 360° visual search task. Our results reinforce the importance of providing some type of cue, with the Gaze Line design being particularly beneficial. We discuss the potential implications of these findings for designing cues specifically for such complex visual search tasks.

We investigate the utility of functional near-infrared spectroscopy (fNIRS) for workload-based adaptive automation through the lens of multiple resource theory. We focus on the criteria of unobtrusiveness, responsiveness, load sensitivity (low vs high load), and load diagnosticity (differentiating types of load). We report a large meta-review, in which we conclude that only a few studies were suitable for evaluating sensitivity and diagnosticity in complex real-world tasks. While these reveal that the fNIRS signal is adequately sensitive to gradations of load level changes (sensitivity), the diagnosticity of fNIRS to different sources of cognitive load remained uncertain. We manipulated mental load of a complex shape sorting task via working memory load (WM) and visual perceptual load (VL), while a secondary auditory task was present throughout. We measured the effect of these manipulations at the group-level using conventional secondary and eyetracking workload measures, as well as hemodynamic response in specific functional regions in the brain, including regions involved in multi-tasking (MT), VL, WM, and auditory load (AL). Our findings revealed that fNIRS is both sensitive and diagnostic to load in complex tasks, with greater sensitivity revealed by deoxyhemoglobin than oxyhemoglobin and the brain regions associated with diagnosticity align with neuroscience literature on perceptual load, WM, and goal-directed multitasking.

Despite knowing exactly what an object looks like, searching for it in a person's visual field is a time-consuming and error-prone experience. In Augmented Reality systems, new algorithms are proposed to speed up search time and reduce human errors. However, these algorithms might not always provide 100% accurate visual cues, which might affect users' perceived reliability of the algorithm and, thus, search performance. Here, we examined the detrimental effects of automation bias caused by imperfect cues presented in the Augmented Reality head-mounted display using the YOLOv5 machine learning model. 53 participants in the two groups received either 100% accurate visual cues or 88.9% accurate visual cues. Their performance was compared with the control condition, which did not include any additional cues. The results show how cueing may increase performance and shorten search times. The results also showed that performance with imperfect automation was much worse than perfect automation and that, consistent with automation bias, participants were frequently enticed by incorrect cues.

While the visual search for targets in a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies by helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective consequences following automation errors. A total of 56 participants searched a three-dimensional environment containing 48 objects in a room, in order to locate a target object that was viewed prior to each trial. They searched either unaided or assisted by one of the three HMD types of cues: an arrow pointing to the target, a plan-view minimap highlighting the target, and a constantly visible icon depicting the appearance of the target object. The cue was incorrect in 17% of the trials for one group of participants and 100% correct for the second group. Through both analysis and modeling of both search speed and accuracy, the results indicated that the arrow and minimap cues depicting location information were more effective than the icon cue depicting visual appearance, both overall, and when the cue was correct. However, there was a tradeoff on the infrequent occasions when the cue erred. The most effective AR-based cue led to a greater automation bias in which the cue was more often blindly followed without careful examination of the raw images. The results speak to the benefits of AR and the need to examine potential costs when AR-conveyed information may be incorrect because of imperfectly reliable systems.

We discuss two general concepts of attention: (1) As a filter of selective attention that selects and admits channels of information from the environment to be processed; (2) as a resource to enable subsequent information processing, constrained by the individual demand of tasks, and particularly the collective demands of multiple tasks needing to be performed concurrently defining the limits of multitasking. In both concept domains we review theories and design principles, and particularly computational models of attentional process and performance outputs. In discussing the filter we consider supervisory control, noticing, visual search, and the way in which limited selective attention constrains the filter can be overcome to avail more parallel processing via techniques of display overlay and object display. In discussing the fuel for multi-tasking, we distinguish between the parallel processing supported by multiple resources, and the sequential processing and task switching required when demands are excessive.

To minimize the risk of operator distraction or overload, system designers must tailor the arrangement of display channels in a visual workspace to the operator’s attentional abilities. Toward this end, computational models that embody the theoretical principles of attention control can inform the designer’s choices and narrow the space of potential display layouts for consideration. Wickens’ SEEV model incorporates four mechanisms of attentional guidance—stimulus salience, effort, expectancy, and information value—along with a probabilistic choice mechanism to predict the steady-state distribution of attention over multiple information channels. An extension of the model also predicts the speed and probability with which participants notice critical signals in a dynamic workspace. Empirical tests in single- and multi-task supervisory monitoring environments confirm that the models predict performance well, capturing the influence of various top-down and bottom-up attentional control mechanisms. Results suggest that the models can serve as theory-motivated tools to guide and assess the design of multi-channel visual workspaces.

Technical developments in computer hardware and software now make it possible to introduce automation into virtually all aspects of human-machine systems. Given these technical capabilities, which system functions should be automated and to what extent? We outline a model for types and levels of automation that provides a framework and an objective basis for making such choices. Appropriate selection is important because automation does not merely supplant but changes human activity and can impose new coordination demands on the human operator. We propose that automation can be applied to four broad classes of functions: 1) information acquisition; 2) information analysis; 3) decision and action selection; and 4) action implementation. Within each of these types, automation can be applied across a continuum of levels from low to high, i.e., from fully manual to fully automatic. A particular system can involve automation of all four types at different levels. The human performance consequences of particular types and levels of automation constitute primary evaluative criteria for automation design using our model. Secondary evaluative criteria include automation reliability and the costs of decision/action consequences, among others. Examples of recommended types and levels of automation are provided to illustrate the application of the model to automation design.

The Engineering Psychology program at the University of Illinois is interdisciplinary in nature and involves the participation of faculty members, undergraduate, graduate and post-doctoral students from the Department of Psychology, Department of Industrial Engineering, the Institute of Aviation, and the Beckman Institute. While the interests of the faculty members and the students run the gamut from nuts and bolts ergonomics to the design of intelligent systems there are two topics, the design of displays and the acquisition of perceptual-motor and cognitive skills, which serve as a unifying focus of research in Human-Computer Interaction.

The ability of operators to detect step changes in the order of control dynamics is investigated as a joint function of a) participatory mode, whether subjects are actively controlling those dynamics or are monitoring an autopilot controlling them, and b) concurrent task workload. Five subjects either tracked or monitored the system dynamics on a two-dimensional pursuit display under single task conditions and concurrently with a "subcritical" tracking task at two difficulty levels Detection performance was faster and only slightly less accurate in the manual as opposed to the autopilot mode. Performance in each mode was derogated by the concurrent tracking requirement, but not by increases in loading task difficulty. Further analysis indicated that manual superiority was attributable to the additional proprioceptive information resulting from operator-control adaptation to the system change.