Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot’s cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.

The aim of this study was to experimentally test the effects of different electromyographic-based prosthetic control modes on user task performance, cognitive workload, and perceived usability to inform further human-centered design and application of these prosthetic control interfaces. We recruited 30 able-bodied participants for a between-subjects comparison of three control modes: direct control (DC), pattern recognition (PR), and continuous control (CC). Multiple human-centered evaluations were used, including task performance, cognitive workload, and usability assessments. To ensure that the results were not task-dependent, this study used two different test tasks, including the clothespin relocation task and Southampton hand assessment procedure-door handle task. Results revealed performance with each control mode to vary among tasks. When the task had high-angle adjustment accuracy requirements, the PR control outperformed DC. For cognitive workload, the CC mode was superior to DC in reducing user load across tasks. Both CC and PR control appear to be effective alternatives to DC in terms of task performance and cognitive load. Furthermore, we observed that, when comparing control modes, multitask testing and multifaceted evaluations are critical to avoid task-induced or method-induced evaluation bias. Hence, future studies with larger samples and different designs will be needed to expand the understanding of prosthetic device features and workload relationships.

Amputees use prosthetic devices to perform activities of daily living. However, some users reject their devices due to the lack of usability or high cognitive workload. Although virtual reality has been studied in this domain for training purposes, there has not been any investigation on usability and cognitive workload of using virtual reality simulations for training of prosthetic devices. The objective of this study was to compare cognitive workload and usability of using virtual reality-based simulation of electromyography based prosthetic devices and physical devices. The findings suggested that using virtual reality simulations were helpful in reducing cognitive workload and increasing perceived usability of prosthetic devices.

Two types of feedback, including knowledge of results (KR) and knowledge of performance (KP) are typically delivered as part of motor training using different sensory modalities. Twenty-four right-handed individuals performed a computer-based psychomotor training task using a contemporary haptic interface. Results revealed that KP was superior to KR for overall task performance retention, whereas KR was good for training specific aspects of performance. A combination of KP and KR through the same modality did not produce an additive positive learning effect for task performance. Auditory and haptic feedback improved retention over visual, but only with the use of multiple information cues. Design guidelines are provided for training systems for motor skill development and/or rehabilitation.

UAV supervisory control interfaces are important for safe operations and mission performance. We reviewed existing UAV interface design and evaluation tools and identified limitations. To address issues with existing methods, we developed an enhanced evaluation tool, the M-GEDIS-UAV. The tool includes detailed criteria for all aspects of UAV control interface design to support operator performance. It also supports quantitative and objective assessment of an interface. We prototyped three UAV information displays, including a digital control display, analog control display, and “massive” data display, as part of a simulated supervisory control interface. Six analysts, including three human factors experts and three novices evaluated the interfaces using the M-GEDIS-UAV. Inter-rater reliability was high for the human factors experts, suggesting training in usability analysis is necessary for tool application. Results also revealed the massive data display to produce significantly lower evaluation scores than the other displays. We concluded that the M-GEDIS-UAV was sensitive to interface manipulations and was most effectively used by human factors experts. Using the M-GEDIS-UAV tool can reveal the majority of design deviations from guidelines early in the design process toward increasing the effectiveness of control interfaces.

The goal of this study was to investigate the use of cognitive modeling to assess the usability of an upper-limb prosthesis with a focus on mental workload responses. Prior studies have investigated usability of upper-limb prostheses with subjective surveys and physiological measures. However, these approaches have limitations, including subject recall of conditions and physiological response contamination by head and body movements and user speech during task performance as well as sensitivity to physical fatigue and room lighting conditions. Cognitive modeling was used to assess mental workload in use of transradial upper-limb prosthesis. A case study was conducted with a participant with upper-limb amputation using two different types of electromyography-based control schemes, including conventional direct control (DC) and pattern recognition (PR) control in order to compare cognitive model outcomes with mental workload assessment using eye-tracking measures. Cognitive models time estimates were also compared with actual task completion time results from the case study to further assess the validity of cognitive modeling as an analytical tool for evaluating upper limb prosthesis usability. Findings of both the cognitive models and case study revealed the PR mode to be more intuitive, reduce cognitive load, and increase efficiency in prosthetic control as compared to the DC mode. Results of the present study revealed that cognitive modeling can be used as an analytical approach for assessing upper-limb prosthetic device usability in terms of workload outcomes. Future studies should validate the present findings with more precise time estimations and a larger user sample size.

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement technique-the NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment, and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex human-machine systems.

The nine papers in this special section celebrate the publication's 50th Golden Jubilee Anniversary. Discusses the history of the publication from its initial launching in 1968. Presents a brief title history over the past 50 years and reports on its mission and scope.

Task outcome feedback is often used to sustain human work motivation and ensure overall productivity. Feedback can also be useful for reducing learning periods for new tasks. Two types of feedback, including knowledge of results (KR) and knowledge of performance (KP), are commonly delivered as part of motor training by using various sensory modalities (e.g., visual, auditory or kinesthetic). Unfortunately, there is a lack of a design basis for such sensory feedback systems to optimize human motivation. Twenty-four right-handed participants were recruited for computer-based psychomotor task training using a contemporary haptic interface. During task performance, KP, KR or combo feedback was presented to participants through one of three sensory modalities, including visual, auditory or haptic. An Intrinsic Motivation Inventory (IMI) questionnaire was used to evaluate participant motivation under each specific condition. Results revealed task outcome related feedback (KR or combo) to be superior to process-based feedback (KP) in terms of sustaining human intrinsic motivation during the training period. The haptic modality was also found to have a stronger motivating effect than the visual or auditory modalities. Findings of this study are beneficial for future feedback designs to sustain human motivation during occupational skill training, such as assembly operations.

The goal of this study was to compare the usability of two control schemes for a transradial myoelectric prosthesis, including conventional direct control (DC) and pattern recognition (PR) control, when used by able-bodied individuals. Three types of response measures were captured to assess the control schemes, including learnability, performance, and cognitive workload. Prior research has applied performance and cognitive workload metrics for evaluation of prosthetics; however, workload measures applied in these studies (e.g., heart rate, electroencephalography, and respiration rate) have many limitations. This study used eye tracking to compare cognitive load implications of the different control schemes for a two degrees-of-freedom myoelectric prosthesis. In total, 12 participants were assigned to either control condition (six persons each) or perform a clothespin relocation task. Results revealed the PR scheme to be more intuitive for users and superior to DC across all response measures. We observed a lower learning percentage (i.e., greater learning potential), lower cognitive load, and greater productivity in task performance. This preliminary study illustrates efficacy of using eye-tracking-based measures of cognitive load and standardize test paradigms for assessment of upper limb prosthetic usability and supports PR prosthetic device control as an intuitive alternative to DC.

Many occupations require both physical exertion and the ability to navigate in an environment, simultaneously. This study investigated how intensity of physical activity influences direction determination and distance estimation. Thirty high fit young males participated in a lab study. Results showed that while high fit young males were accurate in determining direction across levels of physical exertion, they were significantly less accurate in distance estimation under high exertion intensity. Although physical activity level did not influence direction determination accuracy, response time was significantly shorter when participants were subject to low physical loading in comparison to medium and high loading. In addition, we found that distance estimation response time increased as physical workload increased. Findings of this study can be used to enhance presentation of navigation information in occupations that require concurrent physical activity and navigation.

Unprincipled design of unmanned aerial vehicle (UAV) control interfaces can increase operator cognitive workload and degrade performance. It is important to identify optimal interface design features that can serve to prevent cognitive overload under demanding task scenarios and environmental conditions. The present research summarized literature on critical issues in supervisory control interface design, current UAV interface design approaches and existing evaluation methods. A research framework was also proposed for a project to systematically and quantitatively relate UAV control interface features to cognitive workload outcomes. The framework also supports development an effective and efficient interface evaluation tool to predict cognitive workload based on specific design features.

The purpose of this study was to compare the cognitive workload of able-bodied individuals when using a myoelectric prosthetic under direct control (DC) or electromyography pattern recognition (PR) control. Different from existing clinical evaluations involving dual-task performance, pupillography measured with an eye-tracking system was used to quantitatively assess user cognitive workload in using a 2 degree-of-freedom prosthesis for a clothespin task. Test results revealed the PR control to produce superior task performance and to require lower cognitive load than demanded of participants under the DC condition. This study provided evidence of both performance and workload advantages of PR control over DC control. PR control was more intuitive to the prosthesis user and, therefore, required less cognitive effort. Furthermore, the study identified a new effective measure of cognitive workload in upper limb prosthesis use via pupillography.

This research assessed user performance with different laptop touchpad textures. In specific, the study measured discrete movement task time and accuracy. It was hypothesized that texturing would increase task times but improve accuracy by providing users with tactile references. A variable representing the frictional potential of pads was introduced into an established model of discrete movement performance (Fitts’ Law) in an attempt to accurately model user performance under experimental task conditions. Results revealed touchpad texturing to degrade task performance. However, accuracy in pointing tasks was not significantly affected. Results also revealed that the expanded form of Fitts’ Law, including a parameter for representing the frictional potential of pad texturing, was more predictive of actual movement times than the original form of the Law. Results from the study increase understanding of the effects of touchpad texture on human motor control behavior and provide some guidance for future pad design.

The objective of this research was to develop a computer-based system for psychomotor skill assessment and training. The focus was on virtual reality (VR) simulation of an established pattern assembly task incorporating a haptic interface. A prototype VR and haptic-based system was developed to replicate established testing protocols, facilitate automated test scoring, and produce quantitative test output. The prototype was also compared to a native version of the test. Participants completed multiple training trials of the VR or native tests. Training effects were measured by comparing performance on objective baseline and post-tests administered before and after training, respectively. Results revealed training in either the VR or the native versions of the task to produce significant (p<;0.0001) performance increases from baseline, but neither version resulted in significant improvements over the other. The VR-haptic simulation was found to represent a viable psychomotor test and training method.

This study investigated three-dimensional (3D) goal-directed movements in a virtual reality (VR) simulation of a standardized psychomotor control task. Movement trajectories were collected from 22 subjects and parsed based on an existing two-phase model of motor control including ballistic and correction phases. Kinematic measures were also acquired to provide further insight into motor skill learning. Results revealed kinematic measures of total numbers of sub movements and numbers of sub movements in the correction phase to be significantly correlated with psychomotor task scores. A predictive model applied to the 3D movements revealed the correction phase movements to be more predictive of psychomotor performance than the ballistic phase. Findings indicate a greater degree of fine motor skill was required for performance of the psychomotor control task. This research supports the use of high resolution kinematic measures as reliable predictors of psychomotor task performance.

A flight simulation experiment was conducted to assess cockpit automation for mitigating the effects of a distance bias in time-to-contact judgments on converging intruders. Six formats of a prototype cockpit display of traffic information were tested, including all combinations of intruder icon (perceptual cue) format (baseline/conventional, colored, blinking) and a data tag with velocity information (present, absent). Results revealed displays with colored or blinking cues produced greater accuracy in pilot selection of intruders posing the greatest risk and lower response times than for conventional displays not presenting such cues. The addition of the velocity data tag to icons had no effect on intruder selection accuracy but was associated with lower response times. It was concluded that both the perceptual cue format and velocity data tags support superior performance in time-to-contact judgments as compared with conventional displays.

Prior research has revealed existing operating room (OR) patient monitors to provide limited support for prompt and accurate decision making by anesthesia providers during crises. Decision support tools (DSTs) developed for this purpose typically alert the anesthesia provider to existence of a problem but do not recommend a treatment plan. There is a need for a human-centered approach to the design and development of a crisis management DST. A hierarchical task analysis was conducted to identify anesthesia provider procedures in detecting, diagnosing, and treating a critical incident and a cognitive task analysis to elicit goals, decisions, and information requirements. This information was coded in a computational cognitive model using GOMS (Goals, Operators, Methods, Selection rules) Language. An OR monitor interface was prototyped to present output from the cognitive model following ecological interface design principles. A preliminary assessment of the DST was performed with anesthesiology and usability experts. The anesthesiologists indicated they would use the tool in the perioperative environment and would recommend its use by junior anesthesia providers. Future research will focus on formal validation of the DST design approach and comparison of tool output to actual anesthesia provider decisions in real or simulated crises.

The objective of this research was to develop a new computer-based system for psychomotor skill assessment. The focus was on the simulation of the Rey-Osterrieth Complex Figure (ROCF) reproduction test incorporating a haptic interface. Various system functions were created to support customized testing protocols that are based on specific user requirements, facilitate semiautomated scoring of tests, and produce quantitative test output. Advanced technologies of pattern recognition were reviewed and adapted for the system development. This approach yielded an application for recording freehand drawings and recognizing and normalizing drawing strokes for semiautomated scoring according to a standard. The new simulator system was validated by comparison with traditional paper-based tests in which participants were asked to use their nondominant hand to simulate a minor motor impairment. Results demonstrated the simulator to be sensitive to functional differences between dominant and nondominant hand use. The computerized scoring software also appeared to be valid for generating ROCF scores, which were consistent with manual scores determined by a trained rater for the same drawing stimuli.

Due to a major shortage of nurses in the U.S., future healthcare service robots are expected to be used in tasks involving direct interaction with patients. Consequently, there is a need to design nursing robots with the capability to detect and respond to patient emotional states and to facilitate positive experiences in healthcare. The objective of this study was to develop a new computational algorithm for accurate patient emotional state classification in interaction with nursing robots during medical service. A simulated medicine delivery experiment was conducted at two nursing homes using a robot with different human-like features. Physiological signals, including heart rate (HR) and galvanic skin response (GSR), as well as subjective ratings of valence (happy-unhappy) and arousal (excited-bored) were collected on elderly residents. A three-stage emotional state classification algorithm was applied to these data, including: (1) physiological feature extraction; (2) statistical-based feature selection; and (3) a machine-learning model of emotional states. A pre-processed HR signal was used. GSR signals were nonstationary and noisy and were further processed using wavelet analysis. A set of wavelet coefficients, representing GSR features, was used as a basis for current emotional state classification. Arousal and valence were significantly explained by statistical features of the HR signal and GSR wavelet features. Wavelet-based de-noising of GSR signals led to an increase in the percentage of correct classifications of emotional states and clearer relationships among the physiological response and arousal and valence. The new algorithm may serve as an effective method for future service robot real-time detection of patient emotional states and behavior adaptation to promote positive healthcare experiences.

The objective of this study was to demonstrate the utility of an established model of human motor behavior for assessing the fidelity of a virtual reality (VR) and haptic-based simulation for fine motor task performance. This study was also to serve as a basis for formulating general performance-based simulator-design guidelines toward balancing perceived realism with simulator limitations, such as latency resulting from graphic and haptic renderings. A low-fidelity surgical simulator was developed as an example VR for study, and user performance was tested in a simplified tissue-cutting task using a virtual scalpel. The observed aspect of the simulation included a discrete-movement task under different system-lag conditions and settings of task difficulty. Results revealed user performance in the VR to conform with Fitts' law of motor behavior and for performance to degrade with increasing task difficulty and system time lag. In general, the findings of this work support predictions on human performance under various simulator-design conditions using an established model of motor-control behavior and formulation of human-performance-based simulator-design principles.

The objective of this study was to assess the plausibility of using computational cognitive models for evaluating the usability of human–machine interfaces in supervisory control of high-throughput (biological) screening (HTS) operations. Usability evaluations of new interface prototypes were conducted by comparisons with existing technologies. Model assessment occurred through comparison with human test results. Task completion times and the number of errors were recorded during human performance trials, and task time was predicted in cognitive model trials in tests with two HTS interfaces. Computational GOMSL (Goals, Operators, Methods, and Selection rules Language) models were constructed based on a combination of cognitive task analyses (abstraction hierarchy modeling and goal-directed task analysis). The usability tests revealed improvements in task performance with the new prototypes. The cognitive model outputs were correlated with actual human performance, and the approach was considered useful for evaluating the usability of new interfaces in life sciences automation in the future.

This paper introduces high-throughput screening of biological samples in life sciences, as a domain for analysis of human-robot interaction (HRI) and development of usable human interface design principles. High-throughput screening (HTS) processes involve use of robotics and highly automated analytical measurement devices to transport and chemically evaluate biological compounds for potential use as drug derivatives. Humans act as supervisory controllers in HTS processes by performing test planning and device programming prior to experiments, systems monitoring, and real-time process intervention and error correction to maintain experiment safety and output. Process errors are infrequent but can be costly. Two forms of cognitive task analysis were applied to a highly automated HTS process to address different classes of errors, including goal-directed task analysis to describe critical operator decisions and information requirements and abstraction hierarchy modeling to represent HTS process devices and automation integrated in screening lines. The outcomes of the analyses were used as bases for generating supervisory control interface design recommendations to improve existing system usefulness and usability.

This paper describes an effort to identify common metrics for task-oriented human-robot interaction (HRI). We begin by discussing the need for a toolkit of HRI metrics. We then describe the framework of our work and identify important biasing factors that must be taken into consideration. Finally, we present suggested common metrics for standardization and a case study. Preparation of a larger, more detailed toolkit is in progress.

The objective of this study was to demonstrate the use of a computational cognitive model for representing human performance in robotic rover control and to make comparison with actual human performance. In manual control trials, an operator was required to navigate a commercially available rover along a path using a remote workstation. A cognitively plausible GOMSL (Goals, Operators, Methods, Selection rules) model for controlling the rover was constructed based on a task analysis and observations during the human trials. Time-to-navigation completion and path tracking accuracy were recorded during the human performance and cognitive model trials with navigation conditions being identical. Model output demonstrated the GOMSL code to produce more precise control of the rover than human performance, but this was at the cost of time. This result was attributable to limitations of the modeling approach in terms of representing human parallel processing and continuous control. In general, computational GOMS modeling approaches appear to have the potential to describe interactive closed-loop system control with continuous monitoring of feedback and corresponding control actions.