Excessive mental workload can lead to less remaining resources for pilots to perform concurrent tasks during emergency flights, affecting aviation safety. Based on a flight simulator, this study investigated 25 cadet pilots using functional near-infrared spectroscopy (fNIRS) and subjective ratings to assess their mental workload under three subtasks with different equipment failures. fNIRS data included oxyhemoglobin, deoxyhemoglobin, and total hemoglobin signals, yielding 10545 functional connectivity (FC) features from four brain regions: prefrontal, right motor, left motor, and occipital cortexes. A deep residual shrinkage network classified mental workload levels, outperforming convolutional neural network and random forest models with 89.58% accuracy after feature selection employing an interpretable machine learning algorithm. The results suggest that brain FC from three hemoglobin signals could be used to differentiate the three different levels of pilot mental workload. This study could contribute to improving pilot training and supporting the development of pilots’ competencies during emergency scenarios.

Virtual reality (VR) is increasingly being used to promote exercise among older adults. The data captured through VR may be useful indicator of the game user's experience as well as providing insight into functional ability of older adults. This paper presents classifiers to predict game user experience variables using VR data from community-dwelling older adults. Head-kinematic data of the VR headset was collected from 13 participants over a six-week period with three 20-minutes exergame sessions per week (e.g., 360 minutes per participant). Cognitive function was assessed using the Montreal Cognitive Assessment (MoCa) and multisensory response-time (RT). Game user experience was captured through perceived-levels of cybersickness, enjoyment, and exertion after each session. Data was used as references for discrete binary and ternary classification patterns. Combinations of kinematic features were used to train different classifiers: K-nearest-neighbors (KNN), linear discriminant analysis (LDA), support vector machines (SVM), and decision trees. Maximum classification accuracy of 70% was found for MoCa, 68% for perceived exertion, 60% for cybersickness, 59% for multisensory RT, and 53% for perceived enjoyment. Results suggest unobtrusive recording of head kinematics from VR headsets combined with machine learning classifiers could be used to predict cognition, exertion, and game user experience among older adults.

Blink rate has been suggested to be a proxy for cognitive load. However, there are mixed results regarding this association, which is modulated by the visual demand of a task or when a dual-task paradigm is performed. This study investigated blink rate as a measure of cognitive load using a dual task paradigm (i.e., primary task = flying an aircraft; secondary task = auditory oddball task) to specifically assess changes in cognitive load across two flight phases (i.e., cruise, landing). Primary task performance suggested participants focused on maintaining aircraft control across task conditions, but a significant dual-task difference score in auditory oddball task performance (p<0.001) was still evident across flight phases. Blink rate did not show similar differences across flight phases. Our results demonstrate that the utility in using blink rate to measure cognitive load may be limited in its application to more complex naturalistic scenarios when using an auditory dual-task.

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behaviour, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous.

In this article, we used a computational cognitive architecture called queuing network–adaptive control of thought rational–situation awareness (QN–ACTR–SA) to model and simulate the brake perception response time (BPRT) to visual roadway hazards. The model incorporates an integrated driver model to simulate human driving behavior and uses a dynamic visual sampling model to simulate how drivers allocate their attention. We validated the model by comparing its results to empirical data from human participants who encountered on-road and roadside hazards in a simulated driving environment. The results showed that BPRT was shorter for on-road hazards compared to roadside hazards and that the overall model fitness had a mean absolute percentage error of 9.4% and a root mean squared error of 0.13 s. The modeling results demonstrated that QN–ACTR–SA could effectively simulate BPRT to both on-road and roadside hazards and capture the difference between the two contrasting conditions.

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behaviour, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous.

We conducted two experiments to examine the effect of providing automation support and communicating the limitations of the automation on team performance in a simulated navy environment. Two-person teams engaged with a picture compilation task with or without automation support. In the first experiment, there was no explicit explanations provided regarding the automation's limitations. In the second experiment, limitations of the automation were communicated to the participants verbally. A comparison of two experiments revealed that participants classified fewer contacts when the automation support was present. Moreover, communicating the limitations of automation resulted in even fewer classifications than when no information was provided. Possible reasons for these results include confusion created by the additional information and reprioritization. These results highlight the complexity of delivering automation transparency to operators in safety-critical environments. We conclude that automation transparency should be carefully designed and delivered to avoid negative impacts on performance.

This research explores the design, implementation, and evaluation of a prototype augmented reality application that assists operators in performing procedural tasks in control room settings. Our prototype uses a tablet display to supplement an operator’s natural view of existing control panel elements with sequences of interactive visual and attention guiding cues. An experiment, conducted using a nuclear power plant simulator, examined university students completing both standard and emergency operating procedures. The augmented reality condition was compared against two other conditions – a paper-based procedure condition using paper manuals and a computer-based procedure condition using digital procedures presented on a desktop display. The results demonstrated that the augmented reality -based procedure system had benefits in terms of reduced mental workload in comparison to the other two conditions. Regarding task completion time, accuracy, and situation awareness, the augmented reality condition had no significant difference when compared against the computer-based procedure condition but performed better than the paper-based procedure condition. It was also found that the augmented reality condition resulted in fewer intra-team inquiry communication exchanges in comparison to both paper-based and computer-based conditions. The augmented reality condition, however, yielded poorer memory retention score when assessed against the other two conditions.

Modeling typing performance has values in both the theory and design practice of human–computer interaction. Previous models have simulated desktop keyboard transcription typing performance; however, as the increasing prevalence of smartphones, new models are needed to account for mobile phone touchscreen typing. In the current study, we built a model for mobile phone touchscreen typing in an integrated cognitive architecture and tested the model by comparing simulation results with human results. The results showed that the model could simulate and predict interkey time performance in both number typing (Experiment 1) and sentence typing (Experiment 2) tasks. The model produced results similar to the human data and captured the effects of digit/letter position and interkey distance on interkey time. The current work demonstrated the predictive power of the model without adjusting any parameters to fit human data. The results from this study provide new insights into the mechanism of mobile typing performance and support future work simulating and predicting detailed human performance in more complex mobile interaction tasks.

Quantitative estimation of a driver's vigilance level has a great value for improving driving safety and preventing accidents. Previous studies have identified correlations between electroencephalogram (EEG) spectrum power and a driver's mental states such as vigilance and alertness. Studies have also built classification models that can estimate vigilance state changes based on data collected from drivers. In the present study, we propose a system to detect vigilance level using not only a driver's EEG signals but also driving contexts as inputs. We combined a support vector machine with particle swarm optimization methods to improve classification accuracy. A simulated driving task was conducted to demonstrate the reliability of the proposed system. Twenty participants were assigned a 2-h sustained-attention driving task to identify a lead car's brake events. Our system was able to account for 84.1% of experimental reaction times with 162-ms prediction errors. A newly introduced driving context factor, road curves, improved the prediction accuracy by 2-5% with 30-80 ms smaller errors. These findings demonstrated the potential value of the proposed system for estimating driver vigilance level on a time scale of seconds.

Navigation systems have been widely used in outdoor environments, but indoor navigation systems are still in early development stages. In this paper, we introduced an augmented-reality-based indoor navigation application to assist people navigate in indoor environments. The application can be implemented on electronic devices such as a smartphone or a head-mounted device. In particular, we examined Google Glass as a wearable head-mounted device in comparison with handheld navigation aids including a smartphone and a paper map. We conducted both a technical assessment study and a human factors study. The technical assessment established the feasibility and reliability of the system. The human factors study evaluated human-machine system performance measures including perceived accuracy, navigation time, subjective comfort, subjective workload, and route memory retention. The results showed that the wearable device was perceived to be more accurate, but other performance and workload results indicated that the wearable device was not significantly different from the handheld smartphone. We also found that both digital navigation aids were better than the paper map in terms of shorter navigation time and lower workload, but digital navigation aids resulted in worse route retention. These results could provide empirical evidence supporting future designs of indoor navigation systems. Implications and future research were also discussed.

This research comprehensively illustrates the design, implementation and evaluation of a novel marker less environment tracking technology for an augmented reality based indoor navigation application, adapted to efficiently operate on a proprietary head-mounted display. Although the display device used, Google Glass, had certain pitfalls such as short battery life, slow processing speed, and lower quality visual display but the tracking technology was able to complement these limitations by rendering a very efficient, precise, and intuitive navigation experience. The performance assessments, conducted on the basis of efficiency and accuracy, substantiated the utility of the device for everyday navigation scenarios, whereas a later conducted subjective evaluation of handheld and wearable devices also corroborated the wearable as the preferred device for indoor navigation.

Information technologies have been developed to facilitate driving performance and improve safety. However, there is a lack of computational methods that can take into account drivers’ adaptation to driving. That is, how behaviour changes with experience. Modelling the effect of driving experience on driver behaviour is important to the development of in-vehicle information technologies, because drivers at different skill levels may need different types or levels of assistance. Cognitive-architecture-based human performance modelling is a valuable method that can integrate different cognitive aspects underlying human behaviour such as skill levels and support quantitative simulation of behaviour. The study reported in this paper tested and examined computational models built in ACT-R (Adaptive Control of Thought-Rational) to account for the effect of driving experience on collision avoidance braking behaviour. The modelling results were compared with human data collected from a simulated driving experiment. The models produced braking behavioural results similar to the human results. Moreover, model predictions of three other emergent-braking scenarios were generally similar to and in the same order with the empirical results reported in previous studies. Future research can further integrate the method and results into intelligent driver assistance systems such as collision warning systems to better adjust the systems to the need of different drivers with different skill levels.

Research indicates that using a cell phone to talk or text while maneuvering a vehicle impairs driving performance. However, few published studies directly compare the distracting effects of texting using a hands-free (i.e., speech-based interface) versus handheld cell phone, which is an important issue for legislation, automotive interface design and driving safety training. This study compared the effect of speech-based versus handheld texting on simulated driving performance by asking participants to perform a car following task while controlling the duration of a secondary texting task. Results showed that both speech-based and handheld texting impaired driving performance relative to the drive-only condition by causing more variation in speed and lane position. Handheld texting also increased the brake response time and increased variation in headway distance. Texting using a speech-based cell phone was less detrimental to driving performance than handheld texting. Nevertheless, the speech-based texting task still significantly impaired driving compared to the drive-only condition. These results suggest that speech-based interaction disrupts driving, but reduces the levels of performance interference compared to handheld devices. In addition, the difference in the distraction effect caused by speech-based and handheld texting is not simply due to the difference in task duration.