Mental workload recognition is of great significance in preventing human errors and accidents. This study constructed a multimodal recognition scheme to recognize three mental workload states: underload, moderate load, and overload. Based on driving scenarios, these three states were induced in this study by changing the driving modes and situations. Multimodal recognition of underload, moderate load, and overload was performed using electroencephalography (EEG), electrocardiography (ECG), and pupillometry. In addition, various machine learning methods were used to evaluate the recognition performance of different feature combinations. The results showed that the random forest method, trained using spectral power, pupil diameter, and heart rate variability, achieved the highest recognition accuracy of 83.13% for the three mental workload states. This study provides valuable reference information for multimodal recognition of mental workload states.

Perspective-taking and attentional switching are some of the ergonomic challenges that existing teleoperation human-machine interface designs need to address. This study developed two gaze interaction methods, the Eye Stick and the Eye Click, which were based on the joystick metaphor and the navigation metaphor, respectively, to be used in exocentric perspective teleoperation scenarios. We conducted two user studies to test the task performance and the subjective experience of the gaze interaction methods in a virtual ground vehicle teleoperation task. The results showed that compared with a traditional joystick design, the Eye Stick led to a shorter driving distance and the Eye Click led to less task time, and the gaze interaction methods had performance advantages in more difficult mazes. After multiple task sessions, the participants reported that the gaze interaction methods and the traditional joystick were similar in terms of task workload, perceived learnability, and satisfaction; however, the perceived usability of the Eye Stick was not as good as the Eye Click and the traditional joystick. In conclusion, both the Eye Stick and the Eye Click are feasible and promising gaze interaction methods for teleoperation applications with task performance advantages; however, more research is needed to optimize their user experience design.

Intention decoding is an indispensable procedure in hands-free human–computer interaction (HCI). A conventional eye-tracker system using a single-model fixation duration may issue commands that ignore users' real expectations. Here, an eye-brain hybrid brain–computer interface (BCI) interaction system was introduced for intention detection through the fusion of multimodal eye-tracker and event-related potential (ERP) [a measurement derived from electroencephalography (EEG)] features. Eye-tracking and EEG data were recorded from 64 healthy participants as they performed a 40-min customized free search task of a fixed target icon among 25 icons. The corresponding fixation duration of eye tracking and ERP were extracted. Five previously-validated linear discriminant analysis (LDA)-based classifiers [including regularized LDA, stepwise LDA, Bayesian LDA, shrinkage linear discriminant analysis (SKLDA), and spatial-temporal discriminant analysis] and the widely-used convolutional neural network (CNN) method were adopted to verify the efficacy of feature fusion from both offline and pseudo-online analysis, and the optimal approach was evaluated by modulating the training set and system response duration. Our study demonstrated that the input of multimodal eye tracking and ERP features achieved a superior performance of intention detection in the single-trial classification of active search tasks. Compared with the single-model ERP feature, this new strategy also induced congruent accuracy across classifiers. Moreover, in comparison with other classification methods, we found that SKLDA exhibited a superior performance when fusing features in offline tests (ACC = 0.8783, AUC = 0.9004) and online simulations with various sample amounts and duration lengths. In summary, this study revealed a novel and effective approach for intention classification using an eye-brain hybrid BCI and further supported the real-life application of hands-free HCI in a more precise and stable manner.

Previous studies have used the Fitts’ Law to predict the performance of pointing task on touch screens. It was found that moving distance, target width, and the direction of motion would affect the task performance. The present research aimed to investigate the impact of the initial and terminal positions (ITPs) on pointing performance in two Exps. ITPs were divided into two categories: center of a screen and outer of a screen (center ↔ outer) pointing, and outer of a screen to outer of a screen (outer→outer) pointing. In Exp 1, 30 participants performed the center ↔ outer pointing tasks with 8 directions. The results showed that the outer → center movement was significantly faster than the center → outer at 45°, 90°, and 180°. In Exp 2, 30 participants performed the outer → outer pointing tasks with eight directions. The current study revealed that the ITPs influenced the performance of the pointing task, possibly due to human biomechanical characteristics associated with different movements.

Reciprocal cooperation is prevalent in human society. Understanding human reciprocal cooperation in human-agent interaction can help design human-agent systems that promote cooperation and joint performance. Studies have found that people reciprocate cooperative behavior when interacting with computer agents in social dilemma games. However, few studies have investigated human reciprocal cooperation with agents in complex dynamic environments. This article examines the behavioral and neurological patterns of reciprocal cooperation in a hospital management microworld experiment. The participants (n = 30) work with both high- and low-cooperation computer agents to share resources to cope with dynamic demands. Participants' resource sharing behaviors were recorded and their prefrontal cortex (PFC) activation was measured using functional near-infrared spectroscopy (fNIRS) technology. Similar to previous studies conducted with participants in the United States, results demonstrate that participants in China showed reciprocal cooperation behaviors with the agents. Specifically, participants share more resources and achieve higher performance when working with a high-cooperation agent than with a low-cooperation agent. A high activation level is detected in the right dorsolateral PFC when working with a high-cooperation agent. Other PFC activation patterns imply that cooperation could be unnecessarily mentally taxing in certain situations. These findings suggest that human cooperativeness in human-agent systems can be calibrated by an agent's cooperation behavior. System designers should design for appropriate cooperativeness and avoid the inefficient use of system resources. Neurological measures could be a useful tool to investigate the mental process in human-agent cooperation.

BACKGROUND: Electronic health records (EHRs) are increasingly used for clinical and translational research through the creation of phenotype algorithms. Currently, phenotype algorithms are most commonly represented as noncomputable descriptive documents and knowledge artifacts that detail the protocols for querying diagnoses, symptoms, procedures, medications, and/or text-driven medical concepts, and are primarily meant for human comprehension. We present desiderata for developing a computable phenotype representation model (PheRM). METHODS: A team of clinicians and informaticians reviewed common features for multisite phenotype algorithms published in PheKB.org and existing phenotype representation platforms. We also evaluated well-known diagnostic criteria and clinical decision-making guidelines to encompass a broader category of algorithms. RESULTS: We propose 10 desired characteristics for a flexible, computable PheRM: (1) structure clinical data into queryable forms; (2) recommend use of a common data model, but also support customization for the variability and availability of EHR data among sites; (3) support both human-readable and computable representations of phenotype algorithms; (4) implement set operations and relational algebra for modeling phenotype algorithms; (5) represent phenotype criteria with structured rules; (6) support defining temporal relations between events; (7) use standardized terminologies and ontologies, and facilitate reuse of value sets; (8) define representations for text searching and natural language processing; (9) provide interfaces for external software algorithms; and (10) maintain backward compatibility. CONCLUSION: A computable PheRM is needed for true phenotype portability and reliability across different EHR products and healthcare systems. These desiderata are a guide to inform the establishment and evolution of EHR phenotype algorithm authoring platforms and languages.

OBJECTIVE: To review and evaluate available software tools for electronic health record-driven phenotype authoring in order to identify gaps and needs for future development. MATERIALS AND METHODS: Candidate phenotype authoring tools were identified through (1) literature search in four publication databases (PubMed, Embase, Web of Science, and Scopus) and (2) a web search. A collection of tools was compiled and reviewed after the searches. A survey was designed and distributed to the developers of the reviewed tools to discover their functionalities and features. RESULTS: Twenty-four different phenotype authoring tools were identified and reviewed. Developers of 16 of these identified tools completed the evaluation survey (67% response rate). The surveyed tools showed commonalities but also varied in their capabilities in algorithm representation, logic functions, data support and software extensibility, search functions, user interface, and data outputs. DISCUSSION: Positive trends identified in the evaluation included: algorithms can be represented in both computable and human readable formats; and most tools offer a web interface for easy access. However, issues were also identified: many tools were lacking advanced logic functions for authoring complex algorithms; the ability to construct queries that leveraged un-structured data was not widely implemented; and many tools had limited support for plug-ins or external analytic software. CONCLUSIONS: Existing phenotype authoring tools could enable clinical researchers to work with electronic health record data more efficiently, but gaps still exist in terms of the functionalities of such tools. The present work can serve as a reference point for the future development of similar tools.

The aim of this study is to examine the utility of physiological compliance (PC) to understand shared experience in a multiuser technological environment involving active and passive users. Common ground is critical for effective collaboration and important for multiuser technological systems that include passive users since this kind of user typically does not have control over the technology being used. An experiment was conducted with 48 participants who worked in two-person groups in a multitask environment under varied task and technology conditions. Indicators of PC were measured from participants' cardiovascular and electrodermal activities. The relationship between these PC indicators and collaboration outcomes, such as performance and subjective perception of the system, was explored. Results indicate that PC is related to group performance after controlling for task/technology conditions. PC is also correlated with shared perceptions of trust in technology among group members. PC is a useful tool for monitoring group processes and, thus, can be valuable for the design of collaborative systems. This study has implications for understanding effective collaboration.

Distance collaboration technologies affect the way active and passive users interact in technology-mediated systems. Decreases in social and contextual cues in distance collaboration may have a large impact on passive users' perception of active users and the technology. The purpose of this study was to investigate passive users' trust in active users and trust in technology under varied technological conditions and active user performance. A laboratory experiment was conducted using simulated psychomotor tasks distance collaboration scenarios. Participants observed an active user, who performed tasks without being physically present. Their subjective report on trust in the active user, trust in technology and perceived active user's workload, as well as physiological responses, including eye movement, electrodermal activity and cardiovascular activity, were gathered. The results showed that technology conditions affected passive users' subjective reports, specifically; the participants exhibited higher arousal in the affect arousal system during the observation. Furthermore, the passive users seemed to evaluate their trust in the active user according to their trust in technology. This implies that in a distance collaboration context, technology use could affect interpersonal relationships between active and passive users.