Hyperscanning with functional nearinfrared spectroscopy (fNIRS) is an emerging neuroimaging application that measures the nuanced neural signatures underlying social interactions. Researchers have assessed the effect of sex and task type (e.g., cooperation versus competition) on inter-brain coherence during human-to-human interactions. However, no work has yet used deep learning-based approaches to extract insights into sex and task-based differences in an fNIRS hyperscanning context. This work proposes a convolutional neural network-based approach to dyadic sex composition and task classification for an extensive hyperscanning dataset with N = 222 participants. Inter-brain signal similarity computed using dynamic time warping is used as the input data. The proposed approach achieves a maximum classification accuracy of greater than 80 percent, thereby providing a new avenue for exploring and understanding complex brain behavior.

Many producers of automated vehicle systems have begun testing autonomous vehicles on the road. In order to ensure safety and prevent crashes, human drivers are enlisted to monitor autonomous vehicles. However, operators of autonomous systems exhibit negative behavior adaptations in response to prolonged supervision of automation. To prevent the onset of undesirable behaviors in safety drivers, we must investigate driver state and behavior changes during the operation of highly automated vehicles. In the study presented here, we examine the effects of theoretical and practical training on the drivers' response to potentially critical situations in a longitudinal driving simulator study. We also present the effects of encountering a failure of the automated vehicle on driver state and behavior. We conducted a two-part panel driving simulator study (N=28), with an interval of 20-30 days between the training and testing sessions. We found that while participants with training are better prepared for a potential failure of the automation, participants in both conditions show a rise in sleepy or drowsy behavior before a potential failure of automation.

The advent of automated features in modern vehicles requires human factors researchers to find measures other than driving behavior to anticipate the response of drivers in various contexts. Functional near-infrared spectroscopy (fNIRS) is one research tool that allows us to quantify the driver's mental state. However, the underlying mechanisms of fNIRS technology can limit the possible contexts for its application. The pervasive question arises, whether the measurement device at hand is suitable for the research topic in question and is it capable of detecting the phenomenon under investigation? We provide a proof of concept study demonstrating that significant habituation is present when drivers operate new automated driving systems and that fNIRS technology is suitable to detect said driver habituation effects. The study presented here was conducted in a driving simulator and investigated the drivers' cortical activation in three different modes of automation: manual, partially autonomous, and fully autonomous modes.

Information about drivers' mental states can be vital to the design of interfaces for highly automated vehicles. Functional near infrared spectroscopy (fNIRS) is a neuroimaging tool that is fast becoming popular to study the cortical activity of participants in HCI experiments and driving simulator studies in particular. The analysis methods of the fNIRS data create requirements in the experimental design such as repeated measures. In this paper, we present a study of the event related cortical activity of the drivers of manual, partially autonomous, and fully autonomous cars when performing lane changes using functional near infrared spectroscopic measures. We also present the experimental methodology that was adopted to meet the needs of the fNIRS measurement and the subsequent analysis. The study (N=28) was conducted in a driving simulator. Participants drove for approximately 7 minutes and performed 8 lane change maneuvers in each mode of automation. Multiple streams of data including 4 time-synced video recordings, NASA TLX questionnaires and fNIRS data were recorded and analyzed. It was found that the dorsolateral prefrontal cortex activation during lane changes performed in a partially autonomous mode of operation was just as high as that during a manual lane change, showing that drivers of partially automated systems are as cognitively engaged as drivers of manually operated vehicles.