Silent automation failures, where a system fails to detect a hazard without warning, pose a critical safety challenge for partially automated vehicles. While research has mostly focused on takeover requests, how to support a driver in silent failure remains underexplored. We conducted a multi-modal driving simulator study with 48 participants to investigate how different Prospective Situation Awareness Enhancement (PSAE) interfaces, delivered via augmented reality head-up display, affect takeover performance. By integrating behavioral, subjective psychological, and physiological data, our analysis suggests that situational awareness (SA) serves as an important moderating factor through which PSAE interfaces improve takeover performance. Further, we found that providing perceptual cues was most effective in enhancing SA, while communicating system intent was superior for building trust. Finally, we identified a potential correlate of SA in the neuroactivity. Overall, this paper contributes to understanding how transparency-oriented interfaces may support drivers and provides design insights into HMI design for silent failures.

Abstract Multi-source synsemantic domain generalization (MSSDG) for multi-task remote physiological measurement seeks to enhance the generalizability of these metrics and attracts increasing attention. However, challenges like partial labeling and environmental noise may disrupt task-specific accuracy. Meanwhile, given that real-time adaptation is necessary for personalized products, the test-time personalized adaptation (TTPA) after MSSDG is also worth exploring, while the gap between previous generalization and personalization methods is significant and hard to fuse. Thus, we proposed a unified framework for MSSD G and TTP A employing P riors ( GAP ) in biometrics and remote photoplethysmography (rPPG). We first disentangled information from face videos into invariant semantics, individual bias, and noise. Then, multiple modules incorporating priors and our observations were applied in different stages and for different facial information. Then, based on the different principles of achieving generalization and personalization, our framework could simultaneously address MSSDG and TTPA under multi-task remote physiological estimation with minimal adjustments. We expanded the MSSDG benchmark to the TTPA protocol on six publicly available datasets and introduced a new real-world driving dataset with complete labeling. Extensive experiments that validated our approach, and the codes along with the new dataset are in https://github.com/WJULYW/GAP .

While advanced driving assistant systems (ADAS) can offer significant benefits to driving safety, driver behaviors are still critical to driving safety, especially in hazardous scenarios. Though ADAS can handle most of the operational driving tasks, they are still less capable of understanding evolving traffic scenarios and hence are less defensive compared to human drivers. Further, most existing studies focused on the influence of ADAS on the behaviors of passenger vehicles, but the safety of long-haul trucks is of more concern. Thus, it becomes imperative to understand how ADAS affects driver behaviors, especially defensive driving behaviors, among long-haul truck drivers. To address this research gap, a naturalistic driving experiment among long-haul truck drivers was conducted, where 868 right-side-merging events that can allow defensive driving behaviors were extracted. Drivers’ defensive driving behavior decision (i.e., yes or no), defensive driving behavior type (i.e., which type of defensive driving behavior), and time to defensive driving behavior were modeled. Results show that ADAS can facilitate drivers’ defensive driving behaviors, as indicated by a higher percentage of actions and quicker responses when approaching hazardous areas. Further, drivers’ defensive driving behaviors (likelihood and types) changed with the progress of the drive, drowsiness levels, traffic conditions, and ADAS availability. Findings from this study provide insights into understanding truck drivers’ behaviors in the context of driving automation and guide the design of future driver training programs for ADAS users, human-machine interfaces of ADAS, and laws or regulations regarding the adoption of ADAS in long-haul trucks.

Robust and unobtrusive in-vehicle physiological monitoring is crucial for ensuring driving safety and user experience. While remote physiological measurement (RPM) offers a promising non-invasive solution, its translation to real-world driving scenarios is critically constrained by the scarcity of comprehensive datasets. Existing resources are often limited in scale, modality diversity, the breadth of biometric annotations, and the range of captured conditions, thereby omitting inherent real-world challenges in driving. Here, we present PhysDrive, the first large-scale multimodal dataset for contactless in-vehicle physiological sensing with dedicated consideration of various modality settings and driving factors. PhysDrive collects data from 48 drivers, including synchronized RGB, near-infrared camera, and raw mmWave radar data, accompanied by six synchronized ground truths (ECG, BVP, Respiration, HR, RR, and SpO2). It covers a wide spectrum of naturalistic driving conditions, including driver motions, dynamic natural light, vehicle types, and road conditions. We extensively evaluate both signal‑processing and deep‑learning methods on PhysDrive, establishing a comprehensive benchmark across all modalities, and release full open‑source code with compatibility for mainstream public toolboxes. We envision PhysDrive will serve as a foundational resource and accelerate research on multimodal driver monitoring and smart‑cockpit systems.

As a novel AI-powered conversational system, large language models (LLMs) have the potential to be used in various applications. Recent advances in LLMs like ChatGPT have made LLM-based academic tools possible. However, most of the existing studies on the adoption of LLM for academic tasks were based on theoretical or qualitative analyses, which failed to provide empirical evidence on the effects of LLMs on users’ behaviors. Additionally, although previous work has investigated users’ acceptance of conventional conversational systems, little is known about how scholars evaluate LLMs when they are used for academic tasks. Hence, we conducted an empirical field experiment to assess the performance of 48 early-stage scholars on two core academic activities (paper reading and literature reviews) under varying time constraints. Prior to the tasks, participants underwent different training programs about LLM capabilities and limitations. Then, we built a hierarchy dependency network using the Bayesian network. Statistical regression analyses were further conducted to quantify relationships among influential factors of task performance and users’ attitudes toward the LLMs. It was found that young scholars have upheld relatively high academic integrity when using LLMs for academic tasks, and user-LLM performance varied with the task type and time pressure but not with the type of training we used. Further, scholars’ traits can also affect their performance in academic tasks and attitudes towards the LLMs. This work can inspire the future development of LLM-related user training and guide the optimization of LLMs.

Electric vehicles (EVs), though becoming increasingly popular, raise concerns about motion sickness (MS). However, few studies have explored the causes of and solutions to MS in EVs. The regenerative braking (RB), as a unique function in EVs, is believed to cause MS but has not been validated. Thus, we investigated the effect of RB on MS development and explored whether providing auditory motion cues can mitigate MS among passengers. An on-road study with 16 participants who are susceptible to MS was conducted, with the level of RB (low- versus high-level) and auditory motion cues (presence versus absence) as the within-subject factor. Our results confirmed that higher levels of RB can induce MS. Further, providing auditory motion cues can mitigate MS when high-level RB was used. The findings highlight the importance of motion cues in EVs and provide insights into the design of RB systems and corresponding human-machine interaction strategies.

Remote photoplethysmography (rPPG) has been widely applied to measure heart rate from face videos. To increase the generalizability of the algorithms, domain generalization (DG) attracted increasing attention in rPPG. However, when rPPG is extended to simultaneously measure more vital signs (e.g., respiration and blood oxygen saturation), achieving generalizability brings new challenges. Although partial features shared among different physiological signals can benefit multi-task learning, the sparse and imbalanced target label space brings the seesaw effect over task-specific feature learning. To resolve this problem, we designed an end-to-end Mixture of Low-rank Experts for multi-task remote Physiological measurement (PhysMLE), which is based on multiple low-rank experts with a novel router mechanism, thereby enabling the model to adeptly handle both specifications and correlations within tasks. Additionally, we introduced prior knowledge from physiology among tasks to overcome the imbalance of label space under real-world multi-task physiological measurement. For fair and comprehensive evaluations, this paper proposed a large-scale multi-task generalization benchmark, named Multi-Source Synsemantic Domain Generalization (MSSDG) protocol. Extensive experiments with MSSDG and intra-dataset have shown the effectiveness and efficiency of PhysMLE. In addition, a new dataset was collected and made publicly available to meet the needs of the MSSDG. The code and data are available at https://github.com/WJULYW/PhysMLE.

The rapid advancement of large language models (LLMs) such as ChatGPT makes LLM-based academic tools possible. However, little research has empirically evaluated how scholars perform different types of academic tasks with LLMs. Through an empirical study followed by a semi-structured interview, we assessed 48 early-stage scholars’ performance in conducting core academic activities (i.e., paper reading and literature reviews) under different levels of time pressure. Before conducting the tasks, participants received different training programs regarding the limitations and capabilities of the LLMs. After completing the tasks, participants completed an interview. Quantitative data regarding the influence of time pressure, task type, and training program on participants’ performance in academic tasks was analyzed. Semi-structured interviews provided additional information on the influential factors of task performance, participants’ perceptions of LLMs, and concerns about integrating LLMs into academic workflows. The findings can guide more appropriate usage and design of LLM-based tools in assisting academic work.

Electrocardiography (ECG) for objective cognitive load estimation gained increasing attention, and offers a more feasible and non-invasive alternative to traditional methods such as electroencephalography (EEG). Despite the promise of ECG signal, application in real-world scenarios is hampered by the domain shift present in data collected in controlled environments versus real-world settings. We propose a novel plug-in generalizable framework, CogDG-ECG, assessed on a first-introduced multi-source domain generalization (MSDG) protocol for generalized cognitive load estimation. CogDG-ECG bridges the domain gap by extracting domain-invariant features through adversarial learning, and estimating instance-specific unseen features by synthesizing plausible feature statistical variations. A new benchmark based on three datasets and MSDG protocol was introduced, which demonstrates the superiority of our proposed method.

Drivers’ appropriate mental models of and trust in advanced driver assistance systems (ADAS) are essential to driving safety in vehicles with ADAS. Although several previous studies evaluated drivers’ ADAS mental models of and trust in adaptive cruise control and lane-keeping assist systems, research gaps still exist. Specifically, recent developments in ADAS have made more advanced functions available but they have been under-investigated. Furthermore, the widely adopted proportional correctness-based scores may not differentiate drivers’ objective ADAS mental model and subjective bias toward the ADAS. Finally, most previous studies adopted only regression models to explore the influential factors and thus may have ignored the underlying association among the factors. Therefore, our study aimed to explore drivers’ mental models of and trust in emerging ADAS by using the sensitivity (i.e.,d’) and response bias (i.e.,c) measures from the signal detection theory. We modeled the data from 287 drivers using additive Bayesian network (ABN) and further interpreted the graph model using regression analysis. We found that different factors might be associated with drivers’ objective knowledge of ADAS and subjective bias toward the existence of functions/limitations. Furthermore, drivers’ subjective bias was more associated with their trust in ADAS compared to objective knowledge. The findings from our study provide new insights into the influential factors on drivers’ mental models of ADAS and better reveal how mental models can affect trust in ADAS. It also provides a case study on how the mixed approach with ABN and regression analysis can model observational data.

The electrification of vehicle power systems has become a dominant trend worldwide. However, with current technologies, range anxiety is still a major obstacle to the popularization of battery electric vehicles (BEVs). Previous research has found that users’ trust in the BEVs’ range estimation system (RES) is associated with their range anxiety. However, influential factors of trust in RES have not yet been explored. Thus, a questionnaire was designed to model the factors that are directly (i.e., implicit factors) and indirectly (i.e., explicit factors) associated with BEV users’ trust in RES. Following the three-layer automation trust framework (i.e., dispositional trust, situational trust, and learned trust), a questionnaire was designed and administrated online. In total, 367 valid samples were collected from BEV users in mainland China. A mixed approach combining Bayesian network (BN) and regression analyses (i.e., BN–regression mixed approach) was proposed to explore the potential topological relationships among factors. Four implicit factors (i.e., sensitivity to BEV brand, knowledge of RES, users’ emotional stability, and trust in the battery estimation system of their phones) have been found to be directly associated with BEV users’ trust in RES. Furthermore, four explicit factors (i.e., users’ highest education, regional charging infrastructure development, BEV brand, and household income) were found to be indirectly associated with users’ trust in RES. This study further demonstrates the effectiveness of using a BN–regression mixed approach to explore topological relationships among social–psychological factors. Future strategies aiming to modulate trust in RES can target toward factors in different levels of the topological structure.

The utilization of remote photoplethysmography (rPPG) technology has gained attention in recent years due to its ability to extract blood volume pulse (BVP) from facial videos, making it accessible for various applications such as health monitoring and emotional analysis. However, the BVP signal is susceptible to complex environmental changes or individual differences, causing existing methods to struggle in generalizing for unseen domains. This article addresses the domain shift problem in rPPG measurement and shows that most domain generalization methods fail to work well in this problem due to ambiguous instance-specific differences. To address this, the article proposes a novel approach called Hierarchical Style-aware Representation Disentangling (HSRD). HSRD improves generalization capacity by separating domain-invariant and instance-specific feature space during training, which increases the robustness of out-of-distribution samples during inference. This work presents state-of-the-art performance against several methods in both cross and intra-dataset settings.

Remote photoplethysmography (rPPG) is a contactless technique that facilitates the measurement of physiological signals and cardiac activities through facial video recordings. This approach holds tremendous potential for various applications. However, existing rPPG methods often did not account for different types of occlusions that commonly occur in real-world scenarios, such as temporary movement or actions of humans in videos or dust on camera. The failure to address these occlusions can compromise the accuracy of rPPG algorithms. To address this issue, we proposed a novel Condiff-rPPG to improve the robustness of rPPG measurement facing various occlusions. First, we compressed the damaged face video into a spatio-temporal representation with several types of masks. Second, the diffusion model was designed to recover the missing information with observed values as a condition. Moreover, a novel low-rank decomposition regularization was proposed to eliminate background noise and maximize informative features. ConDiff-rPPG ensured consistency in optimization goals during the training process. Through extensive experiments, including intra- and cross-dataset evaluations, as well as ablation tests, we demonstrated the robustness and generalization ability of our proposed model.