As vehicle automation advances to SAE Levels 3 to 5, transitioning driving control from human to system, ensuring automated vehicles (AVs) align with user preferences becomes a challenge. Natural interaction emerges as a common goal, offering ways to convey user interests in a user-friendly manner. However, technical, legal, or design constraints may prevent fulfilling these preferences, leading to potential conflicts. Through an online survey (N=50), potential driver-passenger conflicts and their handling strategies were explored. Subsequently, in a Virtual Reality study (N=14), we applied identified strategies (ranging from distracting to motivating and adhering to social norms) to user-AV interactions using a state-of-the-art language model (GPT-4 Turbo) primed with the strategies to simulate realistic dialogues. Additionally, adaptive communication was compared to non-adaptive communication. Our findings reveal a preference for adaptive communication. Yet, despite using advanced modeling, accurately predicting user interactions remained challenging, with users often trying to outsmart the AI.

Minor violations of traffic regulations are common today and partially socially accepted. Automated vehicles (AVs), however, will be obliged to keep to the letter of the law. This can lead to situations where user requests cause the AV to reach its legal boundaries, creating novel user-vehicle conflicts. To investigate whether traffic-violating driver interests are transferred to the automated context, we conducted an online survey with three conflict-prone scenarios (N=49). The results indicate that legally compliant AV behavior is desired but that users would intervene in the vehicle’s behavior to enforce interests. In a subsequent Virtual Reality study (N=30), we evaluated the effect of legal boundary-handling strategies (Responsibility and Control Shift, Responsibility Shift, No Shift) and other traffic participants’ violating traffic regulations on behavior, conflict, and trust in a legally conflict-prone parking scenario. Results show that conflict is perceived significantly higher in all strategies compared to the manual baseline, while situational trust in the vehicle is higher in the automated conditions but independent of the handling strategy.

Novel display technologies, such as lightfield displays, have become increasingly available. In the automotive domain, these are already in use or are to be used in the future. However, their effect on the user is yet to be explored. Therefore, we conducted a within-subject study (N=15) comparing a baseline visualization of information about automated vehicle functionality with it being visualized either via a LumePad or a LookingGlass display. Interestingly, we found almost no significant differences, thus, indicating that the display technology is less relevant for conveying automated functionality.

The inclusion of in-vehicle sensors and increased intention and state recognition capabilities enable implicit in-vehicle interaction. Starting from a systematic literature review (SLR) on implicit in-vehicle interaction, which resulted in 82 publications, we investigated state and intention recognition methods based on (1) their used modalities, (2) their underlying level of automation, and (3) their considered interaction focus. Our SLR revealed a research gap addressing implicit interaction in highly automated vehicles (HAVs). Therefore, we discussed how the requirements for implicit state and intention recognition methods and interaction based on them are changing in HAVs. With this, open questions and opportunities for further research in this area were identified.