Vision-Language Models (VLMs) are known to struggle with spatial reasoning and visual alignment. To help overcome these limitations, we introduce iVISPAR, an interactive multimodal benchmark designed to evaluate the spatial reasoning capabilities of VLMs acting as agents. iVISPAR is based on a variant of the sliding tile puzzle—a classic problem that demands logical planning, spatial awareness, and multi-step reasoning. The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs’ planning and reasoning skills. We evaluate a broad suite of state-of-the-art open-source and closed-source VLMs, comparing their performance while also providing optimal path solutions and a human baseline to assess the task’s complexity and feasibility for humans. Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance, illustrating the persistent challenge of visual alignment. This underscores critical gaps in current VLM capabilities, highlighting their limitations in achieving human-level cognition. Project website: https://microcosm.ai/ivispar.

The successful integration of Autonomous Vehicles (AVs) into daily life is influenced by various individual and contextual factors. This study examines key factors affecting acceptance and trust in AVs by assessing how interactive experiences influence user perceptions and adoption. Utilizing a Virtual Reality (VR) environment to simulate real-world driving conditions, the research investigates the impact of different levels of vehicle interaction—manual control, semi-autonomous, fully autonomous, and taxi-riding modes—on user and passenger trust, anxiety, perceived ease of use, and overall acceptance. A dual-method approach was employed, combining objective data collection through eye-tracking, which measures engagement and vigilance, with subjective user feedback gathered via the Autonomous Vehicle Acceptance Model (AVAM) questionnaire. The findings reveal a complex relationship between user control and trust in AV systems. While semi-autonomous conditions that incorporate user participation in decision-making enhance trust and intention to use, fully autonomous conditions influence trust but not intention to use significantly. However, semi- autonomous conditions also elicited higher anxiety levels compared to fully autonomous modes, suggesting a trade-off between interaction and comfort. Engagement levels, as indicated by visual metrics such as gaze patterns, were found to be critical for understanding user acceptance. These results highlight the importance of considering both subjective perceptions and objective behaviors in developing AV interfaces. The study contributes significant insights into the cognitive underpinnings of AV acceptance and underscores the need for strategies that both enhance user trust and minimize anxiety by finding an optimal balance between automation and user control. Further research into this balance is recommended to better accommodate the evolving nature of user trust and the cognitive complexities associated with semi-autonomous systems.

Natural eye movements have primarily been studied for over-learned activities such as tea-making, sandwich-making, and hand-washing, which have a fixed sequence of associated actions. These studies demonstrate a sequential activation of low-level cognitive schemas facilitating task completion. However, whether these action schemas are activated in the same pattern when a task is novel and a sequence of actions must be planned in the moment is unclear. Here, we recorded gaze and body movements in a naturalistic task to study action-oriented gaze behavior. In a virtual environment, subjects moved objects on a life-size shelf to achieve a given order. To compel cognitive planning, we added complexity to the sorting tasks. Fixations aligned with the action onset showed gaze as tightly coupled with the action sequence, and task complexity moderately affected the proportion of fixations on the task-relevant regions. Our analysis revealed that gaze fixations were allocated to action-relevant targets just in time. Planning behavior predominantly corresponded to a greater visual search for task-relevant objects before the action onset. The results support the idea that natural behavior relies on the frugal use of working memory, and humans refrain from encoding objects in the environment to plan long-term actions. Instead, they prefer just-in-time planning by searching for action-relevant items at the moment, directing their body and hand to it, monitoring the action until it is terminated, and moving on to the following action.

Autonomous vehicles represent a significant development in our society, and their acceptance will largely depend on trust. This study investigates strategies to increase trust and acceptance by making the cars’ decisions. For this purpose, we created a virtual reality (VR) experiment with a self-explaining autonomous car, providing participants with verbal cues about crucial traffic decisions. First, we investigated attitudes toward self-driving cars among 7850 participants using a simplified version of the Technology Acceptance Model (TAM) questionnaire. Results revealed that female participants are less accepting than male participants, and that there is a general decline among all genders. Otherwise in general, a self-explaining car has a positive impact on trust and perceived usefulness. Surprisingly, it adversely affected the intention to use and perceived ease of use. This entails dissociation of trust from the other items of the questionnaire. Second, we analyzed behavioral of 26 750 participants to investigate the effect of self-explaining systems on head movements during the VR drive. We observed significant differences in head movements during the critical events and the baseline periods of the drive between the three driving conditions. Additionally, we demonstrated positive correlations between head movement parameters and the TAM scores, where trust showed the lowest correlation. This provides further evidence of the dissociation of trust from other TAM items. These findings illustrate the benefits of combining subjective questionnaire data with objective behavioral data. Overall, the outcomes indicate a partial dissociation of self-reported trust from intention to use and objective behavioral data.