Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cultural contexts, respect local sensitivities, and support low-resource languages, all while effectively integrating corresponding visual cues. In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages. ALM-bench challenges existing models by testing their ability to understand and reason about culturally diverse images paired with text in various languages, including many low-resource languages traditionally underrepresented in LMM research. The benchmark offers a robust and nuanced evaluation framework featuring various question formats, including true/false, multiple choice, and open-ended questions, which are further divided into short and long-answer categories. ALM-bench design ensures a comprehensive assessment of a model’s ability to handle varied levels of difficulty in visual and linguistic reasoning. To capture the rich tapestry of global cultures, ALM-bench carefully curates content from 13 distinct cultural aspects, ranging from traditions and rituals to famous personalities and celebrations. Through this, ALM-bench not only provides a rigorous testing ground for state-of-the-art open and closed-source LMMs but also highlights the importance of cultural and linguistic inclusivity, encouraging the development of models that can serve diverse global populations effectively. Our benchmark is publicly available at https://mbzuai-oryx.github.io/ALM-Bench/.

In this paper, we review the existing benchmarks for continuous gesture recognition, e.g., the online analysis of hand movements over time to detect and recognize meaningful gestures from a specific dictionary. Focusing on human–computer interaction scenarios, we classify these benchmarks based on input data types, gesture dictionaries, and evaluation metrics. Specific metrics for the continuous recognition task are crucial for understanding how effectively gestures are spotted in real time within input streams. We also discuss the most effective detection and classification methods proposed for these benchmarks. Our findings indicate that the number and quality of publicly available datasets remain limited, and evaluation methodologies for continuous recognition are not yet standardized. These issues highlight the need for new benchmarks that reflect real-world usage conditions and can support the development of best practices in gesture-based interface design.

An underexplored interaction metaphor in virtual reality (VR) is throwing, with a considerable challenge in achieving accurate and natural results. We conducted an empirical investigation of participants’ performance in a VR throwing task, measuring their accuracy and preferences across Point of Release (PoR) mechanics (manual and automatic) with various input device categories (hand-held, on-body, external) and throwable object types. Participants were tasked with throwing a baseball, a bowling ball, and a football toward targets using 5 input configurations (2 manual and 3 automatic PoR). Results from 30 participants indicate that the overall highest accuracy was achieved with an automatic PoR configuration (on-body tracker). The post-study and VR survey results indicate that the majority of participants preferred a manual PoR configuration (hand-held VR controller-derived) for the throwing direction, throwing speed, and as being the closest to real-life throwing. Our findings are useful for VR researchers and developers who want to implement throwing as a technique in their applications.

Object selection has been explored extensively in the VR research literature. However, the research is typically conducted in constrained experimental setups. It remains unclear whether the designed selection techniques fit the prevalent practical uses and whether the experimental tasks represent important challenges in real applications. To identify and help bridge these gaps, we surveyed current consumer VR applications, containing 206 popular VR game and 3D modeling applications. We extracted 1300+ selection scenarios based on video analyses of these applications and derived a taxonomy to understand common patterns on where and how selections occur. Our findings reveal significant gaps in selection tasks and techniques between research and consumer applications. We also present an interactive visualization tool to help researchers explore the VR object selection scenarios. Finally, we discuss how our work can help researchers and developers evaluate techniques in meaningful tasks and drive the design of techniques.

Virtual Reality (VR) systems are widely used, and it is essential to know if spatial perception in virtual environments (VEs) is similar to reality. Research indicates that users tend to underestimate distances in VR. Prior work suggests that actual distance judgments in VR may not always match the users self-reported preference of where they think they most accurately estimated distances. However, no explicit investigation evaluated whether user preferences match actual performance in a spatial judgment task. We used blind walking to explore potential dissimilarities between actual distance estimates and user-selected preferences of visual complexities, VE conditions, and targets. Our findings show a gap between user preferences and actual performance when visual complexities were varied, which has implications for better visual perception understanding, VR applications design, and research in spatial perception, indicating the need to calibrate and align user preferences and true spatial perception abilities in VR.

We present a taxonomy of human-centered collaborative XR tasks. XR technologies have extended into the realm of collaboration, improving the quality and accessibility of teamwork. However, after a comprehensive assessment of the literature on the interaction between XR technologies and collaboration, no comprehensive method that emphasizes task actions and properties exists to classify collaborative tasks. Thus, our suggested taxonomy represents a classification system for collaborative tasks. After conducting a thorough literature review across different research venues, we conducted several exhaustive classification and review cycles for over 800 papers collected, which resulted in 148 papers retained to create the taxonomy. We dissected the actions and properties that the collaborative endeavors and tasks of these papers encompass as well as the types of categorizations and relations these papers illustrate. We expand on the design choices and usage of our taxonomy, followed by its limitations and future work. We built this taxonomy in order to reduce ambiguities and confusion regarding the design and comprehension of human-based collaborative tasks that use XR technology, which could prove useful in aiding the development and understanding of these tasks. Our taxonomy reveals a framework for understanding how collaborative tasks are designed and a systematic way of classifying different methods by which people can collaborate and interact in environments that involve XR, while still promoting efficient communication, teamwork, goal achievement and productivity.