Exploring 3D surface data often involves navigating complex menus, creating challenges for both experts and new users who may struggle with overloaded interfaces. To address this, we introduce VocalVis , an open-source prototype that leverages real-time voice processing to simplify interaction and analysis of 3D surface data. By capitalizing on advancements in natural language processing, VocalVis enables users to interact with digital content through voice commands, addressing challenges associated with traditional navigation methods. Our study involves both data scientist novices and domain experts, offering insights into how voice interaction can streamline data exploration. The findings demonstrate the potential of voice as a powerful supplementary tool for visually analyzing complex datasets. This innovative approach opens up new possibilities for intuitive, accessible data exploration, making it convenient for a broader range of users to engage with 3D scientific data.

Using speech for hands-free interaction in Virtual Reality (VR) is gaining popularity, supported by advances in natural language processing that enable accurate speech-to-text transcription and intent inference. Prior work has shown that voice-based navigation is effective when hand use is restricted or occupied, particularly when visually salient points in the scene, known as landmarks, can be identified and articulated. However, in many virtual environments, such landmarks may be absent or not easily recognizable, limiting the applicability of voice commands. To address this, we propose three landmark-free, user-centered coordinate systems to support speech-based locomotion in VR: Cartesian, cylindrical, and spherical. Each system uses a distinct encoding and interaction style. The Cartesian system employs a simple three-digit code and requires low cognitive effort. The spherical system prioritizes precision but demands higher mental effort. The cylindrical system combines elements of both, offering a balance between usability and accuracy. We evaluated these systems in a quantitative user study with 24 participants, all with backgrounds in information technology. The study assessed the systems’ effectiveness for object positioning and navigation, comparing them to teleportation, the standard non-voice locomotion method in VR. Results show that the Cartesian system enables faster and more intuitive navigation compared to the spherical system. Distinct usage patterns were observed, with users predominantly focusing on the central field of view during navigation. Our findings indicate that user-centered coordinate systems are practical to implement and present a viable alternative for speech-driven, hands-free navigation in VR.

In various situations, such as clinical environments with sterile conditions or when hands are occupied with multiple devices, traditional methods of navigation and scene adjustment are impractical or even impossible. We explore a new solution by using voice control to facilitate interaction in virtual worlds to avoid the use of additional controllers. Therefore, we investigate three scenarios: Object Orientation, Visualization Customization, and Analytical Tasks and evaluate whether natural language interaction is possible and promising in each of these scenarios. In our quantitative user study participants were able to control virtual environments effortlessly using verbal instructions. This resulted in rapid orientation adjustments, adaptive visual aids, and accurate data analysis. In addition, user satisfaction and usability surveys showed consistently high levels of acceptance and ease of use. In conclusion, our study shows that the use of natural language can be a promising alternative for the improvement of user interaction in virtual environments. It enables intuitive interactions in virtual spaces, especially in situations where traditional controls have limitations. • Presenting an alternative, hands-free interaction system using real-time natural language processing. • Evaluation of VUIs in VR, spanning three primary categories: Orientation, Customization, and Analysis. • Exploring whether voice interaction can serve as a feasible mode of communication with visualizations. • Offering an open-source project to enhance accessibility for voice-based applications.

As locomotion is an important factor in improving Virtual Reality (VR) immersion and usability, research in this area has been and continues to be a crucial aspect for the success of VR applications. In recent years, a variety of techniques have been developed and evaluated, ranging from abstract control, vehicle, and teleportation techniques to more realistic techniques such as motion, gestures, and gaze. However, when it comes to hands-free scenarios, for example to increase the overall accessibility of an application or in medical scenarios under sterile conditions, most of the announced techniques cannot be applied. This is where the use of speech as an intuitive means of navigation comes in handy. As systems become more capable of understanding and producing speech, voice interfaces become a valuable alternative for input on all types of devices. This takes the quality of hands-free interaction to a new level. However, intuitive user-assisted speech interaction is difficult to realize due to semantic ambiguities in natural language utterances as well as the high real-time requirements of these systems. In this paper, we investigate steering-based locomotion and selection-based locomotion using three speech-based, hands-free methods and compare them with leaning as an established alternative. Our results show that landmark-based locomotion is a convenient, fast, and intuitive way to move between locations in a VR scene. Furthermore, we show that in scenarios where landmarks are not available, number grid-based navigation is a successful solution. Based on this, we conclude that speech is a suitable alternative in hands-free scenar-ios, and exciting ideas are emerging for future work focused on developing hands-free ad hoc navigation systems for scenes where landmarks do not exist or are difficult to articulate or recognize.

Since for most consumers the Virtual Reality (VR) experience exceeds that of desktop applications, an increasing number of applications is being transferred from desktop to VR. Industrial and entertainment applications primarily expect for a richer consumer experience, while others, such as surgical applications, seek for improved precision over their desktop counterparts. One way to improve the performance of precision-based VR applications is to provide suitable visualizations. Today, these "suitable" visualizations are mostly transferred from desktop to VR without considering their spatial and temporal performance might change in VR. This may not lead to an optimal solution, which can be crucial for precision-based tasks. Misinterpretation of shape or distance in a surgical or pre-operative simulation can affect the chosen treatment and thus directly impact the outcome. Therefore, we evaluate the performance differences of multiple visualizations for 3D surfaces based on their shape and distance estimation for desktop and VR applications. We conducted a quantitative user study with 56 participants evaluating seven visualizations (Phong, Toon, Fresnel, Pseudo-Chromadepth, Heatmap, Isolines, and Arrow Glyphs). Our results show that the performance of each visualization varies depending on the task, system, and surface type, with VR generally providing improved results. While Isolines are able to improve distance estimation, Phong and Heatmaps are beneficial for shape estimation.