Wenxuan Xu, Arvind Pillai, Subigya Nepal, Amanda C. Collins, Daniel M Mackin, Michael V. Heinz, Tess Z Griffin, Nicholas C. Jacobson, Andrew Campbell. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Virtual reality (VR) allows users to observe and manipulate 3D geometry from multiple viewpoints. Most VR selection work, however, optimizes techniques for selecting entire objects. Selecting a single face on a polyhedron remains underexplored and is more challenging because the interaction must act on a local component while preserving the object's global structure. We introduce a design space tailored to this task with three dimensions: viewing strategy, disambiguation consistency, and interaction metaphor. Guided by this space, we design eight freehand techniques for polyhedral face selection. A within-subjects study with 16 participants evaluates these techniques across polyhedral complexity (two radii; face counts 4, 6, and 12). The results identify three complementary top techniques, reveal tradeoffs between viewing choices and geometric preservation, and yield concrete guidelines for matching techniques to target geometry and task demands. A follow-up study with complex, realistic models confirms the robustness and practical usability of the three techniques. Together, these contributions shift attention from whole object selection to precise component selection in VR and provide actionable methods for 3D modeling, assembly, and texturing.

Proximity-based feedback provides users with real-time guidance as they approach an interaction goal. This type of feedback is particularly useful for tasks that require guidance during the interaction process, such as selecting moving targets. This work explores proximity-based feedback types and modalities to improve the selection of moving targets in VR by leveraging three feedback types that combine visual, auditory, and haptic modalities. We evaluated the performance of these mechanisms through two user studies, analyzing both objective data (e.g., selection time, error rate) and subjective data (e.g., user experience, preferences) to explore the characteristics of feedback types across different modalities and to examine the roles of various modalities within multimodal combinations. Our findings suggest optimal selection mechanisms for developers and should be tailored to different goals: achieving user precision, enabling quick movement to a target, considering task duration, and enhancing entertainment value. We also discuss applications that correspond to these different perspectives.

Target selection is one of the most fundamental tasks in VR interaction systems. Prediction heuristics can provide users with a smoother interaction experience in this process. Our work aims to predict the ray landing pose for hand-based raycasting selection in Virtual Reality (VR) using a Long Short-Term Memory (LSTM)-based neural network with time-series data input of speed and distance over time from three different pose channels: hand, Head-Mounted Display (HMD), and eye. We first conducted a study to collect motion data from these three input channels and analyzed these movement behaviors. Additionally, we evaluated which combination of input modalities yields the optimal result. A second study validates raycasting across a continuous range of distances, angles, and target sizes. On average, our technique’s predictions were within 4.6° of the true landing Pose when 50% of the way through the movement. We compared our LSTM neural network model to a kinematic information model and further validated its generalizability in two ways: by training the model on one user’s data and testing on other users (cross-user) and by training on a group of users and testing on entirely new users (unseen users). Compared to the baseline and a previous kinematic method, our model increased prediction accuracy by a factor of 3.5 and 1.9, re spectively, when 40% of the way through the movement.

Piloting requires users to control and navigate the aircraft within a designated pathway, with a controller that utilizes two joysticks to control the aircraft. This task is representative of various daily and gaming scenarios, such as controlling the aircraft to capture the photo or navigating an object in a game from the start position to the end via a trajectory. In this work, we explore a model (based on the Steering Law) that predicts the piloting time required in spatial-constrained environments. Thus, two user studies are conducted to help us understand the relationship between path complexity (curvature) and spatial constraints (width and height). According to the results, we propose a model that can achieve $52.6 \%$ and $60.6 \%$ improvement in R-square and the Akaike Information Criterion (AIC), respectively. Next, an additional study was conducted to further verify the performance and efficiency of our proposed model with the change of movement direction and orientation. Our model and experimental results can benefit both game and interface designers of applications that require controlling moving objects along specific trajectories in virtual reality environments.