Arm swing is a viable approach for locomotion in virtual reality (VR). However, previous studies have not investigated the effects of arm posture (straight arms and bent arms) and speed estimation methods on speed estimation error, locomotion performance, and user preference. We proposed two data-driven approaches for virtual locomotion based on a one-dimensional convolutional neural network (1-D CNN) and the support vector regression (SVR), respectively, and compared their speed estimation errors using treadmill walking data for both bent arms and straight arms. We then compared the locomotion performance and user preference of the two methods against two existing virtual locomotion methods through virtual locomotion tasks. Experimental results suggest that it is necessary to consider the influence of arm posture and speed estimation methods when designing virtual locomotion methods based on arm swing. The results of this work provide insights into the research and the design of arm-swing-based locomotion interfaces.

ABSTRACT Reliable estimation of torso direction is essential for virtual locomotion as it controls the locomotion direction in virtual environments. Existing approaches rely on external sensors to track torso direction or use the poses and/or movements from the head, hands, or feet to control the direction of travel. In addition, deep learning methods based on a head‐mounted display (HMD) and its controllers were proposed for torso direction estimation but are unable to produce continuous estimates or that the estimation errors were not systematically evaluated. On the other hand, 3D full‐body human motion generation methods based on an HMD and its controllers are capable of estimating torso direction but this capability was largely ignored for its use in virtual steering. We proposed a new method called TCNPoser that can perform torso direction estimation and generate 3D full‐body human motion. Through offline evaluation and a user study, we show that the proposed method achieves the state‐of‐the‐art (SOTA) performance for real‐time torso direction estimation.

Body actions and head gestures are natural interfaces for interaction in virtual environments. Existing methods for in-place body action recognition often require hardware more than a head-mounted display (HMD), making body action interfaces difficult to be introduced to ordinary virtual reality (VR) users as they usually only possess an HMD. In addition, there lacks a unified solution to recognize in-place body actions and head gestures. This potentially hinders the exploration of the use of in-place body actions and head gestures for novel interaction experiences in virtual environments. We present a unified two-stream 1-D convolutional neural network (CNN) for recognition of body actions when a user performs walking-in-place (WIP) and for recognition of head gestures when a user stands still wearing only an HMD. Compared to previous approaches, our method does not require specialized hardware and/or additional tracking devices other than an HMD and can recognize a significantly larger number of body actions and head gestures than other existing methods. In total, ten in-place body actions and eight head gestures can be recognized with the proposed method, which makes this method a readily available body action interface (head gestures included) for interaction with virtual environments. We demonstrate one utility of the interface through a virtual locomotion task. Results show that the present body action interface is reliable in detecting body actions for the VR locomotion task but is physically demanding compared to a touch controller interface. The present body action interface is promising for new VR experiences and applications, especially for VR fitness applications where workouts are intended.