This paper presents an N-ary Gaussian Model for predicting endpoint distributions in pointing tasks across task scenarios. Built on the foundational principles of the Ternary Gaussian model series, our model framework allows researchers to define parameter constraints and automatically refine model combinations, eliminating the need for predefined equations based on data analysis. We utilize the Bayesian Information Criterion (BIC) for model selection, ensuring simplicity while maintaining predictive accuracy. We conducted a comparative analysis against published baselines across 7 diverse datasets, covering 1D, 2D, and 3D tasks, different input modalities, different display devices, and time-constrained scenarios, demonstrating the robustness and generalization of the N-ary Gaussian Model. The N-ary Gaussion model offers an automated solution for modeling pointing uncertainty, and also incorporates cross output device, input modality, and temporal constraint factors into spatial pointing uncertainty modeling for the first time.

While user preferences are important to cross-domain recommendation (CDR), existing methods primarily discover preferences under specific, yet possibly redundant, item features. To this end, we first propose a novel Preference Prototype-Aware (PPA) learning method to quantitatively learn user preferences while minimizing disturbances from the source domain. It introduces a mix-encoder and a proto-decoder. On the one hand, the mix-encoder learns better general representations of interacted items and captures the intrinsic relationships between items across different domains. On the other hand, the proto-decoder implements a learnable prototype matching mechanism to quantitatively perceive user preferences, avoiding disturbances caused by item features from the source domain. Moreover, through experiments on PPA, we observe another two issues that affect existing CDR methods’ performance, i.e., the semantic deficiency caused by sparse item categories and the imbalance weights caused by different user-item distributions. Thus, we further propose a LoRA-based extractor and a domain cross-attention module to alleviate the two issues, respectively. The PPA incorporating with new extractor and attention module is called PPA++. Extensive experiments show that PPA++ outperforms the other state-of-the-art counterparts in four different CDR scenarios.

In this work, we present an overview of current selfie techniques and state-of-the-art selfie systems that aim to simplify and enhance the experience of capturing excellent selfies. We contribute a comprehensive framework that summarizes the process of selfie interaction, categorizing the diverse techniques employed at each phase of the selfie-capturing procedure. Following our review of selfie techniques, we offer recommendations for the design of selfie cameras, interaction methods, and evaluation criteria for selfie interaction techniques. Our reviews on selfie-related technologies hold the potential to enrich human-computer interaction research in this domain and can serve as a valuable resource for informing marketing strategies for both hardware and software selfie applications.

This study explores the design space of two-handed input (i.e., clicking or tapping with the thumb) on the touchpads of controllers for virtual reality (VR) interaction. Four experiments were conducted to fulfill this purpose. Experiment 1 investigated how users employed two VR controllers to perform four representative interaction tasks in VR and identified 14 potentially usable two-handed operations that involved tapping or clicking. Experiments 2 and 3 analyzed user performance of the 14 operations, providing insights into their interaction characteristics in terms of completion time, accuracy, and subjective feedback. In Experiment 4, we designed a command-input technique based on the proposed operations. We verified its effectiveness compared to context menus and marking menus in a VR text entry scenario. Our technique generally had shorter times and similar accuracy to the two menu types. Our work contributes to the design of VR interactions using two-handed controllers.

In the era of Big Data, augmented reality (AR)-based 3D visualization technology is gradually becoming an essential tool of effective information dissemination. However, 3D data visualization inevitably faces a series of challenges, particularly when visualizing large datasets within limited physical spaces. This often results in high spatial density, characterized by smaller target objects and occlusion of data objects, which greatly complicate object selection and subsequent interaction. To address these issues, this paper proposes a multimodal progressive target selection technique-StarPicker, which integrates the SpotLight metaphor and Clock metaphor. By leveraging wrist rotation and gesture interactions, StarPicker facilitates a coarse-to-fine semantic disambiguation and precise selection of target objects. To validate the effectiveness of StarPicker, a comparative user study was conducted against the state-of-the-art techniques (GridWall and FlowerCone) under high-density (up to 360 objects) and small-target (1cm) conditions. Experimental results demonstrate that StarPicker significantly outperforms the baselines in terms of target selection accuracy and user satisfaction, while achieving comparable or better completion times, especially in denser scenes. This work offers a novel approach to target selection and interaction technology in the field of AR-based 3D Big Data visualization.

Various locomotion techniques, such as teleportation, walking in place, redirected walking, and walking, have been proposed. However, conventional methods overlooked the human ability to coordinate multiple modalities (e.g., eyes and feet) for natural virtual locomotion within a small physical space. Inspired by the natural coordination of the eyes and feet in human walking, as well as insights from prior work, this study investigates how eye-foot coordination can be leveraged more effectively for VR teleportation. We present four novel teleportation techniques based on eye-foot coordination, using users' gaze behavior and 3D foot positions as input modalities. A user study with 20 participants compared our techniques with a state-of-the-art foot-based locomotion method. Results demonstrate the superiority of our approaches: task completion times were significantly reduced, NASA-TLX workload scores and SUS usability scores were markedly improved, and participants expressed a clear preference for our techniques over the baseline. These findings provide a strong foundation for the design and implementation of future eye-foot coordinated teleportation methods in VR.

Behavioral authentication has become increasingly popular as a natural method for authentication in Virtual Reality (VR). However, existing studies often overlook the fact that users may perform behavioral authentication in different postures (i.e., sitting, standing, reclining) during VR use. Therefore, understanding how posture variations affect classification accuracy is crucial for designing posture-robust systems. In this study, we conducted a controlled experiment (N = 30) to investigate the impact of posture on classification accuracy during a target-selection task. We collected behavioral trajectory data and analyzed it using multivariate time series classification algorithms, addressing authentication performance under three different postures. In a within-posture authentication, reclining took longer but achieved the highest classification accuracy, with an interaction effect between posture and target vertical layout. In cross-posture authentication, transfers from sitting to standing/reclining were more effective than direct transfers between standing and reclining, with vertical layout crucial for classification accuracy. In mixed-posture training, the cross-posture classification accuracy increased, particularly when standing and reclining data were combined to help the model indirectly learn features of sitting posture. These findings provide valuable insights for designing tasks and data collection strategies that support the development of robust cross-posture authentication systems.

Dragging is a fundamental interaction technique in human-computer interaction. Prior research on dragging has generally been concerned with 2D objects on 2D interactive surfaces, with limited attention to 3D interactive spaces. We performed three experiments to systematically investigate dragging using raycasting in virtual reality (VR), a representative 3D interactive environment. The first experiment examined dragging 2D targets on 2D surfaces placed in VR. The second experiment investigated dragging 3D targets positioned at identical depths in VR. Both experiments examined the effects of target depth, angular target amplitude, and angular target on dragging performance. The third experiment investigated dragging 3D targets with varying depths, focusing on the effects of target layout, target amplitude, and target width. Findings revealed the different impacts of the above factors on dragging interaction. We also proposed a new 3D Fitts' law model tailored for dragging using raycasting in VR, which fits well with our data. Finally, we discuss our results and provide design recommendations for dragging (i.e., translational positioning) tasks using raycasting under the tested experimental conditions in VR.

Investigating users' plausible preferences for body-centric locomotion can help researchers better understand the impact of different factors on this locomotion method and optimize the locomotion configuration for providing a plausible locomotion experience. In this paper, we propose to evaluate users' plausible preferences for body-centric locomotion using a reinforcement learning method. This method can intelligently infer users' plausible preferences for different factor levels involved in the virtual locomotion by proposing possible modifications to the factor levels and asking users to accept or reject the modifications after experiencing the locomotion. We conducted a within-subject experiment to examine the impact of different factors (i.e., body parts used for virtual locomotion, the point of view, auditory feedback, the transfer function, and the coefficients of the transfer function) on users' plausible preferences for body-centric locomotion in sitting and standing postures. The results mainly indicated that (1) users preferred using arm swinging in standing posture, whereas they preferred using head tilting in sitting posture; (2) The point of view was identified as the most important factor in standing posture, while it was less important in sitting posture; (3) Participants showed consistent plausible preferences for auditory feedback, transfer function, and coefficient of the transfer function in both standing and sitting postures. Our research findings can guide the design of VR applications to enhance a plausible walking experience in different postures.

Current eye-gaze interaction technologies for smartphones are considered inflexible, inaccurate, and power-hungry. These methods typically rely on hand involvement and accomplish partial interactions. In this paper, we propose a novel eye-gaze smartphone interaction method named Event-driven Eye-Gaze Operation (E2GO), which can realize comprehensive interaction using only eyes and gazes to cover various interaction types. Before the interaction, an anti-jitter gaze estimation method was exploited to stabilize human eye fixation and predict accurate and stable human gaze positions on smartphone screens to further explore refined time-dependent eye-gaze interactions. We also integrated an event-triggering mechanism in E2GO to significantly decrease its power consumption to deploy on smartphones. We have implemented the prototype of E2GO on different brands of smartphones and conducted a comprehensive user study to validate its efficacy, demonstrating E2GO‘s superior smartphone control capabilities across various scenarios. Demo videos

Cross-modality visible-infrared person reidentification (VI-ReID), which aims to retrieve pedestrian images captured by both visible and infrared cameras, is a challenging but essential task for smart surveillance systems. The huge barrier between visible and infrared images has led to the large cross-modality discrepancy and intraclass variations. Most existing VI-ReID methods tend to learn discriminative modality-sharable features based on either global or part-based representations, lacking effective optimization objectives. In this article, we propose a novel global-local multichannel (GLMC) network for VI-ReID, which can learn multigranularity representations based on both global and local features. The coarse- and fine-grained information can complement each other to form a more discriminative feature descriptor. Besides, we also propose a novel center loss function that aims to simultaneously improve the intraclass cross-modality similarity and enlarge the interclass discrepancy to explicitly handle the cross-modality discrepancy issue and avoid the model fluctuating problem. Experimental results on two public datasets have demonstrated the superiority of the proposed method compared with state-of-the-art approaches in terms of effectiveness.

Current behavioral authentication systems in Virtual Reality (VR) require sustained focused interaction during task execution - an assumption frequently incompatible with real-world constraints across two factors: (1) physical limitations (e.g., restricted hand/eye mobility), and (2) psychological barriers (e.g., task-switching fatigue or break-in-presence). To address this attentional gap, we propose a design framework bridging focused and casual interactions in behavior-based VR authentication (Casual-VRAuth). Based on this framework, we designed an authentication prototype using a modified ball-and-tunnel task (propelling a ball along a circular path), supporting three interaction modes: baseline Touch, and two eyes-free options (Hover/Tapping). Experimental results demonstrate that our framework effectively guides the design of authentication systems with varying interaction engagement levels (Touch > Hover > Tapping) to accommodate scenarios requiring casual interaction (e.g., multitasking or eyes-free operation). Furthermore, we revealed that reducing interaction engagement enhances resistance to mimicry attacks while decreasing cognitive workload and error rates in multitasking or eyes-free environments. However, this approach compromises the average classification accuracy of Interaction behavior under different algorithms (including InceptionTime, FCN, ResNet, CNN, MLP, and MCDCNN). Notably, moderate reduction of interaction engagement enhances authentication speed, while excessive reduction may conversely slow it down. Overall, our work establishes a novel design paradigm for VR authentication that supports casual interactions and offers valuable insights into balancing usability and security.

Body-centric locomotion in Virtual Reality (VR) involves multiple factors, including the point of view, avatar representations, tracked body parts for locomotion control and transfer functions that map body movement to the displacement of the virtual viewpoint. Understanding the role of these factors in evoking a plausible walking experience using within- or between-subject experimental designs based on questionnaires and/or objective measurements can be time-consuming and challenging due to the interrelated effects of these factors. This study employed the subjective matching method to evaluate the sense of plausible walking experience during body-centric locomotion in VR. Five relevant factors that may affect locomotion experience were identified by analyzing existing studies, i.e., point of view, the avatar appearance, body parts for locomotion control, transfer functions and the coefficients of transfer functions. A virtual locomotion experiment with these five factors based on subjective matching was conducted. Results showed that participants regarded the point of view as the most critical factor for walking experience enhancement, followed by body parts, transfer functions, the coefficients of transfer functions and finally the avatar appearance. Additionally, participants’ preferences for different body parts and the coefficients of transfer functions affected the choice of transfer functions. These results could serve as the guidelines for virtual locomotion experience design that involves combinations of multiple factors and can help achieve a plausible walking experience in VR.

As Virtual Reality (VR) applications gain popularity, the need for a secure, usable, and memorable user authentication method becomes crucial. However, security and privacy in such VR applications are often ignored. Current methods are insufficient in preventing man-in-the-room (MITR) attacks, which allow attackers to observe user interactions in VR while remaining invisible, and inputted passwords can easily be stolen. In this study, we propose a dynamic combination of multi-attribute authentication methods for VR, where various 3D objects and their attributes can be created and displayed. Users must select combinations of 3D objects and their attributes provided by our designed principles for identity authentication. We explore the impact of method parameters on security and provide three specific parameter schemes to deploy the practical authentication system. We designed three user studies to evaluate the usability, security, and memorability of our authentication system. The results show that the proposed scheme can effectively resist both shoulder surfing and MITR attacks with unsuccessful attack rates of 100% and 95.83%, respectively. Furthermore, this research provides suggestions to secure VR applications while maintaining usability and enhancing the memorability of the authentication method.

Spatial search tasks are common and crucial in many Virtual Reality (VR) applications. Traditional methods to enhance the performance of spatial search often employ sensory cues such as visual, auditory, or haptic feedback. However, the design and use of bimanual haptic feedback with two VR controllers for spatial search in VR remains largely unexplored. In this work, we explored bimanual haptic feedback with various combinations of haptic properties, where four types of bimanual haptic feedback were designed, for spatial search tasks in VR. Two experiments were designed to evaluate the effectiveness of bimanual haptic feedback on spatial direction guidance and search in VR. The results from the first experiment reveal that our proposed bimanual haptic schemes significantly enhanced the recognition of spatial directions in terms of accuracy and speed compared to spatial audio feedback. The second experiment's findings suggest that the performance of bimanual haptic feedback was comparable to or even better than the visual arrow, especially in reducing the angle of head movement and enhancing searching targets behind the participants, which was supported by subjective feedback as well. Based on these findings, we have derived a set of design recommendations for spatial search using bimanual haptic feedback in VR.

Target selection is one of essential operation made available by interaction techniques in virtual reality (VR) environments. However, effectively positioning or selecting occluded objects is under-investigated in VR, especially in the context of high-density or a high-dimensional data visualization with VR. In this paper, we propose ClockRay, an occluded-object selection technique that can maximize the intrinsic human wrist rotation skills through the integration of emerging ray selection techniques in VR environments. We describe the design space of the ClockRay technique and then evaluate its performance in a series of user studies. Drawing on the experimental results, we discuss the benefits of ClockRay compared to two popular ray selection techniques - RayCursor and RayCasting. Our findings can inform the design of VR-based interactive visualization systems for high-density data.

This paper presents a Shape-Adaptive Ternary-Gaussian model for describing endpoint uncertainty when pointing at moving targets of arbitrary shapes. The basic idea of the model is to combine the uncertainty related to the target shape with the uncertainty caused by the target motion. First, we proposed a model to predict endpoint distribution on static targets based on a Dual-Space Decomposition (DUDE) algorithm. Then, we linearly combined a 2D Ternary-Gaussian model with the newly proposed DUDE-based model to make the 2D Ternary-Gaussian model adaptable to moving targets with random shapes. To verify the performance of our model, we compared it with the original 2D Ternary-Gaussian model and a recent proposed Inscribed Circle model in predicting endpoint distribution. The results show that the proposed model outperformed the two baseline models while maintaining good robustness across different shapes and moving speeds.

We propose to exploit the scanpath prediction technology to simulate human visual system to automatically generate gaze scanpaths for VR/AR applications, to alleviate the equipment and computational cost in foveated rendering. Specifically, we propose a novel deep learning-based scanpath prediction model called Visual ScanPath Transformer (VSPT), to predict human gaze scanpaths in both free viewing and task-driven viewing situations, based on which the VR/AR systems can execute foveated rendering rapidly and cheaply. The proposed VSPT first extracts highly task-related image features from the visual scene, and then explores the global dependency relationships among all the image regions to generate each image region a global feature. Next, VSPT simulates the human visual working memory to consider all the previous fixations’ influences when predicting each fixation. Experimental findings confirm that our model exhibits adherence to classical visual principles during saccadic decision-making, surpassing the current state-of-the-art performance in free-viewing and task-driven (goal-driven and question-driven) visual scenarios.

With the rapid development of natural human-computer interaction technologies, gesture-based interfaces have become popular. Although gesture interaction has received extensive attention from both academia and industry, most existing studies focus on hand gesture input, leaving foot-gesture-based interfaces underexplored, especially in scenarios where the user’s hands are occupied for other interaction tasks such as washing the hair in smart shower rooms. In such scenarios, users often have to perform interactive tasks (e.g., controlling water volume) with their eyes closed when water and shampoo liquid flow along with their head to eyes area. One possible way to address this problem is to use eyes-free (rather than eyes-engaged), foot-gesture-based interactive techniques that allow users to interact with the smart shower system without visual involvement. Through our online survey, 71.60% of the participants (58/81) have the requirements of using foot-gesture-based eyes-free interactions during showers. To this end, we conducted a three-phase study to explore foot-gesture-based interaction to achieve eyes-free interaction in smart shower rooms. We first derived a set of user-defined foot gestures for eyes-free interaction in smart shower rooms. Then, we proposed a taxonomy for foot gesture interaction. Our findings indicated that end-users preferred single-foot (76.1%), atomic (73.3%), deictic (65.0%), and dynamic (76.1%) foot gestures, which markedly differs from the results reported by previous studies on user-defined hand gestures. In addition, most of the user-defined dynamic foot gestures involve atomic movements perpendicular to the ground (40.1%) or parallel to the ground (27.7%). We finally distilled a set of concrete guidelines for foot gesture interfaces based on observing end-users’ mental model and behaviors when interacting with foot gestures. Our research can inform the design and development of foot-gesture-based interaction techniques for applications such as smart homes, intelligent vehicles, VR games, and accessibility design.

It is well documented that exergames are enjoyable to play and can significantly improve older adults’ health and well-being. However, there is limited research on exploring factors affecting these users’ acceptance of such games, especially in virtual reality (VR), a relatively newer technology. This study proposes an extended version of the Technology Acceptance Model (TAM). We use variables from TAM related to older Chinese adults and specific to VR exergames to explore and confirm critical factors that could influence these users’ acceptance of such games in VR. We tested the proposed model with 51 older Chinese adults (aged 65 and above) after playing three commercial VR exergames (Beat Saber, FitXR, Dance Central). Results show that these older adults who are younger and retired and have a higher education, better financial means, and a good health condition have a more positive view of VR exergames. In addition, Perceived Usefulness, Perceived Ease of Use, and Perceived Enjoyment positively affect the intention to play VR exergames. Self-Satisfaction has a positive impact on Perceived Ease of Use and Perceived Usefulness. However, unlike previous studies, our results suggest that Facilitating Conditions have a negative effect on Perceived Ease of Use. Finally, we discuss the theoretical and practical implications of our results.

Body-centric locomotion allows users to control both movement speed and direction with body parts (e.g., head tilt, arm swing or torso lean) to navigate in virtual reality (VR). However, there is little research to systematically investigate the effects of body parts for speed and direction control on virtual locomotion by taking in account different transfer functions(L: linear function, P: power function, and CL: piecewise function with constant and linear function). Therefore, we conducted an experiment to evaluate the combinational effects of the three factors (body parts for direction control, body parts for speed control, and transfer functions) on virtual locomotion. Results showed that (1) the head outperformed the torso for movement direction control in task completion time and environmental collisions; (2) Arm-based speed control led to shorter traveled distances than both head and knee. Head-based speed control had fewer environmental collisions than knee; (3) Body-centric locomotion with CL function was faster but less accurate than both L and P functions. Task time significantly decreased from P, L to CL functions, while traveled distance and overshoot significantly increased from P, L to CL functions. L function was rated with the highest score of USE-S, -pragmatic and -hedonic; (4) Transfer function had a significant main effect on motion sickness: the participants felt more headache and nausea when performing locomotion with CL function. Our results provide implications for body-centric locomotion design in VR applications.

With more and more mobile devices (such as smart phones and tablets) supporting augmented reality (AR), using these devices to sketch in mid-air has become a popular application direction. However, due to the small display size of these devices, the user's field of view is limited and cannot see the context of graphics, which leads to the drawn graphics deviating from their expectations. In this article, we applied sonification technology to mid-air sketching with mobile AR device, and proposed a new method to address this problem. In our first experiment, we verified the feasibility of our method. Our experimental results showed that sonification can effectively reduce the deviation caused by a narrow field of view. In our second experiment, we further explored the application ability of this method in a wider range of sketching. Our experimental results showed that sonification can improve the aspect ratio of the drawn graphics. In addition, the results of the NASA-TLX questionnaire showed that the participants' mental demand and effort decreased significantly, and their subjective performance increased significantly. We proposed a new method, which can effectively improve the accuracy of mid-air sketching with a mobile AR device.

Task-irrelevant distractors affect visuo-motor control for target acquisition and studying such effects has already received much attention in human-computer interaction. However, there has been little research into distractor effects on crossing-based interaction. We thus conducted an empirical study on pen-based interfaces to investigate six crossing tasks with distractor interference in comparison to two tasks without it. The six distractor-related tasks differed in movement precision constraint (directional/amplitude), target size, target distance, distractor location and target-distractor spacing. We also developed and experimentally validated six quantitative models for the six tasks. Our results show that crossing targets with distractors had longer average times and similar accuracy than that without distractors. The effects of distractors varied depending on distractor location, target-distractor spacing and movement precision constraint. When spacing is smaller than 11.27 mm, crossing tasks with distractor interference can be regarded as pointing tasks or a combination of pointing and crossing tasks, which could be better fitted with our proposed models than Fitts’ law. According to these results, we provide practical implications to crossing-based user interface design.

In virtual reality (VR) games, playability and immersion levels are important because they affect gameplay, enjoyment, and performance. However, they can be adversely affected by VR sickness (VRS) symptoms. VRS can be minimized by manipulating users' perception of the virtual environment via the head-mounted display (HMD). One extreme example is the Teleport mitigation technique, which lets users navigate discretely, skipping sections of the virtual space. Other techniques are less extreme but still rely on controlling what and how much users see via the HMD. This research examines the effect on players' performance and gameplay of these mitigation techniques in fast-paced VR games. Our focus is on two types of visual reduction techniques. This study aims to identify specifically the trade-offs these techniques have in a first-person shooter game regarding immersion, performance, and VRS. The main contributions in this paper are (1) a deeper understanding of one of the most popular techniques (Teleport) when it comes to gameplay; (2) the replication and validation of a novel VRS mitigation technique based on visual reduction; and (3) a comparison of their effect on players' performance and gameplay.

With rapid advances in virtual reality (VR) technology, the use of haptics has become important to allow users to feel the physical properties of virtual objects. Current research has focused mainly on either weight variation or changing the center of gravity, which limits the simulation potential and may affect the feeling of immersion. This research explores the design and development of a device that can simulate both weight and center of gravity using low-cost components. Through an iterative design process and continuous testing with users, we arrived at a final prototype, FluidWeight, a device that can be attached to a typical VR handheld controller. FluidWeight uses fluid, which is transported from a central storage to a receptacle attached to the controller. A final experiment shows that users enjoyed using it because it could help increase the sense of realism in VR applications.

Body-centric locomotion allows users to navigate virtual environments with body parts (e.g. head tilt, arm swing or torso lean). Transfer functions are an important determinant of the locus of such a locomotion method. However, there is little known about the effects of transfer functions on virtual locomotion with different body parts. In this work, we selected four typical transfer functions (linear function: L, power function: P, a piecewise function with constant and linear functions: CL, and a piecewise function with constant and power functions: CP) and four body parts (head, arm, torso, and knee) from existing works, and conducted an experiment to evaluate their effects on virtual locomotion under three distances (5, 10, and 15 m) in Virtual Reality (VR). Results show that (1) CP function generally led to the longest task time with a low rate of failed trials, while CL function had the shortest task time with a high rate of failed trials; (2) body parts significantly affected the rate of failed trials, but not task time and final position offset. Head and torso resulted in the lowest and highest rate of failed trials respectively; (3) body parts did not differ in User Experience Questionnaire-Short (UEQ-S), UEQ-S Pragmatic and UEQ-S Hedonic. L was rated as the highest score for UEQ-S, UEQ-S Pragmatic and UEQ-S Hedonic, but CP had the lowest score. According to the results, we provide implications of designing body-centric locomotion with different transfer functions in VR.

In this paper, we present ArmMenu, a command input approach for distant displays. ArmMenu has a circular interface like pie menus and menu selection is performed by proprioception-based lateral arm movements. We implemented ArmMenu with an off-the-shelf body tracking device (Kinect) and conducted two experiments to validate its efficacy. In the first experiment, we explored the design space of ArmMenu by varying the number of menu items, with exposed or hidden menu modes. Users can operate up to 8-item menus with high selection accuracy (>98%). ArmMenu was fast and accurate even with the hidden menu mode. The second experiment compared the performance of ArmMenu and touchless marking menus. While having similar selection accuracy, ArmMenu was faster and more preferable by users. Our studies consequently demonstrate ArmMenu's effectiveness for command input on distant displays.

Recent research advancement of wireless sensing technology has made device-free interaction in the WiFi-enabled IoT environment possible. Although gesture-based interaction with such a smart environment greatly improves usability, it also introduces many security problems, such as shoulder surfing attacks. By spoofing the gestures of legitimate users, the attacker could easily access private information or services and cause even worse consequences. A secure interaction mechanism for this environment is required to prevent attackers without compromising the usability, while the limited recognition ability and low robustness of WiFi sensing make this target extremely challenging. To this end, we propose a secure interaction mechanism called secure interaction via WiFi Signal (SiWi), which provides the ability to resist shoulder surfing attacks without compromising the usability by using just WiFi signals. SiWi innovates in a concurrent interaction/authentication framework with only three elemental gestures (push, swing, and wave) and four types of identity-related imperceptible/hidden features (time distribution, direction, angle, and distance). HMM and Fresnel model-based algorithms are used to recognize the gestures and extract hidden features robustly and efficiently. Extensive experiments in a real implemented system were conducted to investigate the effectiveness of the proposed secure interaction system. The results show that our system can achieve an average accuracy of 93% to identify legitimate users and 97% to resist the spoofer.

Abstract Smartwatches have arguably become a popular wearable device nowadays. It is important to protect privacy data stored in smartwatches from being stolen. This study proposes a novel smartwatch user authentication technique based on the arm-raising gesture, which is the process of moving the arm from one side of the body to the chest height. We conducted two experiments to verify the effectiveness of the proposed technique. In Experiment 1, we investigated the performance of identifying users with the arm-raising gesture. We selected a set of features and applied them to five basic machine learning algorithms (i.e. random forest, simple logistic, naive Bayes, multilayer perceptron and linear classifier). Results with 32 participants show that with combined features, these classifiers generally achieved high authentication accuracy with high true accept rate (TAR) ($\geq $92.1% for random forest, simple logistic and multilayer perceptron), low false accept rate (FAR) ($\leq $0.6%) and large area under the curve (AUC) of receiver operating characteristics) ($\geq $92.4%). In Experiment 2, we examined the performance of identifying the arm-raising gesture across different day-to-day gestures. Results show that the arm-raising gesture can be identified from other eight common gestures with high TAR ($\geq $99.5%), low FAR ($\leq $3.6%) and large AUC ($\geq $99%). Overall, the results indicate that our technique could be a viable alternative for smartwatch user authentication.

Modeling the endpoint uncertainty of moving target selection with crossing is essential to understand factors such as speed-accuracy trade-off and interaction efficiency in crossing-based user interfaces with dynamic contents. However, there have been few studies looking into this research topic in the HCI field. This paper presents a Quaternary-Gaussian model to quantitatively measure the endpoint uncertainty in crossing-based moving target selection. To validate this model, we conducted an experiment with discrete crossing tasks on five factors, i.e., initial distance, size, speed, orientation, and moving direction. Results showed that our model fit the data of μ and σ accurately with adjusted R2 of 0.883 and 0.920. We also demonstrated the validity of our model in predicting error rates in crossing-based moving target selection. We concluded with a set of implications for future designs.

This paper presents the first investigation into using the goal-crossing paradigm for object selection with virtual reality (VR) head-mounted displays. Two experiments were carried out to evaluate ray-casting crossing tasks with target discs in 3D space and goal lines on 2D plane respectively in comparison to ray-casting pointing tasks. Five factors, i.e. task difficulty, the direction of movement constraint (collinear vs. orthogonal), the nature of the task (discrete vs. continuous), field of view of VR devices and target depth, were considered in both experiments. Our findings are: (1) crossing generally had shorter or no longer time, and higher or similar accuracy than pointing, indicating crossing can complement or substitute pointing; (2) crossing tasks can be well modelled with Fitts' Law; (3) crossing performance depended on target depth; (4) crossing target discs in 3D space differed from crossing goal lines on 2D plane in many aspects such as time and error performance, the effects of target depth and the parameters of Fitts' models. Based on these findings, we formulate a number of design recommendations for crossing-based interaction in VR.

Crossing is a fundamental paradigm for target selection in human-computer interaction systems. This paradigm was first introduced to virtual reality (VR) interactions by Tu et al., who investigated its performance in comparison to pointing, and concluded that crossing is generally no less effective than pointing and has unique advantages. However, owing to the characteristics of VR interactions, there are still many factors to consider when applying crossing to a VR environment. Thus, this review summarizes the main techniques for object selection in VR and crossing-related studies. Then, factors that may affect crossing interactions are analyzed from the perspectives of the input space and visual space. The aim of this study is to provide a reference for future studies on target selection based on the crossing paradigm in virtual reality.

Pointing and gesturing with the dominant thumb (DT) and the non-dominant thumb (NT) are two common tasks for bimanual tablet gripping interaction. Understanding the differences between DT and NT is important to pointing and gesturing based interface design on tablets, but is overlooked by previous studies. We therefore conducted two experiments. In the first experiment, participants carried out pointing tasks with DT and NT, respectively, on a tablet. We found DT and NT input differed in pointing time (DT had average shorter time than NT) and required target sizes for fast and accurate pointing (7.7 mm and 9.4 mm in diameter for DT and NT). On the other hand, DT and NT were alike in pointing accuracy. They performed about the same for target size larger than 9.6 mm and distance shorter than 21 mm. Both DT and NT pointing can be modeled by Fitts’ law. In the second experiment, participants performed gesturing tasks with DT and NT respectively on a tablet. The collected data were analyzed using a set of gesture features. Results showed DT and NT were different in features such as articulation time, size ratio and indicative angle difference, but similar in features like aperture, axial symmetry and shape distance. The differences of time and accuracy between DT and NT depended on gesture complexity but not gesture sizes. We discuss these findings with implications for future bimanual thumb interaction design and research.

The study of Human Factors (HF) and Interaction Design (ID) plays a central role in critical systems design. HF discovers and applies information about human behaviour, abilities, limitations and other characteristics to the design of tools, machines, systems, tasks, jobs and environments, for productive, safe, comfortable and effective human use. Successful ID is inherently multidisciplinary, forward looking, and aims to sketch, synthesise and prototype the future. Although the two fields are closely related, there are critical differences in approaches. Interaction designers typically seek to shape, create and explore future solutions, whereas HF researchers seek to operationalize social, psychological and behavioural theory to optimize design, often with constraints, such as error-free interaction, generally for skilled workers. HF work has a long tradition in the workplace, often concerned with dangerous and critical activities performed by skilled operators, whereas ID is increasingly focused on the huge market of discretionary consumers, often concerned with the likes and dislikes of people.

Number entry is a mundane and error-prone task. To find errors, users often rely on visual checks to compare the differences between their instructions and the numbers they have actually input, a task that is difficult for users to do accurately. We therefore propose the use of number-based graphical representations (GRs) as a complement to conventional numeric representations (NR) to enhance visual checks, so users can examine both GRs and NRs to detect errors. We conducted two experiments to explore the issues raised. Experiment 1 examined the effects of GRs and NRs on representation difference detection (i.e. checking if two GRs or NRs are identical). The two representations had a comparative performance by time and error rate. In Experiment 2, we investigated the performance of GRs and NRs with number entry tasks. While extending the task time (increased by 38%), number entry with GRs resulted in significantly fewer errors than without GRs (decreased by 60%). Participants also had a high preference for number entry with GRs. Therefore, the proposed technique is promising for number entry error reduction, and that in safety critical applications improved safety can be achieved.

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference ; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time ; (3) features like time and shape distance , were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.

This article experimentally investigates user performances with various surface textures in steering and gesture input tasks. Results reveal that (a) low friction material makes users spend more time on each task, and (b) although low friction material benefits the smoothness of trajectory, it causes more trajectory errors, and (c) users apply less force or pressure with slippery materials during the tasks. These findings are the more significant because they demonstrate that the common glass surface of most tablet surfaces is not the best kind of surface for optimum accuracy or for user satisfaction. The results suggest that users should be free to change the surface texture of the device in order to get natural and realistic haptic feedback according to different tasks and personal preferences.

This study examined the performance of two scrolling techniques (flick and ring) for document navigation in touch-based mobile phones using three input methods (index finger, pen, and thumb), with specific consideration given to two postures: sitting and walking. The findings are as follows: (a) in both sitting and walking postures, for the three input methods, flick resulted in shorter movement time and fewer crossings than ring, suggesting flick is superior to ring for document navigation; (b) for sitting posture, regarding pen and thumb input, ring led to shorter movement time than flick for large target distances, indicating ring has a potential interaction advantage; (c) regarding sitting and walking postures, both flick and ring document scrolling in touch-based mobile phones can be modeled by the Anderson model (Andersen, 2005). Designers of future scrolling techniques should consider these differences, as well as exploit the advantages and avoid the disadvantages of ring and flick scrolling.

This study quantitatively investigated optimal finger-based entry size in touch-based mobile phones for two commonly used Chinese handwriting input styles: two-handed entry with the nondominant hand holding the device and the index finger of the dominant hand entering characters, and one-handed entry with the dominant hand holding the device and the thumb of the dominant hand being used for character entry. Results were assessed in terms of the number and length of protruding strokes, writing time, stroke writing speed, size ratio, number of writing attempts, and subjective preference. For both one-handed entry and two-handed entry, the optimal entry box size was found to be 2.5 × 2.5 cm. This size entry box is large enough for fast and accurate handwriting with high-entry-area utilization rate and few, short protruding strokes. The experimental results and methodology of this study can be employed in user interface design for handwriting in touch-based mobile phones.

This paper reports an empirical investigation in which participants produced a set of stroke gestures with varying degrees of complexity and in different target sizes using both the finger and the pen. The recorded gestures were then analyzed according to multiple measures characterizing many aspects of stroke gestures. Our findings were as follows: (1) Finger drawn gestures were quite different to pen drawn gestures in basic measures including size ratio and average speed. Finger drawn gestures tended to be larger and faster than pen drawn gestures. They also differed in shape geometry as measured by, for example, aperture of closed gestures, corner shape distance and intersecting points deviation; (2) Pen drawn gestures and finger drawn gestures were similar in several measures including articulation time, indicative angle difference, axial symmetry and proportional shape distance; (3) There were interaction effects between gesture implement (finger vs. pen) and target gesture size and gesture complexity. Our findings show that half of the features we tested were performed well enough by the finger. This finding suggests that "finger friendly" systems should exploit these features when designing finger interfaces and avoid using the other features in which the finger does not perform as well as the pen.