To interact with Augmented Reality (AR) content while walking, the user interfaces (UIs) need to move along with the user without distracting their field of view. This paper investigates on-hand reference frames for AR interaction while walking. First, we conduct a user study evaluating six on-hand reference frames. Results show that the Pinch Grip With Offset (PGWO), which anchors UIs to the pinch grip while floating at a distance, outperforms other on-hand reference frames regarding speed, accuracy, workload, and user preference. Next, we conduct a follow-up study to compare PGWO’s performance with head and torso reference frames, commonly used in previous studies, to see whether PGWO’s benefits hold up against well-established reference frames. Results revealed better performance and higher user preference for PGWO than both reference frames. Finally, we present design recommendations for developing future AR systems that are more efficient and user-friendly for on-the-go interaction.

Current Virtual Reality (VR) Head-Mounted Displays (HMDs) offer limited shortcuts for rapid command access, which often requires users to navigate menus through precise visual targeting at multiple depths. This process can be slow and distracting, particularly during immersive gaming or productivity tasks. While marking menus have shown effectiveness as a shortcut command access mechanism, their performance in VR has not been adequately studied. Moreover, their potential integration with 6-degree-of-freedom (6-DoF) controllers and 2-DoF joysticks in VR environments remains largely unexplored. In this paper, we introduce the Drum Menu, a bimanual shortcut command access technique derived from the idea of traditional pie menus, featuring three input methods, designed for 4-item and 8-item layouts specifically for VR controller command access. Users can select commands by rotating the joystick, drawing a stroke, or pointing in different directions. Bimanual input enables simultaneous access to two menu levels. A controlled user study reveals that drum menus are faster than the unimanual versions for the 4-item layout. Additionally, users prefer the bimanual joystick drum menu with the 4-item layout given its short task time, low error rate, and low physical movement. For the 8-item layout, stroke drum menus are found to be less error-prone for expert users compared to the other techniques.

Teleportation - changing the point of view in 3D space by specifying a position - is one of the most common locomotion solutions in VR. However, it currently lacks a mechanism to adjust the height in 3D space, and it is difficult for users to predict the exact final view after the teleportation. Users are relocated to a place without knowing what the final view will look like. As a result, they often need to perform remedial interactions to achieve their ideal position, which can be time-consuming and effort-intensive. In this paper, we present Fly the Moon to Me (Locomoontion), a novel technique that enables users to bring their destination to themselves through object manipulation. Users first create a copy of the object they want to approach as a preview by selecting it, then bring it to an ideal position and direction using existing object manipulation techniques, and then snap the original object to the preview together with the rest of the world. A controlled experiment with 18 participants via a teleportation task reveals that Locomoontion is more effective than the traditional Point&Teleport technique with grabbing the world as a remedy to adjust the final positioning.

Adult children often help their older parents maintain independence at home, and this informal caregiving can be overwhelming and challenging to balance with other aspects of life. Yet, adult children's requirements tend to be overlooked when designing technologies to foster aging in place. In this work, we use reflexive thematic analysis to study Reddit content to understand the experiences of adult caregivers and the issues that they face in supporting their parents. The pseudonymous data allowed us to explore three research questions: 1) What are the experiences of adult children as informal helpers for elderly parents? 2) How do adult children currently manage their role as informal helpers? and 3) How might digital technology facilitate communication between aging parents and adult children as informal helpers? Our findings lead to a number of design considerations for technologies that support adult children caring for parents who are aging in place, including: building common ground, establishing boundaries, addressing guilt, encouraging in-person communication, community support for adult children, and adapting to changes in role reciprocity.

Augmented Reality User Interfaces commonly stay floated in midair unless explicitly moved by users. This often results in suboptimal performance due to the absence of haptic feedback and arm fatigue. We explore the transition of AR Interfaces to leverage onbody and physical surfaces, e.g., the arm and desk, in addition to mid-air. We begin with a user study to assess the potential of these surfaces for transitioning virtual UIs. Study results indicate a strong user preference for transitioning interfaces from mid-air to physical surfaces when they are available. We further explore three UI transition mechanisms: manual, semi-automatic, and automatic, each with varying levels of automation and user control. Results from a user study reveal that semi-automatic transition is the most preferred method, as it offers a good balance between automation and user control. We conclude with design guidelines for transitioning AR UIs across mid-air, on-body, and physical surfaces.

Pointing is an elementary interaction in virtual and augmented reality environments, and, to effectively support selection, techniques must deal with the challenges of occlusion and depth specification. Most of the previous techniques require two explicit steps to handle occlusion. In this paper, we propose Conductor, an intuitive, plane-ray, intersection-based, 3D pointing technique where users leverage bimanual input to control a ray and intersecting plane. Conductor allows users to use the non-dominant hand to adjust the cursor distance on the ray while pointing with the dominant hand. We evaluate Conductor against Raycursor, a state-of-the-art VR pointing technique, and show that Conductor outperforms Raycursor for selection tasks. Given our results, we argue that bimanual selection techniques merit additional exploration to support object selection and placement within virtual environments.

While many smartphones now include a waterfall (or curved edge) display, this edge is rarely exploited for input. In this paper, we introduce EdgeMark, a thumb-operated, one-handed, gestural menu to provide rapid and subtle access to commands. In a first study, we assess the movement range of the user during uni-manual input to define a spatial range for our EdgeMark menu. In a second study, we compare three potential EdgeMark designs to a marking menu variant to assess accuracy and performance. Our findings indicate that EdgeMark menus can serve as a reliable, effective mechanism for subtle, one-hand command invocation in modern, waterfall-edged smartphones.

Personal smart devices have demonstrated a variety of efficient techniques for pointing and selecting on physical displays. However, when migrating these input techniques to augmented reality, it is both unclear what the relative performance of different techniques will be given the immersive nature of the environment, and it is unclear how viewport-based versus world-based pointing methods will impact performance. To better understand the impact of device and viewing perspectives on pointing in augmented reality, we present the results of two controlled experiments comparing pointing conditions that leverage various smartphone- and smartwatch-based external display pointing techniques and examine viewport-based versus world-based target acquisition paradigms. Our results demonstrate that viewport-based techniques offer faster selection and that both smartwatch- and smartphone-based pointing techniques represent high-performance options for performing distant target acquisition tasks in augmented reality.

We introduce a passive Radio Frequency IDentification (RFID) based system to detect finger gesture input for Human-Computer Interaction applications. The device is simple, inexpensive and does not require calibration to accommodate changes in the device location or the Radio Frequency (RF) environment. This is achieved by connecting the chips of two RFID tags together using a strip transmission line. The key observation is that touching different positions along the transmission line changes the impedance matching between each chip and its antenna, changing Received Signal Strength (RSS) values for each tag. When a finger slides in different directions between key positions along the transmission line, there are relative RSS patterns and trends that are robust to changes in the device location and the RF environment. We implemented and evaluated an detection algorithm and system using a commercial RFID reader and two commercial RFID chips. Results show that precision and recall are greater than 95% and 94% when detecting 10 finger gesture inputs across 48 different device locations.

In this paper, we explore an alternative form of volunteer work, PledgeWork, where individuals, rather than working directly for a charity, make indirect donations by completing tasks provided by a third party task provider. PledgeWork poses novel research questions on issues of user acceptance of on-line volunteerism, on quality and quantity of work performed as a volunteer, and on the benefits low-barrier volunteerism might provide to charities. To evaluate these questions, we conduct a mixed methods study that compares the quality and quantity of work between volunteer workers and paid workers and user attitudes toward PledgeWork, including perceived benefits and drawbacks. We find that PledgeWork can improve the quality of simple tasks and that the vast majority of our participants expressed interest in using our PledgeWork platform to contribute to a charity. Our interview also reveals current problems with volunteering and online donations, thus highlighting additional strengths of PledgeWork.

We describe Tip-Tap, a wearable input technique that can be implemented without batteries using a custom RFID tag. It recognizes 2-dimensional discrete touch events by sensing the intersection between two arrays of contact points: one array along the index fingertip and the other along the thumb tip. A formative study identifies locations on the index finger that are reachable by different parts of the thumb tip, and the results determine the pattern of contacts points used for the technique. Using a reconfigurable 3x3 evaluation device, a second study shows eyes-free accuracy is 86% after a very short period, and adding bumpy or magnetic passive haptic feedback to contacts is not necessary. Finally, two battery-free prototypes using a new RFID tag design demonstrates how Tip-Tap can be implemented in a glove or tattoo form factor.

In gesture recognition, one challenge that researchers and developers face is the need for recognition strategies that mediate between false positives and false negatives. In this article, we examine bi-level thresholding, a recognition strategy that uses two thresholds: a tighter threshold limits false positives and recognition errors, and a looser threshold prevents repeated errors (false negatives) by analyzing movements in sequence. We first describe early observations that led to the development of the bi-level thresholding algorithm. Next, using a Wizard-of-Oz recognizer, we hold recognition rates constant and adjust for fixed versus bi-level thresholding; we show that systems using bi-level thresholding result in significantly lower workload scores on the NASA-TLX and significantly lower accelerometer variance when performing gesture input. Finally, we examine the effect that bi-level thresholding has on a real-world dataset of wrist and finger gestures, showing an ability to significantly improve measures of precision and recall. Overall, these results argue for the viability of bi-level thresholding as an effective technique for balancing between false positives, recognition errors, and false negatives.

Free-space gestural systems are faced with two major issues: a lack of subtlety due to explicit mid-air arm movements, and the highly effortful nature of such interactions. With an ever-growing ubiquity of interactive devices, displays, and appliances with non-standard interfaces, lower-effort and more socially acceptable interaction paradigms are essential. To address these issues, we explore at-one's-side gestural input. Within this space, we present the results of two studies that investigate the use of side-gesture input for interaction. First, we investigate end-user preference through a gesture elicitation study, present a gesture set, and validate the need for dynamic, diverse, and variable-length gestures. We then explore the feasibility of designing such a gesture recognition system, dubbed WatchTrace, which supports alphanumeric gestures of up to length three with an average accuracy of up to 82%, providing a rich, dynamic, and feasible gestural vocabulary.

Bi-level thresholding is a motion gesture recognition technique that mediates between false positives, and false negatives by using two threshold levels: a tighter threshold that limits false positives and recognition errors, and a looser threshold that prevents repeated errors (false negatives) by analyzing movements in sequence. In this paper, we examine the effects of bi-level thresholding on the workload and acceptance of end-users. Using a wizard-of-Oz recognizer, we hold recognition rates constant and adjust for fixed versus bi-level thresholding. Given identical recognition rates, we show that systems using bi-level thresholding result in significant lower workload scores on the NASA-TLX and accelerometer variance. Overall, these results argue for the viability of bi-level thresholding as an effective technique for balancing between false positives, recognition errors and false negatives.

We describe the design and evaluation of a freehand, smartwatch-based, mid-air pointing and clicking interaction technique, called Watchpoint. Watchpoint enables a user to point at a target on a nearby large display by moving their arm. It also enables target selection through a wrist rotation gesture. We validate the use of Watchpoint by comparing its performance with two existing techniques: Myopoint, which uses a specialized forearm mounted motion sensor, and a camera-based (Vicon) motion capture system. We show that Watchpoint is statistically comparable in speed and error rate to both systems and, in fact, outperforms in terms of error rate for small (high Fitts's ID) targets. Our work demonstrates that a commodity smartwatch can serve as an effective pointing device in ubiquitous display environments.

One challenge with modern smartwatches is text input. In this paper we explore the use of gestural interaction with a smartwatch to support text input. The inertial measurement unit of a smartwatch is used to capture gestural interaction by a user, and an external display is used to provide feedback. We examine two specific variants of gesture keyboards: the swype keyboard common on modern smartphones and the cirrin keyboard, a gestural keyboard that supports character input via directional gestures. We show, first, that freearm gestural input as sensed by a smartwatch exhibits similar efficiency as freearm gestural input sensed by motion capture systems. As well, we show that the smartwatch inertial measurement unit can support text input on ubiquitous computing displays.