To interact with Augmented Reality (AR) content while walking, the user interfaces (UIs) need to move along with the user without distracting their field of view. This paper investigates on-hand reference frames for AR interaction while walking. First, we conduct a user study evaluating six on-hand reference frames. Results show that the Pinch Grip With Offset (PGWO), which anchors UIs to the pinch grip while floating at a distance, outperforms other on-hand reference frames regarding speed, accuracy, workload, and user preference. Next, we conduct a follow-up study to compare PGWO’s performance with head and torso reference frames, commonly used in previous studies, to see whether PGWO’s benefits hold up against well-established reference frames. Results revealed better performance and higher user preference for PGWO than both reference frames. Finally, we present design recommendations for developing future AR systems that are more efficient and user-friendly for on-the-go interaction.

Researchers explore various on-body locations for anchoring user interfaces in Augmented Reality (AR), primarily to leverage onbody haptic feedback and enhance usability. However, most studies are conducted in controlled laboratory settings, limiting their applicability to real-world use. Consequently, there remains a lack of research on identifying suitable on-body AR UI placements across diverse social and mobility contexts. To address this gap, we conduct a user study investigating user preferences for anchoring UIs on six on-body locations-palm, back of the palm, inner and outer forearm, and right and left lap-across different mobility conditions (standing, walking, sitting) and social contexts (private, semiprivate, and public settings). Results show that users prefer anchoring AR interfaces on the outer forearm and palm across all conditions. However, participants noted the limited interaction space on hand surfaces. To address this, a follow-up study evaluated six UI layouts anchored to the palm and forearm, comparing body-aligned vs. vertically oriented placements and small vs. enlarged interfaces. Results show that vertically oriented, enlarged layouts yield superior performance, offering context-sensitive guidance for designing AR interfaces that balance comfort, preference, and usability.

Augmented Reality User Interfaces commonly stay floated in midair unless explicitly moved by users. This often results in suboptimal performance due to the absence of haptic feedback and arm fatigue. We explore the transition of AR Interfaces to leverage onbody and physical surfaces, e.g., the arm and desk, in addition to mid-air. We begin with a user study to assess the potential of these surfaces for transitioning virtual UIs. Study results indicate a strong user preference for transitioning interfaces from mid-air to physical surfaces when they are available. We further explore three UI transition mechanisms: manual, semi-automatic, and automatic, each with varying levels of automation and user control. Results from a user study reveal that semi-automatic transition is the most preferred method, as it offers a good balance between automation and user control. We conclude with design guidelines for transitioning AR UIs across mid-air, on-body, and physical surfaces.

Mobile devices have the potential to facilitate remote tasks through Augmented Reality (AR) solutions by integrating digital information into the real world. Although prior studies have explored Mobile Augmented Reality (MAR) for co-located collaboration, none have investigated the impact of various viewing attributes that can influence remote task performance, such as target object viewing angles, synchronization styles, or having a secondary small screen showing other users current view in the MAR environment. In this paper, we explore five techniques considering these attributes, specifically designed for two modes of remote tasks: collaborative and competitive. We conducted a user study employing various combinations of those attributes for both tasks. In both instances, results indicate users' optimal performance and preference for the technique that allows asynchronous viewing of object manipulations on the small screen. Overall, this paper contributes novel techniques for remote tasks in MAR, addressing aspects such as viewing angle and synchronization in object manipulation alongside secondary small-screen interfaces. Additionally, it presents the results of a user study evaluating the effectiveness, usability, and user preference of these techniques in remote settings and offers a set of recommendations for designing and implementing MAR solutions to enhance remote activities.

The small display size of the smartwatches makes it difficult to display large amounts of information on the device. Prior work explored leveraging a second device (e.g., Head-mounted displays) to extend the space where users can access large information space with virtual displays anchored on their wrists. Though researchers showed that having an additional virtual screen increased information bandwidth, little is known about the effect of virtual display sizes on users’ performance. In this paper, we examined the impact of display sizes on spatial memory, workload, and user experience to better understand the prospects of virtually-augmented displays for smartwatches. Results from a user study revealed that a 4.8 inches display size can be the “sweet spot” for the virtually-augmented displays to ensure improved spatial memory performance and better user experience with less workload. Finally, we provided a set of design guidelines focusing to display size, spatial memory, user experience, and workload for designing virtually augmented user interfaces for smartwatches.

User interactions with smartwatches are limited due to one-finger usage on small touch screens. However, smartwatch interaction capabilities can be extended beyond the screen into the around-device space by leveraging multi-finger interactions. In this paper, we explore suitable finger postures and gestures in mid-air and on the back of the palm to extend interaction capabilities on the unmodified commodity smartwatch while ensuring their learnability through discoverability. We conduct a design study to find a set of finger postures and gestures suitable for interacting with off-the-shelf smartwatches. An application is then designed to detect the posture-gesture set with on-device dual cameras while examining their classification accuracy. To facilitate learnability through discoverability of the gesture-posture set, we explore ‘context-aware-hints’ that shows a sub-set of gesture-posture and their associated actions based on the usage context. Results from a user study show that the guidance helps users to discover and learn them quickly and accurately.