Augmented Reality (AR) presents new opportunities for immersive storytelling. However, this immersiveness faces two main hurdles. First, AR’s immersive quality is often confined to visual elements, such as pixels on a screen. Second, crafting immersive narratives is complex and generally beyond the reach of amateurs due to the need for advanced technical skills. We introduce Jigsaw, a system that empowers beginners to both experience and craft immersive stories, blending virtual and physical elements. Jigsaw uniquely combines mobile AR with readily available Internet-of-things (IoT) devices. We conducted a qualitative study with 20 participants to assess Jigsaw’s effectiveness in both consuming and creating immersive narratives. The results were promising: participants not only successfully created their own immersive stories but also found the playback of three such stories deeply engaging. However, sensory overload emerged as a significant challenge in these experiences. We discuss design trade-offs and considerations for future endeavors in immersive storytelling involving AR and IoT.

Walking is a cyclic pattern of alternating footstep strikes, with each pair of steps forming a stride, and a series of strides forming a gait. We conduct a systematic examination of different kinds of intentional variations from a normal gait that could be used as input actions without interrupting overall walking progress. A design space of 22 candidate Gait Gestures is generated by adapting previous standing foot input actions and identifying new actions possible in a walking context. A formative study (n=25) examines movement easiness, social acceptability, and walking compatibility with foot movement logging to calculate temporal and spatial characteristics. Using a categorization of these results, 7 gestures are selected for a wizard-of-oz prototype demonstrating an AR interface controlled by Gait Gestures for ordering food and audio playback while walking. As a technical proof-of-concept, a gait gesture recognizer is developed and tested using the formative study data.

Spatial augmented reality (SAR) can extend desktop computing out of the monitor and into our surroundings, but extending the standard style of mouse input is challenging due to real-world geometry irregularity, gaps, and occlusion. We identify two general approaches for controlling a mouse cursor in SAR: perspective-based approaches based on raycasting, such as Nacenta et. al’s Perspective Cursor, and geometry-based approaches that closely associate cursor movement with surface topology. For the latter, we introduce Everywhere Cursor, a geometry-based approach for indirect mouse cursor control for complex 3D surface geometry in SAR. A controlled experiment compares approaches. Results show the geometry-based Everywhere Cursor improves accuracy and precision by 29% to 60% on average in a tracing task, but when traversing long distances, the perspective-based Perspective Cursor and Raycasting techniques are 22% to 49% faster, albeit with 4% to 10% higher error rates.