Objects are indispensable tools in our daily lives. Recent research has demonstrated their potential to act as conduits for digital interactions with microgestures, however, the primary focus was on situations where the hand firmly grasps an object. We introduce GraspUI, an exploratory design space of object-centric gestures within the seven distinct phases of the grasping process, spanning pre-, during, and post-grasp movements. We conducted ideation sessions with mixed-reality designers from industry and academia to explore gesture integration throughout the entire grasping process. The outcome was 38 storyboards envisioning practical applications. To evaluate the design space’s utility, we performed a video-based assessment with end-users. We then implemented an interactive prototype and quantified the overhead cost of performing proposed gestures through a secondary study. Participants reacted positively to gestures and could integrate them into existing usage of objects. To conclude, we highlight technical and usability guidelines for implementing and extending GraspUI systems.

Although mid-air hand gestures have been widely adopted by VR/AR products (e.g., Quest 2 and HoloLens), some drawbacks remain due to their lack of tangibility and tactile feedback. Opportunistic Tangible User Interfaces could address these shortcomings by repurposing existing objects in one's physical environment. However, there has yet to be a systematic investigation of the gestures that would be desirable when using opportunistic objects or how such gestures would be impacted by such objects. In this work, we conducted an elicitation study to investigate the desirability of object and gesture combinations across a variety of interactions. The results contribute (1) an opportunistic tangible UI gesture set for spatial interfaces, and (2) an Affordance-Based Object Selector Scheme that identifies ideal objects for tangible input given a desired input gesture, based on that object's physical affordances. Arising from these findings is the vision of the Adaptive Tangible User Interface, which supports the on-the-fly composition of tangible interfaces based on the affordances found in the physical environment and a user's input task.

The craft of improvisational quilting involves working without the use of a predefined pattern. Design decisions are made “in the fabric,” with design experimentation tightly interleaved with the creation of the final artifact. To investigate how this type of design process can be supported, and to address challenges faced by practitioners, this paper presents PatchProv, a system for supporting improvisational quilt design. Based on a review of popular books on improvisational quilting, a set of design principles and key challenges to improvisational quilt design were identified, and PatchProv was developed to support the unique aspects of this process. An evaluation with a small group of quilters showed enthusiasm for the approach and revealed further possibilities for how computational tools can support improvisational quilting and improvisational design practices more broadly.