This work seeks to design and evaluate haptic feedback for sequential gestural inputs, where mid-air hand gestures are used to express system commands. Nine haptic patterns are first designed leveraging metaphors. To pursue efficient interaction, we examine the trade-off between pattern duration and recognition accuracy and find that durations as short as 0.3s-0.5s achieve roughly 80%-90% accuracy. We then examine the haptic design for sequential inputs, where we vary when the feedback for each gesture is provided, along with pattern duration, gesture sequence length, and age. Results show that providing haptic patterns right after detected hand gestures leads to significantly more efficient interaction compared with concatenating all haptic patterns after the gesture sequence. Moreover, the number of gestures had little impact on performance, but age is a significant predictor. Our results suggest that immediate feedback with 0.3s and 0.5s pattern duration would be recommended for younger and older users respectively.

We introduce Gripmarks, a system that enables users to opportunistically use objects they are already holding as input surfaces for mixed reality head-mounted displays (HMD). Leveraging handheld objects reduces the need for users to free up their hands or acquire a controller to interact with their HMD. Gripmarks associate a particular hand grip with the shape primitive of the physical object without the need of object recognition or instrumenting the object. From the grip pose and shape primitive we can infer the surface of the object. With an activation gesture, we can enable the object for use as input to the HMD. With five gripmarks we demonstrate a recognition rate of 94.2%; we show that our grip detection benefits from the physical constraints of holding an object. We explore two categories of input objects 1) tangible surfaces and 2) tangible tools and present two representative applications. We discuss the design and technical challenges for expanding the concept.