Many shape-changing interfaces use an array of actuated rods to create a display surface; each rod working as a pixel. However, this approach only supports pixel height manipulation and cannot produce more radical shape changes of each pixel (and thus of the display). Examples of such changes include non-horizontal pixels, pixels that overhang other pixels, or variable gaps between pixels. We present a concept for composing shape-changing interfaces by vertically stacking tilt-enabled modules. Together, stacking and tilting allow us to create a more diverse range of display surfaces than using arrays. We demonstrate this concept through TiltStacks, a shape-changing prototype built using stacked linear actuators and displays. Each tiltable module provides three degrees of freedom (z-movement, roll, and pitch); two more degrees of freedom are added through stacking modules (i.e., planar x- and y-movement).

This paper contributes a novel sensing approach to support on- and above-skin finger input for interaction on the move. WatchSense uses a depth sensor embedded in a wearable device to expand the input space to neighboring areas of skin and the space above it. Our approach addresses challenging camera-based tracking conditions, such as oblique viewing angles and occlusions. It can accurately detect fingertips, their locations, and whether they are touching the skin or hovering above it. It extends previous work that supported either mid-air or multitouch input by simultaneously supporting both. We demonstrate feasibility with a compact, wearable prototype attached to a user's forearm (simulating an integrated depth sensor). Our prototype---which runs in real-time on consumer mobile devices---enables a 3D input space on the back of the hand. We evaluated the accuracy and robustness of the approach in a user study. We also show how WatchSense increases the expressiveness of input by interweaving mid-air and multitouch for several interactive applications.

The size of information spaces often exceeds the limits of even the largest displays. This makes navigating such spaces through on-screen interactions demanding. However, if users imagine the information space extending in a plane beyond the display's boundaries, they might be able to use the space beyond the display for input. This paper investigates Off-Limits, an interaction concept extending the input space of a large display into the space beyond the screen through the use of mid-air pointing. We develop and evaluate the concept through three empirical studies in one-dimensional space: First, we explore benefits and limitations of off-screen pointing compared to touch interaction and mid-air on-screen pointing; next, we assess users' accuracy in off-screen pointing to model the distance-to-screen vs. accuracy trade-off; and finally, we show how Off-Limits is further improved by applying that model to the naïve approach. Overall, we found that the final Off-Limits concept provides significant performance benefits over on-screen and touch pointing conditions.

Word-gesture keyboards enable fast text entry by letting users draw the shape of a word on the input surface. Such keyboards have been used extensively for touch devices, but not in mid-air, even though their fluent gestural input seems well suited for this modality. We present Vulture, a word-gesture keyboard for mid-air operation. Vulture adapts touch based word-gesture algorithms to work in mid-air, projects users' movement onto the display, and uses pinch as a word delimiter. A first 10-session study suggests text-entry rates of 20.6 Words Per Minute (WPM) and finds hand-movement speed to be the primary predictor of WPM. A second study shows that with training on a few phrases, participants do 28.1 WPM, 59% of the text-entry rate of direct touch input. Participants' recall of trained gestures in mid-air was low, suggesting that visual feedback is important but also limits performance. Based on data from the studies, we discuss improvements to Vulture and some alternative designs for mid-air text entry.