The huge influx of mobile display devices is transforming computing into multi-device interaction, demanding a fluid mechanism for using multiple devices in synergy. In this paper, we present a novel interaction system that allows a collocated large display and a small handheld device to work together. The smartphone acts as a physical interface for near-surface interactions on a computer screen. Our system enables accurate position tracking of a smartphone placed on or over any screen by displaying a 2D color pattern that is captured using the smartphone's back-facing camera. As a result, the smartphone can directly interact with data displayed on the host computer, with precisely aligned visual feedback from both devices. The possible interactions are described and classified in a framework, which we exemplify on the basis of several implemented applications. Finally, we present a technical evaluation and describe how our system is unique compared to other existing near-surface interaction systems. The proposed technique can be implemented on existing devices without the need for additional hardware, promising immediate integration into existing systems.

Cord UIs are sensorial augmented cords that allow for simple metaphor-rich interactions to interface with their connected devices. Cords offer a large underexplored space for interactions as well as unique properties and a diverse set of metaphors that make them potentially interesting tangible interfaces. We use cords as input devices and explore different interactions like tying knots, stretching, pinching and kinking to control the flow of data and/or power. We also look at ways to use objects in combination with augmented cords to manipulate data or properties of a device. For instance, placing a clamp on a cable can obstruct the audio signal to the headphones. Using special materials such as piezo copolymer cables and stretchable cords we built five working prototypes to showcase the interactions described in this paper.

Pin-based shape displays not only give physical form to digital information, they have the inherent ability to accurately move and manipulate objects placed on top of them. In this paper we focus on such object manipulation: we present ideas and techniques that use the underlying shape change to give kinetic ability to otherwise inanimate objects. First, we describe the shape display's ability to assemble, disassemble, and reassemble structures from simple passive building blocks through stacking, scaffolding, and catapulting. A technical evaluation demonstrates the reliability of the presented techniques. Second, we introduce special kinematic blocks that are actuated and sensed through the underlying pins. These blocks translate vertical pin movements into other degrees of freedom like rotation or horizontal movement. This interplay of the shape display with objects on its surface allows us to render otherwise inaccessible forms, like overhangs, and enables richer input and output.

The Feedback Fridge is a system that makes the visualization of nutritional data tangible and thereby more understandable to the user. Using RFID (radio-frequency identification) technology and one inflatable airbag inside every compartment of a refrigerator the fridge reacts on the kind of food that is put into it. Is the food healthy or unhealthy? Low in fat or high in fat? By inflating or deflating the airbags correspondingly the user is shown how nutritious or healthy his food is. This visualization also has a certain preventive effect by pushing food out of the fridge if the airbags get too big. The Feedback Fridge can thereby help enhance people's health and lead them to a balanced and nutritious diet.