We present the design of Make a Riddle and TeleStory, educational applications developed on the Siftables platform for children aged 4-7 years. Siftables are hybrid tangible-graphical user interface devices with motion and neighbor sensing, graphical display, and wireless communication. Siftables provide a unique opportunity to give children responsive feedback about the movement and arrangement of a distributed set of objects. We contrast the use case that includes an external display to their use as a standalone application platform. We outline design strategies for communicating information about the affordances of the Siftables and methods of providing dynamic feedback to encourage manipulation and to increase engagement during application use for hybrid tangible-graphical user interfaces.

The Sound of Touch is a new tool for real-time capture and sensitive physical stimulation of sound samples using digital convolution. Our hand-held wand can be used to (1) record sound, then (2) play back the recording by brushing, scraping, striking or otherwise physically manipulating the wand against physical objects. During playback, the recorded sound is continuously filtered by the acoustic interaction of the wand and the material being touched. The Sound of Touch enables a physical and continuous sculpting of sound that is typical of acoustic musical instruments and interactions with natural objects and materials, but not available in GUI-based tools or most electronic music instruments. This paper reports the design of the system and observations of thousands of users interacting with it in an exhibition format. Preliminary user feedback suggests future applications to foley, professional sound design, and musical performance.

This paper outlines Siftables, a novel platform that applies technology and methodology from wireless sensor networks to tangible user interfaces in order to yield new possibilities for human-computer interaction. Siftables are compact devices with sensing, graphical display, and wireless communication. They can be physically manipulated as a group to interact with digital information and media. We discuss the unique affordances that a sensor network user interface (SNUI) such as Siftables provides, as well as the resulting directness between the physical interface and the data being manipulated. We conclude with a description of some gestural language primitives that we are currently prototyping with Siftables.