This paper presents a new sensing and interaction environment for post-stroke and upper extremity limb rehabilitation. The device is a combination of camera-based multitouch sensing and a supporting therapeutic software application that advances the treatment, provides feedback, and records a user's progress. The image-based analysis of hand position provided by a Microsoft Surface is used as an input into a tabletop game environment. Tailored image analysis algorithms assess rehabilitative hand movements. Visual feedback is provided in a game context. Experiments were conducted in a sub-acute rehabilitation center. Preliminary user studies with a stroke-afflicted population determined essential design criteria. Hand and wrist sensing, as well as the goals of the supporting game environment, engage therapeutic flexion and extension as defined by consulted physicians. Participants valued personalization of the activity, novelty, reward and the ability to work at their own pace in an otherwise repetitive therapeutic task. A "character" - game element personifying the participant's movement - was uniquely motivating relative to the media available in the typical therapeutic routine.

SpaceTop is a concept that fuses spatial 2D and 3D interactions in a single workspace. It extends the traditional desktop interface with interaction technology and visualization techniques that enable seamless transitions between 2D and 3D manipulations. SpaceTop allows users to type, click, draw in 2D, and directly manipulate interface elements that float in the 3D space above the keyboard. It makes it possible to easily switch from one modality to another, or to simultaneously use two modalities with different hands. We introduce hardware and software configurations for co-locating these various interaction modalities in a unified workspace using depth cameras and a transparent display. We describe new interaction and visualization techniques that allow users to interact with 2D elements floating in 3D space. We present the results from a preliminary user study that indicates the benefit of such hybrid workspaces.

Touch is fundamental to our emotional well-being. Medical science is starting to understand and develop touch-based therapies for autism spectrum, mood, anxiety and borderline disorders. Based on the most promising touch therapy protocols, we are presenting the first devices that simulate touch through haptic devices to bring relief and assist clinical therapy for mental health. We present several haptic systems that enable medical professionals to facilitate the collaboration between patients and doctors and potentially pave the way for a new form of non-invasive treatment that could be adapted from use in care-giving facilities to public use. We developed these prototypes working closely with a team of mental health professionals.

We present Picture This! a new input device embedded in children's toys for video composition. It consists of a new form of interaction for children's capturing of storytelling with physical artifacts. It functions as a video and storytelling performance system in that children craft videos with and about character toys as the system analyzes their gestures and play patterns. Children's favorite props alternate between characters and cameramen in a film. As they play with the toys to act out a story, they conduct film assembly. We position our work as ubiquitous computing that supports children's tangible interaction with digital materials. During user testing, we observed children ages 4 to 10 playing with Picture This!. We assess to what extent gesture interaction with objects for video editing allows children to explore visual perspectives in storytelling. A new genre of Gesture Object Interfaces as exemplified by Picture This relies on the analysis of gestures coupled with objects to represent bits.

We introduce Jabberstamp, the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they created.

The paper presents a novel approach to collecting, editing and performing visual and sound clips in real time. The cumbersome process of capturing and editing becomes fluid in the improvisation of a story, and accessible as a way to create a final movie. It is shown how a graphical interface created for video production informs the design of a tangible environment that provides a spontaneous and collaborative approach to video creation, selection and sequencing. Iterative design process, participatory design sessions and workshop observations with 10-12 year old users from Sweden and Ireland are discussed. The limitations of interfacing video capture, editing and publication in a self-contained platform are addressed.

This sketch presents a real-time system that aims to bridge a gap between machine listening technology, and self-editing, self-synchronizing video: a movie organizes itself from "listening" to music. In our current demonstration, a series of short video clips of "Cati" dancing, originally shot at different tempi, are arbitrarily sequenced, always in sync with the analyzed beat, i.e., if the music slows down, the dance slows down accordingly. When no beat is found, e.g., the music is mellow, then Cati stops dancing and waits, apparently bored.

This sketch presents a new approach to improvising movies according to the inter-relationship between personal videos and the story of an experience. Textable Movie is a graphical interface that invites a storyteller of any age to compose and visualize movies, images and sound environments while writing a story; the system self-selects and self-edits movies in real time based on textual input from the teller. Textable Movie aims to exalt the imagination of its authors (writer, and film-maker) by immersing them in real time, in a co-constructed narration.