Recent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field's grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.

This innovative practice full paper finds that story-telling through animatronics can offer a combination of creativity and engineering in order to engage students in an inquiry-based approach to STEM learning. Animatronics is the art and science of bringing a story to life through robotic puppetry. Typically reserved for trained engineers with ample resources, animatronics can be an expressive yet inaccessible medium. We aim to allow students to write, create, and perform an animatronic story, learning valuable technical skills along the way. Our study, which took place over the course of eight weeks, investigates the use of our Paper Animatronics Kit in a Grade 2 and Grade 6 classroom, following the progression of their creative writing, animatronic building, and mentoring processes. We present results of our qualitative analysis of student and teacher interviews and group discussions.

We present ShArc, a precision, geometric measurement technique for building multi-bend/shape sensors. ShArc sensors are made from flexible strips that can be dynamically formed into complex curves in a plane. They measure local curvature by noting the relative shift between the inner and outer layers of the sensor at many points and model shape as a series of connected arcs. Unlike jointed systems where angular errors sum with each joint measured, ShArc sensors do not accumulate angular error as more measurement points are added. This allows for inexpensive, robust sensors that can accurately model curves with multiple bends. To demonstrate the efficacy of this technique, we developed a capacitive ShArc sensor and evaluated its performance. We conclude with examples of how ShArc sensors can be employed in applications like gesture input devices, user interface controllers, human motion tracking and angular measurement of free-form objects.

The PumpSpark Fountain Development Kit includes a controller, eight miniature water pumps, and various accessories to allow rapid prototyping of fluidic user interfaces. The controller provides both USB and logic-level serial interfaces, yielding fast (~100ms), high-resolution (8-bit) control of water streams up to about 1 meter high. Numerous example applications built using the PumpSpark kit are presented. The kit has been the subject of a student contest with over 100 students, demonstrating its utility in rapid prototyping of fluidic systems.

While pen computing has become increasingly more popular, device responsiveness, or latency, still plagues such interaction. Although there have been advances in digitizer technology over the last few years, commercial end-to-end latencies are unfortunately similar to those found with touchscreens, i.e., 65 - 120 milliseconds. We report on a prototype stylus-enabled device, the High Performance Stylus System (HPSS), designed to display latencies as low as one millisecond while users ink or perform dragging tasks.

This paper presents a new sensing and interaction environment for post-stroke and upper extremity limb rehabilitation. The device is a combination of camera-based multitouch sensing and a supporting therapeutic software application that advances the treatment, provides feedback, and records a user's progress. The image-based analysis of hand position provided by a Microsoft Surface is used as an input into a tabletop game environment. Tailored image analysis algorithms assess rehabilitative hand movements. Visual feedback is provided in a game context. Experiments were conducted in a sub-acute rehabilitation center. Preliminary user studies with a stroke-afflicted population determined essential design criteria. Hand and wrist sensing, as well as the goals of the supporting game environment, engage therapeutic flexion and extension as defined by consulted physicians. Participants valued personalization of the activity, novelty, reward and the ability to work at their own pace in an otherwise repetitive therapeutic task. A "character" - game element personifying the participant's movement - was uniquely motivating relative to the media available in the typical therapeutic routine.

Although advances in touchscreen technology have provided us with more precise devices, touchscreens are still laden with latency issues. Common commercial devices present with latency up to 125ms. Although these levels have been shown to impact users' perception of the responsiveness of the system [16], relatively little is known about the impact of latency on the performance of tasks common to direct-touch interfaces, such as direct physical manipulation.

We present Phone as a Pixel: a scalable, synchronization-free, platform-independent system for creating large, ad-hoc displays from a collection of smaller devices. In contrast to most tiled-display systems, the only requirement for participation is for devices to have an internet connection and a web browser. Thus, most smartphones, tablets, laptops and similar devices can be used. Phone as a Pixel uses a color-transition encoding scheme to identify and locate displays. This approach has several advantages: devices can be arbitrarily arranged (i.e., not in a grid) and infrastructure consists of a single conventional camera. Further, additional devices can join at any time without re-calibration. These are desirable properties to enable collective displays in contexts like sporting events, concerts and political rallies. In this paper we describe our system, show results from proof-of-concept setups, and quantify the performance of our approach on hundreds of displays.

Software designed for direct-touch interfaces often utilize a metaphor of direct physical manipulation of pseudo "real-world" objects. However, current touch systems typically take 50-200ms to update the display in response to a physi-cal touch action. Utilizing a high performance touch de-monstrator, subjects were able to experience touch latencies ranging from current levels down to about 1ms. Our tests show that users greatly prefer lower latencies, and noticea-ble improvement continued well below 10ms. This level of performance is difficult to achieve in commercial compu-ting systems using current technologies. As an alternative, we propose a hybrid system that provides low-fidelity visu-al feedback immediately, followed by high-fidelity visuals at standard levels of latency.

Microsoft Surface is distinguished among commercial multi-touch systems by its ability to interact with tagged objects. In this work, we examine a new class of tagged objects where the tag is dynamic---it changes in response to some sensed variable. As an example, a drinking glass is described which can sense when a refill should be offered. The glass is completely passive, containing no electronic components or moving parts, and works with an unmodified Microsoft Surface.

A pressure sensitive computer keyboard is presented that independently senses the force level on every depressed key. The design leverages existing membrane technologies and is suitable for low-cost, high-volume manufacturing. A number of representative applications are discussed.

Model-driven engineering proposes to develop software systems by first creating an executable model of the system design and then transforming this model into an implementation. This paper discusses the design of an automatic code generation system that transforms such models into product implementations for highly reliable, industrial-strength systems. It provides insights, practical considerations, and lessons learned when developing code generators for applications that must conform to the constraints imposed by real-world, high-performance systems. Automatic code generation has played a large part in dramatically increasing both the quality and the reliability of software for these systems.

A number of projects within the computer graphics, computer vision, and human-computer interaction communities have recognized the value of using projected structured light patterns for the purposes of doing range finding, location dependent data delivery, projector adaptation, or object discovery and tracking. However, most of the work exploring these concepts has relied on visible structured light patterns resulting in a caustic visual experience. In this work, we present the first design and implementation of a high-resolution, scalable, general purpose invisible near-infrared projector that can be manufactured in a practical manner. This approach is compatible with simultaneous visible light projection and integrates well with future Digital Light Processing (DLP) projector designs -- the most common type of projectors today. By unifying both the visible and non-visible pattern projection into a single device, we can greatly simply the implementation and execution of interactive projection systems. Additionally, we can inherently provide location discovery and tracking capabilities that are unattainable using other approaches.

In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.

In this paper, we show how traditional physical interface components such as switches, levers, knobs and touch screens can be easily modified to identify who is activating each control. This allows us to change the function per-formed by the control, and the sensory feedback provided by the control itself, dependent upon the user. An auditing function is also available that logs each user's actions. We describe a number of example usage scenarios for our tech-nique, and present two sample implementations.

Video projectors have typically been used to display images on surfaces whose geometric relationship to the projector remains constant, such as walls or pre-calibrated surfaces. In this paper, we present a technique for projecting content onto moveable surfaces that adapts to the motion and location of the surface to simulate an active display. This is accomplished using a projector based location tracking techinque. We use light sensors embedded into the moveable surface and project low-perceptibility Gray-coded patterns to first discover the sensor locations, and then incrementally track them at interactive rates. We describe how to reduce the perceptibility of tracking patterns, achieve interactive tracking rates, use motion modeling to improve tracking performance, and respond to sensor occlusions. A group of tracked sensors can define quadrangles for simulating moveable displays while single sensors can be used as control inputs. By unifying the tracking and display technology into a single mechanism, we can substantially reduce the cost and complexity of implementing applications that combine motion tracking and projected imagery.

Toolkits and other tools have dramatically reduced the time and technical expertise needed to design and implement graphical user interfaces (GUIs) allowing high-quality, iterative, user-centered design to become a common practice. Unfortunately the generation of functioning prototypes for physical interactive devices as not had similar support -- it still requires substantial time and effort by individuals with highly specialized skills and tools. This creates a divide between a designers' ability to explore form and interactivity of product designs and the ability to iterate on the basis of high fidelity interactive experiences with a functioning prototype. To help overcome this difficulty we have developed the Calder hardware toolkit. Calder is a development environment for rapidly exploring and prototyping functional physical interactive devices. Calder provides a set of reusable small input and output components, and integration into existing interface prototyping environments. These components communicate with a computer using wired and wireless connections. Calder is a tool targeted toward product and interaction designers to aid them in their early design process. In this paper we describe the process of gaining an understanding of the needs and workflow habits of our target users to generate a collection of requirements for such a toolkit. We describe technical challenges imposed by these needs, and the specifics of design and implementation of the toolkit to meet these challenges.

Recent advances in computer video projection open up new possibilities for real-time interactive, persuasive displays. Now a display can continuously adapt to a viewer so as to maximize its effectiveness. However, by the very nature of persuasion, these displays must be both immersive and subtle. We have been working on technologies that support this application including multi-projector and implicit interaction techniques. These technologies have been used to create a series of interactive persuasive displays that are described.

In this paper we present a system for providing tactile feedback for stylus-based touch-screen displays. The Haptic Pen is a simple low-cost device that provides individualized tactile feedback for multiple simultaneous users and can operate on large touch screens as well as ordinary surfaces. A pressure-sensitive stylus is combined with a small solenoid to generate a wide range of tactile sensations. The physical sensations generated by the Haptic pen can be used to enhance our existing interaction with graphical user interfaces as well as to help make modern computing systems more accessible to those with visual or motor impairments.

Projection technology typically places several constraints on the geometric relationship between the projector and the projection surface to obtain an undistorted, properly sized image. In this paper we describe a simple, robust, fast, and low-cost method for automatic projector calibration that eliminates many of these constraints. We embed light sensors in the target surface, project Gray-coded binary patterns to discover the sensor locations, and then prewarp the image to accurately fit the physical features of the projection surface. This technique can be expanded to automatically stitch multiple projectors, calibrate onto non-planar surfaces for object decoration, and provide a method for simple geometry acquisition.

This paper describes how to instrument the physical world so that objects become self-describing, communicating their identity, geometry, and other information such as history or user annotation. The enabling technology is a wireless tag which acts as a radio frequency identity and geometry (RFIG) transponder. We show how addition of a photo-sensor to a wireless tag significantly extends its functionality to allow geometric operations - such as finding the 3D position of a tag, or detecting change in the shape of a tagged object. Tag data is presented to the user by direct projection using a handheld locale-aware mobile projector. We introduce a novel technique that we call interactive projection to allow a user to interact with projected information e.g. to navigate or update the projected information.The ideas are demonstrated using objects with active radio frequency (RF) tags. But the work was motivated by the advent of unpowered passive-RFID, a technology that promises to have significant impact in real-world applications. We discuss how our current prototypes could evolve to passive-RFID in the future.

A technique for creating a touch-sensitive input device is proposed which allows multiple, simultaneous users to interact in an intuitive fashion. Touch location information is determined independently for each user, allowing each touch on a common surface to be associated with a particular user. The surface generates location dependent, modulated electric fields which are capacitively coupled through the users to receivers installed in the work environment. We describe the design of these systems and their applications. Finally, we present results we have obtained with a small prototype device.

A system that uses an ear proximity sensor to actively manage periods of distraction during telephone conversations is described. We detect when the phone is removed from the ear, record any incoming audio, and play it back when the phone is returned to the ear. By dropping silent intervals and speeding up playback with a pitch-preserving algorithm, we quickly return to real-time without the loss of information. This real-time audio buffering technique also allows us to create a user-activated, lossless instant replay function.