Figure 1: An overview of the HapticHearing system pipeline, from speech input processing, haptic customization, to output wearable devices with custom PCB driver and vibrotactile actuators.

Buildings increasingly incorporate sensing and actuation techniques to automate the regulation of temperature, lighting, ventilation, and more. This trend seeks to minimize human intervention, justified by the promise of enhancing energy optimization. However, it has been widely acknowledged that loss of control over environmental conditions can lead to a diminished perception of comfort and compromised long-term user awareness and satisfaction. How can we envision building systems that can interact with building inhabitants and engage them at the “right” time and place? In this work, we address this challenge through three key contributions: 1) AirSpecs, a novel smart glasses-based system that enables holistic sensing of Indoor Environmental Quality (IEQ) and physiological data, 2) an objective method for assessing environmental awareness using a peripheral LED light and a proposed comfort awareness process, and 3) design implications for addressing the fluidity of comfort. Over five days, 30 participants across three continents used the AirSpecs device and its accompanying mobile application. Through a mixed-methods analysis of user interactions, post-experience surveys, interviews, and co-design sessions, we present findings and design scenarios that demonstrate the potential of our contributions to enhance occupant engagement and comfort in smart buildings in the future. • Novel eyewear and apps enable access-anywhere interaction and personal attachment. • Comfort awareness transits between subliminal, preconscious, and conscious states. • Focus, ambient aid, and reflection modes can be designed for three awareness states. • Users prefer to interact with a semi-automated building system.

Tapis Magique is a pressure-sensitive, knitted large-scale electronic textile carpet that generates three-dimensional sensor data based on body location, postures, and gestures and drives an immersive sonic environment in real-time. Inspired by traditional textile arts and cultures, we applied an artistic approach into technological textile design and merged new materials, sensing technologies, and digital fabrication with contemporary dance and music into one united and harmonious piece of object and performance. In this pictorial, we present and discuss our textile design rationale, fabrication strategies, hardware systems, musical mappings, as well as our collaborative effort with a professional dancer to demonstrate the interactive electronic textile carpet. Our work challenges the conventional relationship between choreography and music and unveils dancer and/or sound artist's creative possibilities of agency, intimacy, and improvisation over the choreomusical performance through a novel textile interface.

Deeply immersive experiences are intrinsically rewarding; evoking them for another is a cornerstone of success in artistic or design practice. At the same time, modern interfaces have created a state of ’partial continuous attention’, and frequent self-interruption is more common than ever. In this paper, we propose a smart-glasses based interaction to quantify self-interruption dynamics in naturalistic settings, in which a slowly changing peripheral LED is monitored as a secondary task by the user. We demonstrate that this interaction captures useful information about a user’s state of engagement in real-world conditions. These data can provide designers and artists novel, objective insight into the depth of immersive experience evoked in real-world settings.

In the open-plan workspace, which has grown to more than 70% of all offices in the United States, openness and flexibility came at the cost of personal environmental control. Recent evidence has shown a decrease in worker satisfaction due to the lack of privacy and increased noise level and distraction. In this article, we present a vision for using context-aware multimodal augmentation to improve productivity and well-being in the open-plan office. We introduce the mediated atmosphere table (MAT)—a workstation combined with a network of custom environmental control devices (lighting, audio, video, airflow, heating, and scent) to alter the user’s local environment and to improve the restorative quality of the user’s personal space in the open-plan office. A preliminary user study ($N=38$) examined the effect on stress development based on subjective measures of perception, as well as objective measures extracted from recordings of heart rate variability. Our findings show that MAT significantly ($p< 0.05$) affects occupants’ perception, as well as their physiological response in an open-plan research workspace. Furthermore, we found a significant difference between experimental conditions with and without scent. We provide an exploratory look at the effect of scent and the applications of MAT.

A growing need for on-device machine learning has led to an increased interest in light-weight neural networks that lower model complexity while retaining performance. While a variety of general-purpose techniques exist in this context, very few approaches exploit domain-specific properties to further improve upon the capacity-performance trade-off. In this paper, extending our prior work [1], we train a network to emulate the behaviour of an audio codec and use this network to construct a loss. By approximating the psychoacoustic model underlying the codec, our approach enables light-weight neural networks to focus on perceptually relevant properties without wasting their limited capacity on imperceptible signal components. We adapt our method to two audio source separation tasks, demonstrate an improvement in performance for small-scale networks via listening tests, characterize the behaviour of the loss network in detail, and quantify the relationship between performance gain and model capacity. Our work illustrates the potential for incorporating perceptual principles into objective functions for neural networks.

We present Sonoflex, a thin-form, embroidered dynamic speaker made without using a permanent magnet. Our design consists of two flat spiral coils, stacked on top of each other, and is based on an isolated, thin (0.15 mm) enameled copper wire. Our approach allows for thin, lightweight, and textile speakers and does not require high voltage as in electrostatic speakers. We show how the speaker can be designed and fabricated and evaluate its acoustic properties as a function of manufacturing parameters (size, turn counts, turn spacing, and substrate materials). The experiment results revealed that we can produce audible sound with a broad frequency range (1.5 kHz - 20 kHz) with the embroidered speaker with a diameter of 50 mm. We conclude the paper by presenting several applications such as audible notifications and near-ultrasound communication.

The way we perceive a sound depends on many aspects- its ecological frequency, acoustic features, typicality, and most notably, its identified source. In this paper, we present the HCU400: a dataset of 402 sounds ranging from easily identifiable everyday sounds to intentionally obscured artificial ones. It aims to lower the barrier for the study of aural phenomenology as the largest available audio dataset to include an analysis of causal attribution. Each sample has been annotated with crowd-sourced descriptions, as well as familiarity, imageability, arousal, and valence ratings. We extend existing calculations of causal uncertainty, automating and generalizing them with word embeddings. Upon analysis we find that individuals will provide less polarized emotion ratings as a sound's source becomes increasingly ambiguous; individual ratings of familiarity and imageability, on the other hand, diverge as uncertainty increases despite a clear negative trend on average.

Generative audio models based on neural networks have led to considerable improvements across fields including speech enhancement, source separation, and text-to-speech synthesis. These systems are typically trained in a supervised fashion using simple element-wise l1 or l2 losses. However, because they do not capture properties of the human auditory system, such losses encourage modelling perceptually meaningless aspects of the output, wasting capacity and limiting performance. Additionally, while adversarial models have been employed to encourage outputs that are statistically indistinguishable from ground truth and have resulted in improvements in this regard, such losses do not need to explicitly model perception as their task; furthermore, training adversarial networks remains an unstable and slow process. In this work, we investigate an idea fundamentally rooted in psychoacoustics. We train a neural network to emulate an MP3 codec as a differentiable function. Feeding the output of a generative model through this MP3 function, we remove signal components that are perceptually irrelevant before computing a loss. To further stabilize gradient propagation, we employ intermediate layer outputs to define our loss, as found useful in image domain methods. Our experiments using an autoencoding task show an improvement over standard losses in listening tests, indicating the potential of psychoacoustically motivated models for audio generation.

Mutual capacitance-based multi-touch sensing is now a ubiquitous and high-fidelity input technology. However, due to the complexity of electrical and signal processing requirements, it remains very challenging to create interface prototypes with custom-designed multi-touch input surfaces. In this paper, we introduce Multi-Touch Kit, a technique enabling electronics novices to rapidly prototype customized capacitive multi-touch sensors. In contrast to existing techniques, it works with a commodity microcontroller and open-source software and does not require any specialized hardware. Evaluation results show that our approach enables multi-touch sensors with a high spatial and temporal resolution and can accurately detect multiple simultaneous touches. A set of application examples demonstrates the versatile uses of our approach for sensors of different scales, curvature, and materials.

We share a study on the public adoption the Chibitronics circuit sticker toolkit, an open source, commercially available hardware toolkit for learning and creating electronics on paper. We examine sales data over a two-and-a-half-year period from November 2013, when the kit was launched commercially, to June 2016. We also look at publicly available project documentation from users during this period. We find that the Chibitronics user community confounds norms for traditional technology-making communities, especially in gender demographics. We explore the artifacts and types of documentation produced by users to learn about the various backgrounds, values, and goals of subcommunities, which includes educators, Makers, and crafters. In particular, we focus on artifacts from the craft community as a surprising and distinctive subset of technology creators. The diversity in public engagement shows how paper electronics tools like Chibitronics can be an effective approach for engaging new and broader audiences to participate in technology creation.

Currently, the manufacturing of self-actuating and self-sensing robots requires non-standard manufacturing techniques and assembly steps to integrate electrical and mechanical systems. In this work, we developed a novel manufacturing technique, where such robots can be produced at a flexible electronics factory. We developed the technique using standard industrial machines, processes, and materials. Using a lamination process, we were able to integrate air pouches or shape memory alloy (SMA) inside a polyamide-based flexible circuit to produce bending actuators. The bend angle of the actuators is sensed with a chain of inertial measurement units integrated on the actuator. Air-pouch actuators can produce a force of a 2.24N, and a maximum bend angle of 74 degrees. To demonstrate, we manufactured a five-legged robot with the developed actuators and bend sensors, with all the supporting electronics (e.g., microcontrollers, radio) directly integrated into the flexible printed circuit. Such robots are flat and lightweight (15 grams) and thus conveniently compact for transportation and storage. We believe that our technique can allow inexpensive and fast prototyping and deployment of self-actuating and self-sensing robots.

We introduce Rovables, a miniature robot that can move freely on unmodified clothing. The robots are held in place by magnetic wheels, and can climb vertically. The robots are untethered and have an onboard battery, microcontroller, and wireless communications. They also contain a low-power localization system that uses wheel encoders and IMU, allowing Rovables to perform limited autonomous navigation on the body. In the technical evaluations, we found that Rovables can operate continuously for 45 minutes and can carry up to 1.5N. We propose an interaction space for mobile on-body devices spanning sensing, actuation, and interfaces, and develop application scenarios in that space. Our applications include on-body sensing, modular displays, tactile feedback and interactive clothing and jewelry.

This paper presents ChainFORM: a linear, modular, actuated hardware system as a novel type of shape changing interface. Using rich sensing and actuation capability, this modular hardware system allows users to construct and customize a wide range of interactive applications. Inspired by modular and serpentine robotics, our prototype comprises identical modules that connect in a chain. Modules are equipped with rich input and output capability: touch detection on multiple surfaces, angular detection, visual output, and motor actuation. Each module includes a servo motor wrapped with a flexible circuit board with an embedded microcontroller.

Chibitronics circuit stickers are a new toolkit for crafting with electronics. Circuit stickers are modular, flexible circuit components with conductive adhesive pads. They feel like stickers but behave like electronic units and can be stuck to any conductive connector, such as copper foil tape, conductive ink or conductive thread, to make circuits. The toolkit includes, LEDs, sensors, pre-programmed function generators and a programmable microcontroller. Along with the circuit stickers, we have also created a suite of educational support resources such as the Circuit Sticker Sketchbook, which contains lessons and activities for crafting circuits onto templates inside the book. In this paper we describe the design and initial public adoption of the Chibitronics circuit sticker toolkit.

We present a wearable system that uses ambient electromagnetic interference (EMI) as a signature to identify electronic devices and support proxemic interaction. We designed a low cost tool, called EMI Spy, and a software environment for rapid deployment and evaluation of ambient EMI-based interactive infrastructure. EMI Spy captures electromagnetic interference and delivers the signal to a user's mobile device or PC through either the device's wired audio input or wirelessly using Bluetooth. The wireless version can be worn on the wrist, communicating with the user;s mobile device in their pocket. Users are able to train the system in less than 1 second to uniquely identify displays in a 2-m radius around them, as well as to detect pointing at a distance and touching gestures on the displays in real-time. The combination of a low cost EMI logger and an open source machine learning tool kit allows developers to quickly prototype proxemic, touch-to-connect, and gestural interaction. We demonstrate the feasibility of mobile, EMI-based device and gesture recognition with preliminary user studies in 3 scenarios, achieving 96% classification accuracy at close range for 6 digital signage displays distributed throughout a building, and 90% accuracy in classifying pointing gestures at neighboring desktop LCD displays. We were able to distinguish 1- and 2-finger touching with perfect accuracy and show indications of a way to determine power consumption of a device via touch. Our system is particularly well-suited to temporary use in a public space, where the sensors could be distributed to support a popup interactive environment anywhere with electronic devices. By designing for low cost, mobile, flexible, and infrastructure-free deployment, we aim to enable a host of new proxemic interfaces to existing appliances and displays

We present NailO, a nail-mounted gestural input surface. Using capacitive sensing on printed electrodes, the interface can distinguish on-nail finger swipe gestures with high accuracy (>92%). NailO works in real-time: we miniaturized the system to fit on the fingernail, while wirelessly transmitting the sensor data to a mobile phone or PC. NailO allows one-handed and always-available input, while being unobtrusive and discrete. Inspired by commercial nail stickers, the device blends into the user's body, is customizable, fashionable and even removable. We show example applications of using the device as a remote controller when hands are busy and using the system to increase the input space of mobile phones.

SensorTape is a modular and dense sensor network in a form factor of a tape. SensorTape is composed of interconnected and programmable sensor nodes on a flexible electronics substrate. Each node can sense its orientation with an inertial measurement unit, allowing deformation self-sensing of the whole tape. Also, nodes sense proximity using time-of-flight infrared. We developed network architecture to automatically determine the location of each sensor node, as SensorTape is cut and rejoined. Also, we made an intuitive graphical interface to program the tape. Our user study suggested that SensorTape enables users with different skill sets to intuitively create and program large sensor network arrays. We developed diverse applications ranging from wearables to home sensing, to show low deployment effort required by the user. We showed how SensorTape could be produced at scale using current technologies and we made a 2.3-meter long prototype.

We present an augmented airbrush that allows novices to experience the art of spray painting. Inspired by the thriving field of smart tools, our handheld device uses 6DOF tracking, augmentation of the airbrush trigger, and a specialized algorithm to restrict the application of paint to a preselected reference image. Our device acts both as a physical spraying device and as an intelligent assistive tool, providing simultaneous manual and computerized control. Unlike prior art, here the virtual simulation guides the physical rendering ( inverse rendering ), allowing for a new spray painting experience with singular physical results. We present our novel hardware design, control software, and a user study that verifies our research objectives.

Wrist-worn sensors (microphones, time-of-flight cameras, etc) have gained the attention of Human Computer Interaction (HCI) and body sensor researchers for their potential ability to aid interaction with wearable devices. In this paper, we use wrist-worn sensor modalities to evaluate free-hand microgesture usability. Our goal was to determine which microgestures should be included in a potential universal microgesture language and identify any underlying microgestural usability principles. Through a brief pilot user study recording microgesture task time and user accuracy, we were able to explore trends in common usability aspects. Results of the user study showed that free-hand microgestures, even at small physical and temporal scale, have significant effects on task time and user accuracy. Further analysis through multiple comparisons identified which microgestures produce relatively more accurate and efficient interaction. Physical commonalities between such microgestures prompted theories concerning why certain microgestures produce more efficient results. Based on findings and proposed theories, we give suggestions concerning a universal microgestural language and microgestural application development.

We present a multimodal on-surface and near-surface sensing technique for planar, curved and flexible surfaces. Our technique leverages temporal multiplexing of signals coming from a universal interdigitated electrode design, which is printed as a single conductive layer on a flexible substrate. It supports sensing of touch and proximity input, and moreover is capable of capturing several levels of pressure and flexing. We leverage recent developments in conductive inkjet printing as a way to prototype electrode patterns, and combine this with our hardware module for supporting the full range of sensing methods. As the technique is low-cost and easy to implement, it is particularly well-suited for prototyping touch- and hover-based user interfaces, including curved and deformable ones.

As our world becomes more instrumented, sensors are appearing in our homes, cars, and on our bodies. These sensors are connected to a diverse set of systems and protocols driven by cost, power, bandwidth, and more. Despite this heterogeneous infrastructure, we need to be able to build applications t

In this paper we present WristFlex, an always-available on-body gestural interface. Using an array of force sensitive resistors (FSRs) worn around the wrist, the interface can distinguish subtle finger pinch gestures with high accuracy (>80 %) and speed. The system is trained to classify gestures from subtle tendon movements on the wrist. We demonstrate that WristFlex is a complete system that works wirelessly in real-time. The system is simple and light-weight in terms of power consumption and computational overhead. WristFlex's sensor power consumption is 60.7 uW, allowing the prototype to potentially last more then a week on a small lithium polymer battery. Also, WristFlex is small and non-obtrusive, and can be integrated into a wristwatch or a bracelet. We perform user studies to evaluate the accuracy, speed, and repeatability. We demonstrate that the number of gestures can be extended with orientation data from an accelerometer. We conclude by showing example applications.

We present an approach to combining digital fabrication and craft, demonstrating a hybrid interaction paradigm where human and machine work in synergy. The FreeD is a hand-held digital milling device, monitored by a computer while preserving the makers freedom to manipulate the work in many creative ways. Relying on a pre-designed 3D model, the computer gets into action only when the milling bit risks the objects integrity, preventing damage by slowing down the spindle speed, while the rest of the time it allows complete gestural freedom. We present the technology and explore several interaction methodologies for carving. In addition, we present a user study that reveals how synergetic cooperation between human and machine preserves the expressiveness of manual practice. This quality of the hybrid territory evolves into design personalization. We conclude on the creative potential of open-ended procedures within this hybrid interactive territory of manual smart tools and devices.

WristQue combines environmental and inertial sensing with precise indoor localization into a wristband wearable device that serves as the user's personal control interface to networked infrastructure. WristQue enables users to take control of devices around them by pointing to select and gesturing to control. At the same time, it uniquely identifies and locates users to deliver personalized automatic control of the user's environment. In this paper, the hardware and software components of the WristQue system are introduced, and a number of applications for lighting and HVAC control are presented, using pointing and gesturing as a new human interface to these networked systems.

In this paper, we present an approach to combining digital fabrication and craft, emphasizing the user experience. While many researchers strive to enable makers to design and produce 3D objects, our research seeks to present a new fabrication approach to make unique, one-of-a-kind artifacts. To that end, we developed the FreeD, a hand-held digital milling device. The system is guided and monitored by a computer while preserving the maker's freedom to sculpt and carve, and to manipulate the work in many creative ways. Relying on a predesigned 3D model, the computer gets into action only when the milling bit risks the object's integrity, by slowing down the spindle's speed or by drawing back the shaft, while the rest of the time it allows complete gestural freedom. We describe the key concepts of our work and its motivation, present the FreeD's architecture and technology, and discuss two projects made with the tool.

We propose cutting as a novel paradigm for ad-hoc customization of printed electronic components. As a first instantiation, we contribute a printed capacitive multi-touch sensor, which can be cut by the end-user to modify its size and shape. This very direct manipulation allows the end-user to easily make real-world objects and surfaces touch-interactive, to augment physical prototypes and to enhance paper craft. We contribute a set of technical principles for the design of printable circuitry that makes the sensor more robust against cuts, damages and removed areas. This includes novel physical topologies and printed forward error correction. A technical evaluation compares different topologies and shows that the sensor remains functional when cut to a different shape.

In this paper we explore human-computer interaction for carving, building upon our previous work with the FreeD digital sculpting device. We contribute a new tool design (FreeD V2), with a novel set of interaction techniques for the fabrication of static models: personalized tool paths, manual overriding, and physical merging of virtual models. We also present techniques for fabricating dynamic models, which may be altered directly or parametrically during fabrication. We demonstrate a semi-autonomous operation and evaluate the performance of the tool. We end by discussing synergistic cooperation between human and machine to ensure accuracy while preserving the expressiveness of manual practice.

For wearable computing to become more widely accepted, the associated Human-Computer Interface must move past today's keyboard, keypad, touch screen, or other bulky hand-held interfaces to allow a user to specify input through their fingers without taking their eyes and attention off their immediate focus. Accordingly, we have developed a wearable system to track hand gestures with passive RFID sensor tags. This system is composed of an ultra-high frequency (UHF) reader and small, passive, finger-worn tags powered by transmit RF energy, each equipped with a variety of sensors that could be used to detect gestures. The primary physical goals of the system were to be comfortable and wearable without interfering with other everyday activities while tracking particular hand movements that could be used to control a wearable computer or aid in interaction with ubiquitous or other wearable devices. This paper first introduces our hardware, then gives some example user interface implementations, such as a mouse scrolled by hand position and a click specified by finger proximity, entering input by touching fingers, setting options when moving the hand to a particular spot of the user's apparel labeled with a passive RFID tag, and otherwise mapping control onto motion of the hand, arm, and fingers. The overall system was fully functional, but as this is an early implementation, it was still very much limited by transmit power and antenna efficiency, due to the constraints on the size of the passive tags. Means of scaling to lower power and smaller size are suggested.

In this paper we describe the design and implementation of a new versatile, scalable and cost-effective sensate surface. The system is based on a new conductive inkjet technology, which allows capacitive sensor electrodes and different types of RF antennas to be cheaply printed onto a roll of flexible substrate that may be many meters long. By deploying this surface on (or under) a floor it is possible to detect the presence and whereabouts of users through both passive and active capacitive coupling schemes. We have also incorporated GSM and NFC electromagnetic radiation sensing and piezoelectric pressure and vibration detection. We report on a number of experiments which evaluate sensing performance based on a 2.5m x 0.3m hardware test-bed. We describe some potential applications for this technology and highlight a number of improvements we have in mind.

Documentaries are typically captured in a very structured way, using teams to film and interview people. We developed an autonomous method for capturing structured cinéma vérité style documentaries through an interactive robotic camera, which was used as a mobile physical agent to facilitate interaction and story gathering within a ubiquitous media framework. We sent this robot out to autonomously gather human narrative about its environment. The robot had a specific story capture goal and leveraged humans to attain that goal. The robot collected a 1st person view of stories unfolding in real life, and as it engaged with its subjects via a preset dialog, these media clips were intrinsically structured. We evaluated this agent by way of determining "complete" vs. "incomplete" interactions. "Complete" interactions were those that generated viable and interesting videos, which could be edited together into a larger narrative. It was found that 30% of the interactions captured were "complete" interactions. Our results suggested that changes in the system would only produce incrementally more "complete" interactions, as external factors like natural bias or busyness of the user come into play. The types of users who encountered the robot were fairly polar; either they wanted to interact or did not - very few partial interactions went on for more than 1 minute. Users who partially interacted with the robot were found to treat it rougher than those who completed the full interaction. It was also determined that this type of limited-interaction system is best suited for short-term encounters. At the end of the study, a short cinéma vérité documentary showcasing the people and activity in our building was easily produced from the structured videos that were captured, indicating the utility of this approach.

We present a novel approach to the creation of user-generated, documentary video using a distributed network of sensor-enabled video cameras and wearable on-body sensor devices. The wearable sensors are used to identify the subjects in view of the camera system and label the captured video with real-time human-centric social and physical behavioral information. With these labels, massive amounts of continually recorded video can be browsed, searched, and automatically stitched into cohesive multimedia content. This system enables naturally occurring human behavior to drive and control a multimedia content creation system in order to create video output that is understandable, informative, and/or enjoyable to its human audience. The collected sensor data is further utilized to enhance the created multimedia content such as by using the data to edit and/or generate audio score, determine appropriate pacing of edits, and control the length and type of audio and video transitions directly from the content of the captured media. We present the design of the platform, the design of the multimedia content creation application, and the evaluated results from several live runs of the complete system.

The ChainMail system is a scalable electronic sensate skin that is designed as a dense sensor network. ChainMail is built from small (1"x1") rigid circuit boards attached to their neighbors with flexible interconnects that allow the skin to be conformally arranged and manipulated. Each board contains an embedded processor together with a suite of thirteen sensors, providing dense, multimodal capture of proximate and contact phenomena. This system forms a sensate lining that can be applied to an object, device, or surface to enable interactivity. Under extended testing, we demonstrate a flexible skin to detect and respond to a variety of stimuli while running quickly and efficiently.

The six articles in this special issue focus on the technology and applications of body area networking.

We present iCount, a new energy meter design. For many systems that have a built-in switching regulator, adding a single wire between the regulator and the microcontroller enables real-time energy metering. iCount measures energy usage by counting the switching cycles of the regulator. We show that the relationship between load current and switching frequency is quite linear and demonstrate that this simple design can be applied to a variety of regulators. Our particular implementation exhibits a maximum error of less than plusmn20% over five decades of current draw, a resolution exceeding 1 muJ, a read latency of 15 mus, and a power overhead that ranges from 1% when the node is in standby to 0.01 % when the node is active, for a typical workload. The basic iCount design requires only a pulse frequency modulated switching regulator and a microcontroller with an externally-clocked counter.

We describe a wireless wearable system that was developed to provide quantitative gait analysis outside the confines of the traditional motion laboratory. The sensor suite includes three orthogonal accelerometers, three orthogonal gyroscopes, four force sensors, two bidirectional bend sensors, two dynamic pressure sensors, as well as electric field height sensors. The "GaitShoe" was built to be worn in any shoe, without interfering with gait and was designed to collect data unobtrusively, in any environment, and over long periods. The calibrated sensor outputs were analyzed and validated with results obtained simultaneously from the Massachusetts General Hospital, Biomotion Laboratory. The GaitShoe proved highly capable of detecting heel-strike and toe-off, as well as estimating foot orientation and position, inter alia.

In this paper, we present a wireless sensor platform designed for processing multipoint human motion with low latency and high resolution. One application considered here is interactive dance, in which a choreographer wishes to translate the movements of multiple dancers into real-time audio or video content to accompany the performance. This can only be accomplished using a distributed measurement system capable of responding quickly with enough information to describe the expressive movements of multiple people. Similar requirements exist for biomechanical analysis, especially in the context of athletic training, where high resolution is demanded, and instant feedback is also desirable. Our approach to addressing such aggressive requirements involves a high-speed wireless network of compact inertial measurement units (IMUs) that can be worn at various locations on the body. Each device is equipped with its own 1Mbps radio link and a full six-axis IMU, as well as a capacitive node-to-node proximity sensor. Currently, the system supports real-time data collection and processing for up to 25 nodes with 100Hz full state updates, thereby handling much higher data rates than its predecessors. With locally buffered data, sample rates of up to 1kHz have been achieved successfully. Early results discussed here demonstrate the feasibility of our design through testing with both dancers and professional athletes.

In this paper, we introduce the "Plug" sensor network, a ubiquitous networked sensing platform ideally suited to broad deployment in environments where people work and live. The backbone of the Plug sensor network is a set of 35 sensor-, radio-, and computation-enabled power strips distributed throughout the third oor of the MIT Media Lab. A single Plug device fulfills all the functional requirements of a normal power strip (i.e., four 120V, 60Hz electrical outlets; surge protector circuit; standard electrical connector to a US-style wall socket), and can be used without special training. Additionally, each Plug has a wide range of sensing modalities (e.g., sound, light, electrical current and voltage, vibration, motion, and temperature) for gathering data about how it is being used and its nearby environment. To our knowledge, the Plug sensor network is the first to embody the idea of designing sensor nodes to seamlessly become a part of their environment, rather than play the role of alien, if unobtrusive, observers. We argue this design principle is essential for sensor networks to succeed in the realm of ubiquitous computing.

This paper presents a framework for power-efficient detection in embedded sensor systems. State detection is structured as a decision tree classifier that dynamically orders the activation and adjusts the sampling rate of the sensors (termed groggy wakeup), such that only the data necessary to determine the system state is collected at any given time. This classifier can be tuned to trade-off accuracy and power in a structured, parameterized fashion. An embedded instantiation of these classifiers, including real-time sensor control, is described. An application based on a wearable gait monitor provides quantitative support for this framework. The decision tree classifiers achieved roughly identical detection accuracies to those obtained using support vector machines while drawing three times less power. Both simulation and real-time operation of the classifiers demonstrate that our multi-tiered classifier determines states as accurately as a single-trigger (binary) wakeup system while drawing as little as half as much power and with only a negligible increase in latency.

This paper describes CargoNet, a system of low-cost, micropower active sensor tags that seeks to bridge the current gap between wireless sensor networks and radio-frequency identification (RFID). CargoNet was aimed at applications in environmental monitoring at the crate and case level for supply-chain management and asset security. Custom-designed circuits and sensors were utilized to minimize power consumption and cost in a practical prototype. The CargoNet nodes are capable of asynchronous multimodal wakeup on exceptional events at extremely low power (Quasi-Passive Wakeup) with adjustable thresholds that adapt to dynamic environments. Accordingly, CargoNet has been seen to monitor, log, and report conditions inside a typical shipping crate while consuming under 25 microwatts of average power. To demonstrate the feasibility of the prototype system, several tests and deployments were conducted in the laboratory and aboard various transport conveyances.

This article outlines several projects aimed at generating electrical energy by passively tapping a variety of human body sources and activities. After summarizing different energy harvesting modalities and techniques, I spotlight work done in my research group at the MIT Media Laboratory, including a system that scavenges electricity from the forces exerted on a shoe during walking. This system uses a flexible piezoelectric foil stave to harness sole-bending energy and a reinforced PZT dimorph to capture heel-strike energy. The piezoelectric generators drive a battery-less, active RF tag, which transmits a short-range wireless ID while walking, thereby enabling location based services and active environments. Other systems that we have developed are also discussed, including a battery-less pushbutton that can send an RF ID code with a single push, sensor nodes that harvest mobility rather than energy, and power management schemes that exploit sensor diversity to achieve energy efficiency.

This paper describes the design and architecture of the UbER-Badge, a wireless sensor node and wearable display designed to facilitate group interaction in large meetings and acquire a wide range of data for analyzing social dynamics. The platform design and its application suite are described. Data is presented that shows the social patterns developing across large events and indicates that certain individual characteristics (interest, affiliation) can be determined from the sensor data from deployments of this system with groups of over 100 people.

We have designed and constructed a modular platform for use in compact wireless sensing. This platform is based around a series of circuit boards (or panes), each of which instantiates a specific sensing modality - e.g. inertial sensing, tactile sensing or ambient sensing. As opposed to similar architectures, this system treats the sensor panes as discrete design objects that have data collection as their primary purpose. The main goals achieved by this design are ease of prototyping (and revision) and encapsulation of design knowledge. We describe a number of different projects instantiated with this system. Key among them are a wearable gait measurement system which exploited the modularity of the system, a novel musical controller which benefited from the design encapsulation and a study of dynamically low-power sensor nodes based on the structure of the panes (i.e. multiple sensors for a single modality). Several other applications are discussed. Lessons learned in the areas of software design, sensor placement and mechanical stability are commented on.

Over the last three years we have built and experimented with the Pushpin computing wireless sensor network platform. The Pushpin platform is a tabletop multihop wireless sensor network testbed comprised of 100 nodes arbitrarily placed within a one-square-meter area. The Pushpin platform's concise form factor and extreme node density allow for fine-grained control of its environment and immediate user interaction, thereby uniquely situating it between simulated and real world sensor networks. This paper details our salient successes and lessons learned along the way. We also discuss how these experiences have shaped our vision of the future of wireless sensor networks and some concrete research directions to follow.

This paper introduces a novel interface for digitally-augmented cooperative play. We present the concept of the athletic-tangible interface, a new class of interaction which uses tangible objects and full-body motion in physical spaces with digital augmentation. We detail the implementation of PingPongPlus, a reactive ping-pong table, which features a novel sound-based ball tracking technology. The game is augmented and transformed with dynamic graphics and sound, determined by the position of impact, and the rhythm and style of play. A variety of different modes of play and initial experiences with PingPongPlus are also described.

A non-contact sensor based on the interaction of a person with electric fields for human-computer interface is investigated. Two sensing modes are explored: an external electric field shunted to ground through a human body, and an external electric field transmitted through a human body to stationary receivers. The sensors are low power (milliwatts), high resolution (millimeter) low cost (a few dollars per channel), have low latency (millisecond), high update rate (1 kHz), high immunity to noise (>72 dB), are not affected by clothing, surface texture or reflectivity, and can operate on length scales from microns to meters. Systems incorporating the sensors include a finger mouse, a room that knows the location of its occupant, and people-sensing furniture. Haptic feedback using passive materials is described. Also discussed are empirical and analytical approaches to transform sensor measurements into position information. KEYWORDS: user interface, input device, gesture interface, non-...