SkinMorph is an on-skin interface which can selectively transition between soft and rigid states to serve as a texture-tunable wearable skin output. This texture change is made possible through the material design of smart hydrophillic gels. These gels are soft in resting state, yet when activate by heat (>36°C), they generate a micro-level structural change which results in observable stiffening. These gels are encapsulated in thin silicone patterned with resistive wires through a sew-and-transfer fabrication approach. We demonstrate application examples using the texture-tunable skin overlay as wearable, interactive protection for scenarios including: a carpal tunnel splint for rehabilitation, a protective layer for joints when engaging in high impact activities, and foot pads when wearing uncomfortable shoes. Our evaluation shows that the gel is 10 times stiffer when activated, and that users find the device skin-conformable.

Jewelry and accessories have long been objects for decorating the human body; however they remain static and non-interactive. This work explores opportunities for accessory-like kinetic wearables and their association with individual style. We developed Kino, a kinetic accessory system which enables both aesthetic and functional clothing-specific design possibilities. We engaged both fashion designers and every-day users to unpack envisioned use cases and perceptions of the system. Participants viewed the devices not as gadgets but as companions due to their close proximity to the body. They envisioned a wide range of usage scenarios, highlighting the complexity of mobility in relation to personal style. We observe how mobility offers opportunities for fluid representations of self, which is unachievable though static clothing and accessories. We also outline how personalized aesthetics is important for the meaning making of novel on-body devices.

People use more and more applications and devices that quantify daily behavior such as the step count or phone usage. Purely presenting the collected data does not necessarily support users in understanding their behavior. In recent research, concepts such as learning by reflection are proposed to foster behavior change based on personal data. In this paper, we introduce user-made predictions to help users understand personal behavior patterns. Therefore, we developed an Android application that tracks users' screen-on and unlock patterns on their phone. The application asks users to predict their daily behavior based on their former usage data. In a user study with 12 participants, we showed the feasibility of leveraging user-made predictions in a quantified self approach. By trying to improve their predictions over the course of the study, participants automatically discovered new insights into personal behavior patterns.

We introduce Rovables, a miniature robot that can move freely on unmodified clothing. The robots are held in place by magnetic wheels, and can climb vertically. The robots are untethered and have an onboard battery, microcontroller, and wireless communications. They also contain a low-power localization system that uses wheel encoders and IMU, allowing Rovables to perform limited autonomous navigation on the body. In the technical evaluations, we found that Rovables can operate continuously for 45 minutes and can carry up to 1.5N. We propose an interaction space for mobile on-body devices spanning sensing, actuation, and interfaces, and develop application scenarios in that space. Our applications include on-body sensing, modular displays, tactile feedback and interactive clothing and jewelry.

We present Amphibian, a simulator to experience scuba diving virtually in a terrestrial setting. While existing diving simulators mostly focus on visual and aural displays, Amphibian simulates a wider variety of sensations experienced underwater. Users rest their torso on a motion platform to feel buoyancy. Their outstretched arms and legs are placed in a suspended harness to simulate drag as they swim. An Oculus Rift head-mounted display (HMD) and a pair of headphones delineate the visual and auditory ocean scene. Additional senses simulated in Amphibian are breath motion, temperature changes, and tactile feedback through various sensors. Twelve experienced divers compared Amphibian to real-life scuba diving. We analyzed the system factors that influenced the users' sense of being there while using our simulator. We present future UI improvements for enhancing immersion in VR diving simulators.

We present a novel system for automatically generating immersive and interactive virtual reality (VR) environments using the real world as a template. The system captures indoor scenes in 3D, detects obstacles like furniture and walls, and maps walkable areas (WA) to enable real-walking in the generated virtual environment (VE). Depth data is additionally used for recognizing and tracking objects during the VR experience. The detected objects are paired with virtual counterparts to leverage the physicality of the real world for a tactile experience. Our approach is new, in that it allows a casual user to easily create virtual reality worlds in any indoor space of arbitrary size and shape without requiring specialized equipment or training. We demonstrate our approach through a fully working system implemented on the Google Project Tango tablet device.

We present NailO, a nail-mounted gestural input surface. Using capacitive sensing on printed electrodes, the interface can distinguish on-nail finger swipe gestures with high accuracy (>92%). NailO works in real-time: we miniaturized the system to fit on the fingernail, while wirelessly transmitting the sensor data to a mobile phone or PC. NailO allows one-handed and always-available input, while being unobtrusive and discrete. Inspired by commercial nail stickers, the device blends into the user's body, is customizable, fashionable and even removable. We show example applications of using the device as a remote controller when hands are busy and using the system to increase the input space of mobile phones.

We explore creating objects for expressive communication in the workspace. As an initial step, we created MugShots, a coffee mug with a wireless OLED display that switches between public and private social interaction modes. Targeted for the workplace, MugShots is an intimate communication device in the personal office space, yet alternates to become a social catalyst to trigger conversation when brought to public areas. We present a prototype of MugShots along with a 21-person study, providing initial discussions and insight on designing objects for communication.

We present Mime, a compact, low-power 3D sensor for unencumbered free-form, single-handed gestural interaction with head-mounted displays (HMDs). Mime introduces a real-time signal processing framework that combines a novel three-pixel time-of-flight (TOF) module with a standard RGB camera. The TOF module achieves accurate 3D hand localization and tracking, and it thus enables motion-controlled gestures. The joint processing of 3D information with RGB image data enables finer, shape-based gestural interaction.

We present a tablet-based system to collaboratively track discussion topics and ideas in a seminar-style discussion classroom. Each student uses his or her own tablet to share text ideas in a synchronized, visual environment. The system is designed to promote diverse participation and increase engagement. Our findings are based on observations of twelve class sessions and interviews with participating students. Instead of simply introducing an additional text-based communication channel into the classroom, we find that the system creates a new "stage" (in the Goffman sense) on which students could perform in ways that the main spoken stage could not support. This stage coexists with spoken communication, and augments how students attend to the material and each other. We conclude that spoken participation alone poses barriers for some participants and the addition of a non-oral, text-based stage can help establish equitable and engaging discussions in the class.

Video content is being consumed in a host of new ways-viewers are no longer restricted to same-time or same-place viewing. However, the experience of watching with a group is inherently social and often desirable despite the physical distribution of group members. This paper introduces Back Talk, a system designed to create a sociable television watching experience. We enhance TV viewing with an auditory environment around a listener. We have explored and leveraged the richness of audio to convey presence of remote viewers. We have developed a novel framework for capturing and translating engagement of an individual into a set of audio cues that are played spatially around a listener. Such auditory enhancements can augment video content consumption in the future.

We present an indoor positioning system that measures location using disturbances of the Earth's magnetic field caused by structural steel elements in a building. The presence of these large steel members warps the geomagnetic field in a way that is spatially varying but temporally stable. To localize, we measure the magnetic field using an array of e-compasses and compare the measurement with a previously obtained magnetic map. We demonstrate accuracy within 1 meter 88% of the time in experiments in two buildings and across multiple floors within the buildings. We discuss several constraint techniques that can maintain accuracy as the sample space increases.

In this paper, we propose a novel concept for a social TV application targeting the demographic of viewers enjoying live sports events, such as road bicycle racing. We intend to enhance the viewing experiences of spectators with sensor-fitted bikes tied to an interactive biking environment on television. The system enables a new form of personalized, physical, and virtual-reality interaction between viewers and a TV program, as well as interactions within or between communities of friends. We also describe a prototype we have implemented to demonstrate the feasibility of our idea. The prototype, my second bike, uses a 3D mirrored world environment (Google Earth) to visually represent participating spectators, competing athletes and outdoor bikers. We contend that the system has the potential to attract and support a large user base on account of its scalability, ease of deployment and ability to promote audience participation in live sports events on TV.

Going My Way is a mobile user-aware route planner. The system collects GPS data of a user's everyday locations and provides directions from an automatically selected set of landmarks that are close to the destination, informed by the user's usual travel patterns. In this paper, we present a brief description of the system, the results of a preliminary experiment in memory and recognition of landmarks, in addition to the results of a user evaluation of the system.

This paper describes a physically embodied and animated user interface to an interactive call handling agent, consisting of a small wireless animatronic device in the form of a squirrel, bunny, or parrot. A software tool creates movement primitives, composes these primitives into complex behaviors, and triggers these behaviors dynamically at state changes in the conversational agent's finite state machine. Gaze and gestural cues from the animatronics alert both the user and co-located third parties of incoming phone calls, and data suggests that such alerting is less intrusive than conventional telephones.

Despite the ready availability of digital recording technology and the continually decreasing cost of digital storage, browsing audio recordings remains a tedious task. This paper presents evidence in support of a system designed to assist with information comprehension and retrieval tasks from a large collection of recorded speech. Two techniques are employed to assist users with these tasks. First, a speech recognizer creates necessarily error-laden transcripts of the recorded speech. Second, audio playback is time-compressed using the SOLAFS technique. When used together, subjects are able to perform comprehension tasks with more speed and accuracy.

This paper discusses the software architecture of Impromptu, a mobile IP-based audio computing platform, with an associated set of network-based applications and services. Impromptu merges the communication properties and universal mobility of the telephone with the multi-tasking and open protocol world of the handheld PC. Its supporting architecture handles multiple streaming audio applications, provides speech services for consistent audio user interfaces across applications, and enables user management of these varied applications running simultaneously.

Current asynchronous voice messaging interfaces, like voicemail, fail to take advantage of our conversational skills. TalkBack restores conversational turn-taking to voicemail retrieval by dividing voice messages into smaller sections based on the most significant silent and filled pauses and pausing after each to record a response. The responses are composed into a reply, alternating with snippets of the original message for context. TalkBack is built into a digital picture frame; the recipient touches a picture of the caller to hear each segment of the message in turn. The minimal interface models synchronous interaction and facilitates asynchronous voice messaging. TalkBack can also present a voice-annotated slide show which it receives over the Internet.

Impromptu is a mobile audio device which uses wireless Internet Protocol (IP) to access novel computer-mediated voice communication channels. These channels show the richness of IP-based communication as compared to conventional mobile telephony, adding audio processing and storage in the network, and flexible, user-centered call control protocols. These channels may be synchronous, asynchronous, or eventtriggered, or even change modes as a function of other user activity. The demands of these modes plus the need to navigate with an entirely non-visual user interface are met with a number of audio-oriented user interaction techniques.

This paper addresses the problem that a listener experiences when attempting to capture information presented during a lecture, meeting, or interview. Listeners must divide their attention between the talker and their notetaking activity. We propose a new device-the Audio Notebook-for taking notes and interacting with a speech recording. The Audio Notebook is a combination of a digital audio recorder and paper notebook, all in one device. Audio recordings are structured using two techniques: user structuring based on notetaking activity, and acoustic structuring based on a talker's changes in pitch, pausing, and energy. A field study showed that the interaction techniques enabled a range of usage styles, from detailed review to high speed skimming. The study motivated the addition of phrase detection and topic suggestions to improve access to the audio recordings. Through these audio interaction techniques, the Audio Notebook defines a new approach for navigation in the audio domain.

Mobile workers need seamless access to communication and information services while on the move. However, current solutions overwhelm users with intrusive interfaces and ambiguous notifications. This article discusses the interaction techniques developed for Nomadic Radio, a wearable computing platform for managing voice and text-based messages in a nomadic environment. Nomadic Radio employs an auditory user interface, which synchronizes speech recognition, speech synthesis, nonspeech audio, and spatial presentation of digital audio, for navigating among messages as well as asynchonous notific ation of nely arrived messages. Emphasis is placed on an auditory modality as Nomadic Radio is designed to be used while performing other tasks in a user's everyday environment; a range of auditory cues provides peripheral awareness of incoming messages. Notification is adaptive and cntext sensitive; messages are presented as more or less obtrsive based on importance inferred from content filtering, whether the user is engaged in conversation and his or her own recent responses to prior messages. Auditory notifications are dynamically scaled from ambient sound through recorded voice cues up to message summaries. Iterative design and a preliminary user evaluation suggest that audio is an appropriate medium for mobile messaging, but that care must be taken to minimally intrude on the wearer's social and physical environment.

Mobile workers need seamless access to communication and information services on portable devices. However current solutions overwhelm users with intrusive and ambiguous notifications. In this paper, we describe scaleable auditory techniques and a contextual notification model for providing timely information, while minimizing interruptions. Users actions influence local adaptation in the model. These techniques are demonstrated in Nomadic Radio, an audio-only wearable computing platform.

Article Audio hallway: a virtual acoustic environment for browsing Share on Author: Chris Schmandt M.I.T. Media Laboratory M.I.T. Media LaboratoryView Profile Authors Info & Claims UIST '98: Proceedings of the 11th annual ACM symposium on User interface software and technologyNovember 1998 Pages 163–170https://doi.org/10.1145/288392.288597Online:01 November 1998Publication History 27citation674DownloadsMetricsTotal Citations27Total Downloads674Last 12 Months12Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Browsing audio data is not as easy as browsing printed documents because of the temporal nature of sound.This paper presents a browsing environment that provides a spatial interface for temporal navigation of audio data, taking advantage of human abilities of simultaneous listening and memory of spatial location.Instead of fast-forwarding or rewinding, users browse the audio data by switching their attention between moving sound sources that play multiple portions of a single audio recording.The motion of the sound sources maps temporal position within the audio recording onto spatial location, so that listeners can use their memory of spatial location to find a specific topic.This paper describes the iterative design approach toward the audio browsing system, including the development of user interface devices.

No abstract available.

Article MailCall: message presentation and navigation in a nonvisual environment Share on Authors: Matthew Marx Applied Language Technologies, Cambridge, Massachusetts and MIT Media Laboratory, E15-252, 20 Ames St., Cambridge, MA Applied Language Technologies, Cambridge, Massachusetts and MIT Media Laboratory, E15-252, 20 Ames St., Cambridge, MAView Profile , Chris Schmandt MIT Media Laboratory, E15-252, 20 Ames St., Cambridge, MA MIT Media Laboratory, E15-252, 20 Ames St., Cambridge, MAView Profile Authors Info & Claims CHI '96: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsApril 1996 Pages 165–172https://doi.org/10.1145/238386.238467Online:13 April 1996Publication History 16citation543DownloadsMetricsTotal Citations16Total Downloads543Last 12 Months10Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The NewsComm system delivers personalized news ~nd other program material as audio to mobile users through a hand-held playback device.This paper focuses on the iterative design and user testing of the hand-held interface.The interface was ftrst designed and tested in a softwareonly enviroument and then ported to a custom hardware platform.The hand-held device enables navigation through audio recordings based on structural information which is extracted from the audio using digital signal processing techniques.The interface design ~00resses the problems of designing a hand-held and primarily non-visual interface for accessing large amounts of StlUCtured audio recordings.

Workgroups that defy traditional boundmies require successful communication among people whose interests, schedules, and locations may differ and are likely to change rapidly.CLUES is a dynamic personalized message filter that facilitates effective communication by prioritizing voice and text messages using personal information found in an individual's work environment.CLUES infers message timeliness by considering calendar appointments, outgoing messages and phone calls, and by correlating these "clues" via a personal rolodex.Experience shows that CLUES can be especially useful to mobile users with high message traffic who often access their messages over the telephone.

Communication is about people, not machines. But as firms and families alike spread out geographically, we rely increasingly on telecommunications tools to keep us “connected”. The challenge of such systems is to enable conversation between individuals without computational infrastructure getting in the way. This paper compares two speech-based communication systems, Phoneshell and Chatter, in how they deal with the keys to communication: proper names. Chatter, a conversational system using speech-recognition, improves upon the hierarchical nature of the touch-tone based Phoneshell by maintaining context and enabling use of anaphora. Proper names can present particular problems for speech recognizers, so an interface algorithm for reliable name specification by spelling is offered. Since individual letter recognition is non-robust, Chatter implicitly disambiguates strings of letters based on context. We hypothesize that the right interface can make faulty speech recognition as usable as TouchTones—even more so.

This paper describes Phoneshell, a telephone based application providing remote voice access to personal desktop databases such as voice mail, email, calendar, and rolodex.Several forms of information can also be faxed on demand.Phoneshell offers its users numerous opportunities to record voice entries into its underlying databases; this new utility for stored voice as a data type, necessitates multimedia support for the traditional graphical user interfaces to these same databases.The experiences of a small Phoneshell user community are discussed, with emphasis on key features which are most important to its success.The underlying software architecture used by Phoneshell includes a toolkit for building interactive telephone-based services.

Although talking is an integral part of collaboration, there has been little computer support for acquiring and accessing the contents of conversations. Our approach has focused on ubiquitous audio , or the unobtrusive capture of speech interactions in everyday work environments. Speech recognition technology cannot yet transcribe fluent conversational speech, so the words themselves are not available for organizing the captured interactions. Instead, the structure of an interaction is derived from acoustical information inherent in the stored speech and augmented by user interaction during or after capture. This article describes applications for capturing and structuring audio from office discussions and telephone calls, and mechanisms for later retrieval of these stored interactions. An important aspect of retrieval is choosing an appropriate visual representation, and this article describes the evolution of a family of representations across a range of applications. Finally, this work is placed within the broader context of desktop audio, mobile audio applications, and social implications.

Article Free Access Share on Ubiquitous audio: capturing spontaneous collaboration Authors: Debby Hindus View Profile , Chris Schmandt View Profile Authors Info & Claims CSCW '92: Proceedings of the 1992 ACM conference on Computer-supported cooperative workDecember 1992 Pages 210–217https://doi.org/10.1145/143457.143481Published:01 December 1992Publication History 47citation735DownloadsMetricsTotal Citations47Total Downloads735Last 12 Months12Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Voice technology is just beginning to gain a foothold in the information processing world. Applications such as voice mail, credit verification, order entry and airline reservation systems are slowly being introduced. Critics of voice systems frequently point out their limitations with little understanding of their power or advantages. One key determinant of the success or failure of voice systems is the USER INTERFACE. It is important that the dialogue structure, prompts, system feedback and error messages be designed based on user input, testing and evaluation.Another key determinant of the success of voice systems is the careful matching of users, tasks and environment to the technology. Voice technology is often broken down into 3 major categories.Speech compression. Processing allows analog patterns of human speech to be digitized. Once stored in digital form, the information can be transmitted and played back.Common Applications: Voice Mail, Voice Annotation.Text-to-Speech. Processing allows computer-stored text to be translated and retrieved via a “synthesized” voice.Common applications: Remote retrieval of information (E-Mail, Calendar Appointments), Aids to Visually or Vocally Handicapped.Speech Recognition. Processing allows analog patterns of human speech to be translated into their text-based equivalent or into computer commands.Common Applications: Data Entry, Voice Activated Typewriter.

An interactive stereoscopic computer graphic workspace is described. A conventional frame store is used for three-dimensional display, with left/right eye views interlaced in video and viewed through PLZT shutter glasses. The video monitor is seen reflected from a half silvered mirror which projects the graphics into a workspace, into which one can reach and manipulate the image directly with a “magic wand”. The wand uses a magnetic six degree-of-freedom digitizer. In an alternative configuration, a graphics tablet was placed within the workspace for input intensive tasks.

“Put That There” is a voice and gesture interactive system implemented at the Architecture Machine Group at MIT. It allows a user to build and modify a graphical database on a large format video display. The goal of the research is a simple, conversational interface to sophisticated computer interaction. Natural language and gestures are used, while speech output allows the system to query the user on ambiguous input.