This paper addresses the problem that a listener experiences when attempting to capture information presented during a lecture, meeting, or interview. Listeners must divide their attention between the talker and their notetaking activity. We propose a new device-the Audio Notebook-for taking notes and interacting with a speech recording. The Audio Notebook is a combination of a digital audio recorder and paper notebook, all in one device. Audio recordings are structured using two techniques: user structuring based on notetaking activity, and acoustic structuring based on a talker's changes in pitch, pausing, and energy. A field study showed that the interaction techniques enabled a range of usage styles, from detailed review to high speed skimming. The study motivated the addition of phrase detection and topic suggestions to improve access to the audio recordings. Through these audio interaction techniques, the Audio Notebook defines a new approach for navigation in the audio domain.

Listening to a speech recording is much more difficult than visually scanning a document because of the transient and temporal nature of audio. Audio recordings capture the richness of speech, yet it is difficult to directly browse the stored information. This article describes techniques for structuring, filtering, and presenting recorded speech, allowing a user to navigate and interactively find information in the audio domain. This article describes the SpeechSkimmer system for interactively skimming speech recordings. SpeechSkimmer uses speech-processing techniques to allow a user to hear recorded sounds quickly, and at several levels of detail. User interaction, through a manual input device, provides continuous real-time control of the speed and detail level of the audio presentation. SpeechSkimmer reduces the time needed to listen by incorporating time-compressed speech, pause shortening, automatic emphasis detection, and nonspeech audio feedback. This article also presents a multilevel structural approach to auditory skimming and user interface techniques for interacting with recorded speech. An observational usability test of SpeechSkimmer is discussed, as well as a redesign and reimplementation of the user interface based on the results of this usability test.

No abstract available.

This panel addresses issues in designing audio-based user interactions for small, personal computing devices, or PDAs. One issue is the nature of interacting with an auditory PDA and the interplay of affordances and form factors. Another issue is how both new and traditional metaphors and interaction concepts might be applied to auditory PDAs. The utility and design of nonspeech cues are discussed, as are the aesthetic issues of persona and narrative in designing sounds. Also discussed are commercially available sound and speech components and related hardware tradeoffs. Finally, the social implications of auditory interactions are explored, including privacy, fashion and novel social interactions.

This paper describes a technique to automatically locate emphasized segments of a speech recording based on pitch. These salient portions can be used in a variety of applications, but were originally designed to be used in an interactive system that enables high-speed skimming and browsing of speech recordings. Previous techniques to detect emphasis have used Hidden Markov Models; emphasized regions in close temporal proximity were found to successfully create useful summaries of the recordings. The new research described herein presents a simpler technique to detect salient segments and summarize a recording without using statistical models that require large amounts of training data. The algorithm adapts to the pitch range of a speaker, then automatically selects the regions of highest pitch activity as a measure of emphasis. INTRODUCTION Pitch ("fundamental frequency" or "F0") provides information in speech that is important for comprehension and understanding, and can also be exp...

Skimming or browsing audio recordings is much more difficult than visually scanning a document because of the temporal nature of audio. By exploiting properties of spontaneous speech it is possible to automatically select and present salient audio segments in a time-efficient manner. Techniques for segmenting recordings and a prototype user interface for skimming speech are described. The system developed incorporates time-compressed speech and pause removal to reduce the time needed to listen to speech recordings. This paper presents a multi-level approach to auditory skimming, along with user interface techniques for interacting with the audio and providing feedback. Several time compression algorithms ami an adaptive speech detection technique are also stuntnarized.

Distributed client/server models are becoming increasingly prevalent in multimedia systems and advanced user interface design. A multimedia application, for example, may play and record audio, use speech recognition input, and use a window system for graphical I/O. The software architecture of such a system can be simplified if the application communicates to multiple servers (e.g., audio servers, recognition servers) that each manage different types of input and output. This paper describes tools for rapidly prototyping distributed asynchronous servers and applications, with an emphasis on supporting highly interactive user interfaces, temporal media, and multi-modal I/O.