Effective use of multimedia collections requires efficient and intuitive methods of searching and browsing. This work considers databases which store music and explores how these may best be searched by providing input queries in some musical form. For the average person, humming several notes of the desired melody is the most straightforward method for providing this input, but such input is very likely to contain several errors. Previously proposed implementations of so-called query-by-humming systems are effective only when the number of input errors is small. We conducted experiments which revealed that the expected error rate for user queries is much higher than existing algorithms can tolerate. We then developed algorithms based on approximate matching techniques which deliver much improved results when comparing error-filled vocal user queries against a music collection.

Presents a new method for auralization of the vorticity of a streamline in a vector field. This technique involves using a composite tone formed by superimposing sine waves of various amplitudes whose frequency and amplitude vary in such a way as to give the perception that the resulting sound increases or decreases endlessly in pitch without ever extending beyond the listener's range of audible frequencies. Continuous clockwise or counterclockwise rotations of a streamline resulting from vorticity can then be displayed aurally as an apparently continuous increase or decrease in pitch.

Fundamental problems will confront those who wish to take full advantage of the power of tomorrow's multimodal environments.We argue that our recently introduced concept of metawidget, when embedded within a high level, networked user interface server, can support the effective implementation of complex multimedia applications.We develop algorithms which enable a multimodal system to select the "best" combination of representations for the various "information packets" in a display at any moment.If no acceptable display can be generated from the available representations, methods are needed for creating new and useful, if not beautiful, representation(s) to resolve the impasse.A running example is provided to motivate and clarify the discussion.

The authors analyze two intertwined and fundamental issues concerning computer-to-human communication in the multimodal interface; the interplay between sound and graphics, and the role of object persistence. The observations lead to metawidgets as abstract entities capable of manifesting themselves to users as image, as sound, or as various combinations and/or sequences of the two media. The authors show examples of metawidgets in action and discuss mechanisms for choosing among alternative media for metawidget instantiation. Two experimental microworlds implemented to explore these ideas are described.>

The data type correspondence problem arises when one tries to interconnect, within a single program, modules written in different languages and therefore within the framework of different type systems. An interconnection model is proposed which hides details such as interlanguage data type mappings and low level communications, which are irrelevant or confusing to most programmers. Using this model, the authors develop an automated solution to the problem of finding, for a given type in a given programming language, the corresponding type in another programming language. They compare this approach with others which have been proposed in the literature, and discuss a prototype implementation in detail.>

The potential utility of dividing the information flowing from computer to human among several sensory modalities is investigated by means of a rigorous experiment which compares the effectiveness of auditory and visual cues in the performance of a visual search task. The results indicate that a complex auditory cue can be used to replace cues traditionally presented in the visual modality. Implications for the design of multimodal workstations are discussed.

We discuss the design and implementation of VISAGE, an object-oriented user interface framework that is part of GE's FACE Flexible Architecture Compiler Environment [1]. High-level services include a domain-independent graph editor that provides standard methods for viewing and editing hierarchical graphs, including graph layout. Editors that are customized to a specific application are built by inheritance of the domain-independent graph editor's functionality and overloading of the display, selection and editing methods. At a lower layer, presentation objects are responsible for parsing user input, displaying application output, providing mouse sensitivity and command dialogue execution. These presentation objects separate the application from its user interface, and thus promote system modularity.