The natural role of sound in actions involving mechanical impact and vibration suggests the use of auditory display as an augmentation to virtual haptic interfaces. In order to budget available computational resources for sound simulation, the perceptually tolerable asynchrony between paired haptic-auditory sensations must be known. This paper describes a psychophysical study of detectable time delay between a voluntary hammer tap and its auditory consequence (a percussive sound of either 1, 50, or 200 ms duration). The results show Just Noticeable Differences (JNDs) for temporal asynchrony of 24 ms with insignificant response bias. The invariance of JND and response bias as a function of sound duration in this experiment indicates that observers cued on the initial attack of the auditory stimuli.

A recent development in advanced interface technologies is the virtual acoustic display, a system that presents threedimensional auditory information over headphones [20]. The utility of such a display depends on the accuracy with which listeners can localize the virtual, or simulated, sound sources. Synthesis of virtual sources involves the digital filtering of stimuli using filters based on acoustic HeadRelated Transfer Functions (HRTFs) measured in human ear-canals. In practise, measurement of the HRTFs of each potential user of a 3-D display may not be feasible. Thus, a critical research question is whether listeners from the general population can obtain adequate localization cues from stimuli based on non-individualized filters. In the present study, 16 inexperienced listeners judged the apparent spatial location (azimuth and elevation) of wideband noisebursts that were presented either over loudspeakers in the fkee-field (an anechoic or non-reverberent environment) or over headphones. The headphone stimuli were synthesized using HRTFs from a representative subject in a previous study [23]. Localization of both t%efield and virtual sources was quite accurate for 12 of the subjects, 2 showed poor elevation accuracy in both free-field and headphone conditions, and 2 showed degraded elevation accuracy only with virtual sources. High rates of confusion errors (reversals in judgments of azimuth and elevation) were also observed for some of the subjects and tended to increase for the virtual sources. In general, the data suggest that most listeners can obtain useful directional information from an auditory display without requiring the use of individually-tailored HRTFs, particularly for the dimension of azimuth. However, the high rates of confusion errors remain problematic. Several stimulus characteristics which may help to minimize these errors are discussed.

As with most research in information displays, virtual displays have generally emphasized visual information. Many investigators, however, have pointed out the importance of the auditory system as an information channel. We believe that a three-dimensional auditory display can substantially enhance situational awareness by combining spatial and semantic information to form dynamic, multidimensional patterns of acoustic events which convey meaning about objects in the spatial world of the user. Such a display can be realized with an array of real sound sources or loudspeakers (Doll et. al., 1986). The signal-processing device being developed at NASA-Ames maximizes flexibility and portability by synthetically generating three-dimensional sound in realtime for delivery through headphones. Unlike conventional stereo, sources can be perceived outside the head at discrete distances and directions from the listener. The 3-D auditory display is currently being integrated with Ames' Virtual Interactive Environment Workstation (VIEW) which allows the user to explore and interact with a 360-degree synthesized or remotely-sensed world using a head-mounted, wide-angle, stereoscopic display controlled by operator position, voice, and gesture.Applications of a three-dimensional auditory display involve any context in which the user's spatial awareness is important, particularly when visual cues are limited or absent. Examples include advanced teleconferencing environments, monitoring telerobotic activities in hazardous situations, and scientific "visualization" of multi-dimensional data. (e.g., Doll, et. al., 1986; Foley, 1987; Fisher, et. al., 1988; Brooks, 1988).

The authors describe the real-time acoustic display capabilities developed for the virtual environment workstation (VIEW) project. The acoustic display is capable of generating localized acoustic cues in real time over headphones. An auditor symbology, a related collection of representational auditory objects or icons, can be designed using the auditory cue editor, which links both discrete and continuously varying acoustic parameters with information or events in the display. During a given display scenario, the symbology can be dynamically coordinated in real time with three-dimensional visual objects, speech, and gestural displays. The types of displays feasible with the system range from simple warnings and alarms to the acoustic representation of multidimensional data or events.>