We describe the construction and use of a laboratory facility for testing the performance of speech-to-speech translation devices. Approximately 1500 English phrases from various military domains were recorded as spoken by each of 30 male and 12 female English speakers with variation in speaker accent, for a total of approximately 60,000 phrases available for experimentation. We describe an initial experiment using the facility which shows the impact of environmental noise and speaker variability on phrase recognition accuracy for two commercially available one-way speech-to-speech translation devices configured for English-to-Arabic.

The intelligibility of speech transmitted through low-rate coders is severely degraded when high levels of acoustic noise are present in the acoustic environment. Recent advances in nonacoustic sensors, including microwave radar, skin vibration, and bone conduction sensors, provide the exciting possibility of both glottal excitation and, more generally, vocal tract measurements that are relatively immune to acoustic disturbances and can supplement the acoustic speech waveform. We are currently investigating methods of combining the output of these sensors for use in low-rate encoding according to their capability in representing specific speech characteristics in different frequency bands. Nonacoustic sensors have the ability to reveal certain speech attributes lost in the noisy acoustic signal; for example, low-energy consonant voice bars, nasality, and glottalized excitation. By fusing nonacoustic low-frequency and pitch content with acoustic-microphone content, we have achieved significant intelligibility performance gains using the DRT across a variety of environments over the government standard 2400-bps MELPe coder. By fusing quantized high-band 4-to-8-kHz speech, requiring only an additional 116 bps, we obtain further DRT performance gains by exploiting the ear's insensitivity to fine spectral detail in this frequency region.

In order for a test method to yield meaningful results the equipment and procedures used to collect and format data for that test must be accurate and consistent. A laboratory evaluating narrowband digital voice coders must be prepared to generate data for a variety of test methods across numerous voice systems without introducing extraneous variability. This paper describes the functions of the host laboratory for the Department of Defense (DoD) 2400 bps vocoder selection process. The generation of calibrated, representative audio input material and the collection of processed speech is discussed. The inclusion of calibration and reference speech systems, a channel simulator, and a common testbed interface is presented. The implementation of a communicability test method is described. Strict attention to detail in preparation and procedure allow these methods to be the basis for any voice communication system evaluation and selection process.