HE IEEE Signal Processing Society has its roots in an area where acoustics, speech, and signal processing converge, as was reflected in the former name of the society when it was founded in 1974. The interface between acoustics, speech, and signal processing is still an area of great interest to the society, with many fundamental problems still unsolved. Research is driven by applications where acoustic signals have to be captured, transmitted, and/or reproduced in an acoustic environment that includes echoes, noise, and reverberation Considering human/machine interfaces as a major area of applications, it is obvious that signal processing becomes more challenging as the distance between humans and the machines increases, as the signal bandwidth increases, and as the acoustic environment becomes more complex and hostile. Increasingly sophisticated algorithms have been developed since the mid-1970s and along with the availability of greatly increased and affordable computational power, multichannel signal processing algorithms naturally evolved for exploiting the spatial dimension of acoustic signals. The importance and popularity of this field was well reflected by the large number of submissions to this special issue. The volume of high-quality papers could not be fitted into the page budget allotted to us. Thus, we regrettably had to decide to publish some of them in a second section of this special issue as part of a regular issue of the TRANSACTIONS in early 2005. For sound reproduction, where we want to provide a pair of desired signals at the listeners’ ear drums, seamless human/machine interfaces based on multichannel techniques have been implemented since the invention of stereo systems. However, providing the true spatial sound experience in large listening spaces became possible only with new multichannel signal processing techniques, such as wavefield synthesis. Still, major challenges remain, especially phase-true equalization of listening room acoustics and the cancellation of local noise sources and interferers. On the other hand, acquisition of audio and speech signals has been a research topic since the invention of the microphone and still today presents major challenges for the signal processing community. Structurally the simplest problem, the acoustic feedback from loudspeakers into microphones is addressed by acoustic echo cancellation: From the single-channel case which has been investigated since the 1970s, research has moved on to stereo and multichannel reproduction, recently culminating in a new wave-domain adaptive filtering concept which has been presented for the first time at ICASSP 2004. For removing unwanted interference and noise from desired signals, multichannel techniques utilize spatial diversity to discriminate between desired and undesired components, either by exploiting different spatial coherence properties or by beamforming, which directs a beam of increased sensitivity towards the desired source. For traditional beamforming, source localization is necessary if the location of the source is not known a priori. Reflecting current research emphasis in these areas, the paper by Cohen addresses filtering of the

Network echo canceler chips are designed to handle several channels simultaneously. With the processing speeds now available, a single chip might handle several hundred channels. In current implementations, however, the adaptation algorithm is designed for a single channel, and the computations are replicated N/sub c/ times, where N/sub c/ is the number of channels. With such an implementation, the computational requirement is N/sub c/ times the peak load for a single channel. The number of computations required in each channel, however, varies widely over time. Therefore, a considerable reduction in computational load can be achieved by designing the system for the average load plus a margin to account for load variations. The reduction in complexity is achieved by exploiting three features: (a) the inherent pauses in conversations; (b) the sparseness of network echo paths; and (c) the fact that an adaptive filter does not need to be updated when the error signal is small. It is shown that, in principle, such a design can reduce the computational load by a very large factor - perhaps as large as thirty. It remains to be seen whether a customized hardware architecture can be implemented to take advantage fully of the proposed algorithm.

Echo cancelers which cover longer impulse responses (/spl ges/64 ms) are desirable. Long responses create a need for more rapidly converging algorithms in order to meet the specifications for network echo cancelers devised by the ITU (International Telecommunication Union). In general, faster convergence implies a higher sensitivity to near-end disturbances, especially "double-talk." Previously, a fast converging algorithm called the proportionate NLMS (normalized least mean squares) algorithm (PNLMS) has been proposed. This algorithm exploits the sparseness of the echo path in order to increase the convergence rate. A robust version of PNLMS has also been presented which combines a double-talk detector with techniques from robust statistics to make the algorithm insensitive to double-talk. This paper presents a generalization of the robust PNLMS algorithm to a robust proportionate aAffine projection algorithm (APA) called PAPA that converges very fast.

There is a need for echo cancelers for echo paths with long impulse responses (/spl ges/64 ms). This in turn creates a need for more rapidly converging algorithms in order to meet the specifications for network echo cancelers. Faster convergence, however, in general implies a higher sensitivity to near-end disturbances, especially "double-talk." Previously, a fast converging algorithm has been proposed called proportionate normalized least mean squares (PNLMS) algorithm. This algorithm exploits the sparseness of the echo path and has the advantage that no detection of active coefficients is needed. In this paper we propose a method for making the PNLMS algorithm more robust against double-talk. The slower divergence rate of these algorithms in combination with a standard Geigel double-talk detector improves the performance of a network echo canceler considerably during double-talk. The principle is based on a scaled nonlinearity which is applied to the residual error signal. This results in the robust PNLMS algorithm which diverges much slower than PNLMS and standard NLMS. Tradeoff between convergence and divergence rate is easily adjusted with one parameter and the added complexity is about seven instructions per sample which is less than 0.3% of the total load of a PNLMS algorithm with 512 filter coefficients. A generalization of the robust PNLMS algorithm to a robust proportionate affine projection algorithm (APA) is also presented. It converges very fast, and unlike PNLMS, is not as dependent on the assumption of a sparse echo path response. The complexity of the robust proportionate APA of order two is roughly the same as that of PNLMS.

One promising application in communications is desktop conferencing, which can involve several participants over a widely distributed area. Synthesized stereophonic sound will enable a listener to spatially separate one remote talker from another and thereby improve understanding. In such a scenario, we assume we are located in a hands-free environment where the composite acoustic signal is presented over loudspeakers, thus requiring acoustic echo cancellation. In this paper, we explain some of the methods that can be used to synthesize stereo sound and how such methods can be combined efficiently with stereo acoustic echo cancellation in the face of several difficult problems.

Stereophonic sound becomes more and more important in a growing number of applications (such as teleconferencing, multimedia workstations, televideo gaming, etc.) where spatial realism is demanded. Such hands-free systems need stereophonic acoustic echo cancelers (AECs) to reduce echos that result from coupling between loudspeakers and microphones in full-duplex communication. We propose a new stereo AEC based on two experimental observations: (a) the stereo effect is due mostly to sound energy below about 1 kHz and (b) comb filtering above 1 kHz does not degrade auditory localization. The principle of the proposed structure is to use one stereo AEC at low frequencies (e.g. below 1 kHz) with nonlinear transformations on the input signals and another stereo AEC at higher frequencies (e.g. above 1 kHz) with complementary comb filters on the input signals.

We consider the problem of detecting stop consonants in continuously spoken speech. We pose the problem as one of finding the optimal filter (linear or non-linear) that operates on a particular appropriately chosen representation. We discuss the performance of several variants of a canonical stop detector and consider its implications for human and machine speech recognition.

Virtual environment research has focused on interactive image generation and has largely ignored acoustic modeling for spatialization of sound.Yet, realistic auditory cues can complement and enhance visual cues to aid navigation, comprehension, and sense of presence in virtual environments.A primary challenge in acoustic modeling is computation of reverberation paths from sound sources fast enough for real-time auralization.We have developed a system that uses precomputed spatial subdivision and "beam tree" data structures to enable real-time acoustic modeling and auralization in interactive virtual environments.The spatial subdivision is a partition of 3D space into convex polyhedral regions (cells) represented as a cell adjacency graph.A beam tracing algorithm recursively traces pyramidal beams through the spatial subdivision to construct a beam tree data structure representing the regions of space reachable by each potential sequence of transmission and specular reflection events at cell boundaries.From these precomputed data structures, we can generate high-order specular reflection and transmission paths at interactive rates to spatialize fixed sound sources in real-time as the user moves through a virtual environment.Unlike previous acoustic modeling work, our beam tracing method: 1) supports evaluation of reverberation paths at interactive rates, 2) scales to compute highorder reflections and large environments, and 3) extends naturally to compute paths of diffraction and diffuse reflection efficiently.We are using this system to develop interactive applications in which a user experiences a virtual environment immersively via simultaneous auralization and visualization.

We discuss the philosophy of evaluating estimated impulse responses for applications in which the overall scaling or gain is irrelevant, as, for example, in blind channel identification. We argue that no matter how the estimate is obtained, the performance should be independently evaluated using an error measure that is appropriate for a problem of this class. Of several possible error measures, the normalized projection error, which is obtained by minimizing the l/sub 2/ norm over all possible gain values, seems to be the most natural and consistent. An example demonstrates that using an inappropriate error measure can produce misleading results.

Teleconferencing systems employ acoustic echo cancelers to reduce echoes that result from coupling between the loudspeaker and microphone. To enhance the sound realism, two-channel audio is necessary. However, in this case (stereophonic sound) the acoustic echo cancellation problem is more difficult to solve because of the necessity to uniquely identify two acoustic paths. We explain these problems in detail and give an interesting solution which is much better than previously known solutions. The basic idea is to introduce a small nonlinearity into each channel that has the effect of reducing the interchannel coherence while not being noticeable for speech due to self masking.

In many applications, such as teleconferencing, multimedia workstations, televideo gaming, etc., stereo sound is already, or will soon be, implemented to give spatial realism that mono systems cannot offer. In such hands-free systems, stereophonic acoustic echo cancelers are absolutely necessary for full-duplex communication. We propose a new acoustic echo canceler (AEC) based on a fundamental experimental observation that the stereo effect is due mostly to sound energy below about 1000 Hz. The principle of the hybrid mono/stereo AEC is to use stereophonic sound with a stereo AEC at low frequencies (e.g., below 1000 Hz) and monophonic sound with a conventional mono AEC at higher frequencies (e.g., above 1000 Hz). This solution is a good compromise between the complexity of a full-band stereo AEC and spatial realism. For the stereo case, we borrow from a previous innovation and add a small nonlinearity into each channel in order to accurately identify the two receiving room impulse responses.

Teleconferencing systems employ acoustic echo cancelers (AECs) to reduce echos that result from coupling between the loudspeaker and microphone. To enhance the sound realism, two-channel audio is necessary. However, in this case (stereophonic sound) the acoustic echo cancellation problem is more difficult to solve because of the necessity to uniquely identify two acoustic paths. We explain these problems in detail and give an interesting solution which is much better than previously known solutions. The basic idea is to introduce a small nonlinearity into each channel that has the effect of reducing the interchannel coherence while not being noticeable for speech due to self masking.

As teleconferencing systems evolve to an ever more lifelike and transparent audio/video medium, it will be necessary to incorporate multichannel audio, which at a minimum involves two channels, i.e., stereophonic sound. However, before full-duplex stereophonic teleconferencing can be deployed, the acoustic echo cancellation (AEC) problem must be solved in this regime. This paper draws attention to a fundamental problem of multichannel AEC that concerns the nonunique nature of the estimated receiving room impulse responses. Various potential solutions are discussed but are shown to be not entirely satisfactory.>

An important problem in speech coding is the quantization of linear predictive coefficients (LPCs) with the smallest possible number of bits while maintaining robustness to a large variety of speech material and transmission media. Since direct quantization of LPCs is known to be unsatisfactory, the authors consider this problem for an equivalent representation, namely, the line spectral frequencies (LSFs). To achieve an acceptable level of distortion a scalar quantizer for LSFs requires a 36 bit codebook. The authors derive a 30 bit two-quantizer scheme which achieves a performance equivalent to this scalar quantizer. The two-quantizer format consists of both a vector and a scalar quantizer such that for each input, the better quantizer is used. The vector quantizer is designed from a training set that reflects the joint density (for coding efficiency) and which ensures coverage (for robustness). The scalar quantizer plays a pivotal role in dealing better with regions of the space that are sparsely covered by its vector quantizer counterpart. A further reduction of 1 bit is obtained by formulating a new adaptation algorithm for the vector quantizer and doing a dynamic programming search for both quantizers. The method of adaptation takes advantage of the ordering of the LSFs and imposes no overhead in memory requirements. The dynamic programming search is feasible due to the ordering property. Subjective tests in a speech coder reveal that the 29 bit scheme produces equivalent perceptual quality to that when the parameters are unquantized.>

This paper reviews methods for mapping from the acoustical properties of a speech signal to the geometry of the vocal tract that generated the signal. Such mapping techniques are studied for their potential application in speech synthesis, coding, and recognition. Mathematically, the estimation of the vocal tract shape from its output speech is a so-called inverse problem, where the direct problem is the synthesis of speech from a given time-varying geometry of the vocal tract and glottis. Different mappings are discussed: mapping via articulatory codebooks, mapping by nonlinear regression, mapping by basis functions, and mapping by neural networks. Besides being nonlinear, the acoustic-to-geometry mapping is also nonunique, i.e., more than one tract geometry might produce the same speech spectrum. The authors show how this nonuniqueness can be alleviated by imposing continuity constraints.>

A modified Baum-Welch algorithm is developed and applied to estimating parameters of error source models that belong to the class of hidden Markov models (HMM). Such models arise in the description of bursty error statistics in communication channels. A key element used repeatedly for estimating parameters of such models is the computation of the likelihood of given sequences of observations. Several recursive methods are available for efficiently computing this likelihood. However, even recursive methods can require prohibitive amounts of computation if the observation sequences are very long. Modifications of the Baum-Welch reestimation algorithm that significantly reduces the computational requirements when the observation sequences contain long stretches of identical observations are discussed. The algorithms are used here to estimate parameters of a binary error source model using the results of computer simulation.>

The use of cumulant-based LP (linear prediction) analysis for speech recognition in the presence of noise is proposed. This method assumes the speech signal to be non-Gaussian. It is shown that cepstral coefficients derived by this method are quite insensitive to additive Gaussian noise which can be white or colored. The performance of a recognizer based on these estimates is compared to the performance of one that uses LP estimates derived from the autocorrelation function. It is found that at low SNR (below about 20 dB) the cumulant-based estimates outperform the autocorrelation-based estimates. At higher SNRs the reverse is true. The reasons for this behavior are not yet understood. However, it is shown that, by combining the two estimates, one can achieve recognition accuracy that is better than that of the conventional recognizer at all SNRs.>

The authors introduce a delayed-decision method based on dynamic programming for accessing such codebooks. The idea is to retrieve a sequence of shapes for a given sequence of spectra. It is shown that dynamic programming provides smooth parameter trajectories and phonetically reasonable tract shapes, even for difficult segments of speech, for example, during low-energy transitional portions of an utterance. >

The authors address the problem of how to accommodate varying speaker characteristics by selecting appropriate control parameters for a vocal tract/vocal cord model. This problem is basic to speech coding based on articulatory modeling. Experience shows that, for example, nominal vocal-tract length of the speaker, speech amplitude scale factor, as well as some glottal characteristics, cannot be accurately estimated on a frame-by-frame basis. The solution to this problem is found by a search procedure employing several frames of speech for estimating global speaker characteristics, in the form of scaling parameters, which is separate from the local (frame-to-frame) optimization of other parameters.>

It is shown that postfiltering circuits based on higher order LPC (linear predictive coding) models can provide very low distortion in terms of special tilt. Thus, they can provide better speech enhancement than circuits based on the backward-adaptive pole-zero predictor in ADPCM (adaptive digital pulse code modulation). Quantitative criteria for designing postfiltering circuits based on higher-order LPC models are discussed. These postfilters are particularly attractive for systems where high-order LPC analysis is an integral part of the coding algorithm. In a subjective test that used a computer-simulated version of these circuits, enhanced ADPCM obtained a mean opinion score of 3.6 at 16 kb/s.>

This paper proposes the use of a vocal cord and tract model for speech coding at bit rates below 4.8 kb/s. For this, a key requirement is the ability to derive model parameters from an input speech signal. Our approach to this problem employs an acoustic analysis front-end, a linked codebook of vocal-tract configurations and related acoustic characteristics, and an optimizing articulatory synthesizer. While the acoustic front-end is relatively straight-forward involving LPC, pitch, and voicing analyses, the codebook design and usage, as well as the specific method for optimizing the model parameters are new. The codebook is intended to provide good starting values for an iterative optimization, thus alleviating the problem of locking on to a locally optimum solution. In a first stage of optimization, the best vocal tract configuration found in the codebook is refined by varying only the vocal tract parameters. Then, in a second stage of optimization, the best match is found between the glottal waveform of the model and the inverse filtered input speech.

The performance of a recognizer based on the Itakura spectral distortion measure deteriorates when speech signals are corrupted by noise, specially if it is not feasible to train and to test the recognizer under similar noise conditions. To alleviate this problem, we consider a more noise-resistant, weighted spectral distortion measure which weights the high SNR regions in frequency more than the low SNR regions. For the weighting function we choose a "bandwidth broadened" test spectrum; it weights spectral distortion more at the peaks than at the valleys of the spectrum. The amount of weighting is adapted according to an estimate of SNR, and becomes essentially constant in the noise-free case. The new measure has the dot product form and computaional efficiency of the Itakura distortion measure in the autocorrelation domain. It has been tested on a 10 speaker, isolated digit data base in a series of speaker independent speech recognition experiments. Additive white Gaussian noise was used to simulate different SNR conditions (from 5 dB to ∞ dB). The new measure performs as well as the original unweighted Itakura distortion measure at high SNR's, and significantly better at medium to low SNRs. At an SNR of 5 dB, the new measure achieves a digit error rate of 12.49% while the original Itakura distortion gives an error rate of 27.6%. The equivalent SNR improvement at low SNR's, is about 5 - 7 dB.

In this paper we describe an algorithm for optimizing a microphone array with the objective of receiving a desired speech signal from a known direction, in the presence of broad band noise sources with unknown locations and possibly slowly varying characteristics. Each microphone feeds a tapped-delay-line filter, and the outputs of the filters are summed to give the final output. The tap weights of the filters are adjusted adaptively to minimize the output power subject to a constraint on the allowed distortion of the desired speech signal. Since the quality of a speech signal is quite insensitive to phase distortion, the constraint is on the magnitude transfer function from the source to the output. It is this choice of constraint that differentiates our approach from other methods of adapting arrays.

High quality speech at low bit rates (e.g., 2400 bits per second) is one of the important objectives of current speech research. As part of long range activity on this problem we have developed an efficient computer program that will serve as a tool for investigating whether articulatory speech synthesis may achieve this low bit rate. At a sampling frequency of 20 kHz the most comprehensive version of the program, including nasality and frication, runs at about five times real time on a Cray-1 computer.

To use probabilistic functions of a Markov chain to model certain parameterizations of the speech signal, we extend an estimation technique of Liporace to the eases of multivariate mixtures, such as Gaussian sums, and products of mixtures. We also show how these problems relate to Liporace's original framework.

In this paper we extend previous work on isolated word recognition based on hidden Markov models by replacing the discrete symbol representation of the speech signal by a continuous Gaussian mixture density. In this manner the inherent quantization error introduced by the discrete representation is essentially eliminated. The resulting recognizer was tested on a vocabulary of the 10 digits across a wide range of talkers and test conditions, and shown to have an error rate at least comparable to that of the best template recognizers and significantly lower than that of the discrete symbol hidden Markov model system. Several issues involved in the training of the continuous density models and in the implementation of the recognizer are discussed.

The theory of vector quantization (VQ) of linear predictive coding (LPC) coefficients has established a wide variety of techniques for quantizing LPC spectral shape to minimize overall spectral distortion. Such vector quantizers have been widely used in the areas of speech coding and speech recognition. The conventional vector quantizer utilizes only spectral shape information and essentially disregards the energy or gain term associated with the optimal LPC fit to the signal being modelled. In this paper we present a method of incorporating LPC spectral shape and energy into the codebook entries of the vector quantizer. To do this we postulate a distortion measure for comparing two LPC vectors which uses a weighted sum of an LPC shape distortion and a log energy distortion. Based on this combined distortion measure we have designed and studied vector quantizers of several sizes for use in isolated word speech recognition experiments. We have found that a fairly significant correlation exists between LPC shape and signal energy; hence a combined LPC shape plus energy vector quantizer with a given distortion requires far fewer codebook entries than one in which LPC shape and energy are quantized separately. Based on isolated word recognition tests on both a 10-digit and a 129 word airlines vocabulary, we have found improvements in recognition accuracy by using the VQ with both LPC shape and energy over that obtained using a VQ with LPC shape alone.

A method for speaker independent isolated digit recognition based on modeling entire words as discrete probabilistic functions of a Markov chain is described. Training is a three part process comprising conventional methods of linear prediction coding (LPC) and vector quantization of the LPCs followed by an algorithm for estimating the parameters of a hidden Markov process. Recognition utilizes linear prediction and vector quantization steps prior to maximum likelihood classification based on the Viterbi algorithm. Vector quantization is performed by a K-means algorithm which finds a codebook of 64 prototypical vectors that minimize the distortion measure (Itakura distance) over the training set. After training based on a 1,000 token set, recognition experiments were conducted on a separate 1,000 token test set obtained from the same talkers. In this test a 3.5% error rate was observed which is comparable to that measured in an identical test of an LPC/DTW (dynamic time warping) system. The computational demand for recognition under the new system is reduced by a factor of approximately 10 in both time and memory compared to that of the LPC/DTW system. It is also of interest that the classification errors made by the two systems are virtually disjoint; thus the possibility exists to obtain error rates near 1% by a combination of the methods. In describing our experiments we discuss several issues of theoretical importance, namely: 1) Alternatives to the Baum-Welch algorithm for model parameter estimation, e.g., Lagrangian techniques; 2) Model combining techniques by means of a bipartite graph matching algorithm providing improved model stability; 3) Methods for treating the finite training data problem by modifications to both the Baum-Welch algorithm and Lagrangian techniques; and 4) Use of non-ergodic Markov chains for isolated word recognition. We note that the experiments reported here are the first in which a direct comparison is made between two conceptually different (i.e. parametric and non-parametric) methods of treating the non-stationarity problem in speech recognition by implicitly dividing the speech signal into quasi-stationary intervals.

A wide variety of distance measures for comparing sets of LPC coefficients have been proposed. The most popular one has been the log likelihood ratio test as proposed by Itakura [1]. Although this measure is both computationally efficient, and theoretically sound, de Souza, in a recent paper [2], found that the measured distribution of LPC distances was significantly different from the one predicted by theory. De Souza then proposed alternative statistical tests for measuring LPC distances. In this paper we present both theoretical and experimental results which show that there is, in general, excellent agreement between theory and practice when using the log likelihood ratio to measure LPC distance. Included in the paper are discussions of the effects of LPC analysis method (covariance or auto-correlation), the use of fixed pre-emphasis, and the effects of additive, uncorrelated noise on the distributions.

In this paper we present new and computationally efficient algorithms for computing some statistical properties of finite languages. In particular, the relative redundancy which measures grammatical constraint, is computed for several speech recognition task languages which have appeared in the literature.