Our past research has focused on wired microphone arrays having a large number of elements and wide aperture. The wiring for these large arrays is tedious and only a tree structure with intermediate multiplexing of signals works well. A wireless array alleviates this immense wiring issue, albeit other difficulties are introduced including interferences in the wireless transmissions, synchronization, the need for automatic self-calibration, and module power, size and mounting. This paper introduces our first successful wireless array system, HMA-III. It discusses the desired attributes for such an array and the tradeoffs made to assemble a first working system. Results for the automatic self-calibration are presented.

Large-aperture microphone arrays can be used to capture and enhance speech from individual talkers in noisy, multi-talker, and reverberant environments. However, they must be calibrated, often more than once, to obtain accurate 3-dimensional coordinates for all microphones. Direct-measurement techniques, such as using a measuring tape or a laser-based tool are cumbersome and time-consuming. Some previous methods that used acoustic signals for array calibration required bulky hardware and/or fixed, known source locations. Others, which allowed more flexible source placement, often have issues with real data, have reported results for 2D only, or work only for small arrays. This paper describes a complete and robust method for automatic calibration using acoustic signals which is simple, repeatable, accurate, and has been shown to work for a real system. The method requires only a single transducer (speaker) with a microphone attached above its center. The unit is freely moved around the focal volume of the microphone array generating a single long recording from all the microphones. After that, the system is completely automatic. We describe the free source method (FrSM), validate its effectiveness and present accuracy results against measured ground truth. The performance of FrSM is compared to that from several other methods for a real 128-microphone array.

Two new methods for locating multiple sound sources using a single segment of data from a large-aperture microphone array are presented. Both methods employ the proven-robust steered response power using the phase transform (SRP-PHAT) as a functional. To cluster the data points into highly probable regions containing global peaks, the first method fits a Gaussian mixture model (GMM), whereas the second one sequentially finds the points with highest SRP-PHAT values that most likely represent different clusters. Then the low-cost global optimization method, stochastic region contraction (SRC), is applied to each cluster to find the global peaks. We test the two methods using real data from five simultaneous talkers in a room with high noise and reverberation. Results are presented and discussed.

An important application for microphone arrays is to extract high-quality output from a single wideband source in multi-source and adverse environments. Methods based on blind-source separation and beamforming have been proposed in the literature. In this paper, we propose a new algorithm for isolating sources which may be considered an alternate approach to adaptive beamforming. The proposed method assigns every time-frequency point to one of the sources in the environment or as background noise using an SRP-PHAT-based discriminator. It then creates the desired source's output by spectrally subtracting the inappropriate time-frequency points. After giving the details of the procedure, we show results from real data from our large-aperture microphone array. Though, clearly a relatively small subset, listening results to date for two or three talkers have yielded high-quality isolations. For two talkers, in particular, results are presented which indicate that the method correctly identifies and removes about 90% of those time-frequency points where the interfering signal is dominant, without affecting the desired signal significantly.

Knowing the orientation of a talker in the focal area of a large-aperture microphone array enables the development of better beamforming algorithms (to obtain higher-quality speech output), improves source-location/tracking algorithms, and allows better selection and control of cameras in a video conference situation. Measurements in an anechoic room (e.g., Chu and Warnock, 2002) have quantified the average frequency-dependent magnitude (source radiation pattern) of the human speech source showing a front-to-back difference in magnitude that increases with frequency by about 8 dB/decade reaching about 18 dB at 8000 Hz. These amplitude differences, while severely masked by both coherent and noncoherent noise in a real environment, are the most extractable phenomena from a talker's orientation when compared to other phenomena such as phase differences due to the source or effects due to diffraction at the mouth. In this paper, we propose a robust, source-radiation-pattern-based method for extraction of the azimuth angle of a single talker for whom an accurate point-source location estimate is known. The method requires no a priori training and has been tested in more than 100 situations with real human talkers having various locations and orientations in a room equipped with a large aperture microphone array. We compare these results against earlier published algorithms and find that the method proposed herein is the most robust and is sufficient to be considered for a real time system.

In this paper we present a new method for locating multiple sound sources using only a local segment of data from a large-aperture microphone array. The result of this work may be used directly or as an open-loop input to a tracking algorithm. The proposed method employs the proven-robust steered response power using the phase transform as a functional, agglomerative clustering, and low-cost global optimization (stochastic region contraction). We test the algorithm with five simultaneous “talkers” under very difficult conditions in a real room but using electrical speakers instead of human talkers to have a controlled experiment. Results are presented and discussed.

Knowing the orientation of a talker allows a a large-aperture microphone array to select and control cameras better in a teleconferencing situation, improve source-location estimation, and, often, improve beamforming. In 2004, we introduced a baseline algorithm for determining orientation azimuth. Recent testing showed the baseline algorithm behaved poorly when the source was not in the center of the focal area for the array. Here, we describe a second-generation algorithm, A2, that has overcome many of the baselinepsilas short-falls. It still extracts the estimate from microphone energies, but is improved by 1) using a narrow-band, high-frequency analysis, rather than the broad band of the baseline algorithm, 2) using spectral subtraction for uncorrelated noise removal and 3) fitting the processed microphone energies to an ideal model for the direct-wave energy. Most important is that 3) incorporates inverse-square-law effects properly on the direct wave only, which was not the case in the baseline. Results from an advanced simulator are presented to illustrate the issues. Then, A2 and baseline algorithm results are compared using about 60 direct recordings from a human talker in a typical and noisy environment using our 448-microphone array. These show that A2 is a significant improvement.

Microphone arrays often operate in the near field, which complicates the problem of determining a source location from time-difference-of-arrival (TDOA) measurements typically derived from generalized cross-correlation functions. Each TDOA satisfies the equation of a hyperboloid in space and methods have been developed to either solve for intersecting hyperboloids or make some approximation to them, keeping source-location determination a nonlinear, somewhat complex problem. We introduce a closed-form, analytic solution for the problem (the GS algorithm). It is so simple that we were surprised that, until very recently, there have been no other solutions similar to ours. The method uses a minimum of five microphones in three dimensions, one more than other solutions, but, for nonsingular layouts of the microphones, it is very fast and accurate. First, the new method is compared to other closed-form methods for accuracy and sensitivity to noise using simulated data. Then, several variants of GS are compared to two other real-time algorithms, LEMSalg and SRP-PHAT, using real, human-talker data from a large array in a noisy room.

In most microphone array applications, it is essential to localize sources in a noisy, reverberant environment. It has been shown that computing the steered response power (SRP) is more robust than faster, two-stage, direct time-difference of arrival methods. The problem with computing SRP is that the SRP space has many local maxima and thus computationally-intensive grid-search methods are used to find a global maximum. Grid search is too expensive for a real-time system. Several papers have addressed this issue. In this paper we propose using stochastic region contraction (SRC) to make computing the SRP practical. We discuss one important SRP method, computing it from the phase transform (SRP-PHAT), review SRC, and show the computational saving. Using real data from human talkers, we show that SRC saves computation by more than two orders of magnitude with almost no loss in accuracy.

For about 60 years, the spectrogram has been an excellent visualization tool for observing the acoustic and prosodic features of a speech signal. However, a visual tool has been lacking for information that is typically represented by the speech signal's phase. When a talker is remote from one or more microphones, the times of arrival of the direct wave and those of reflections become important. In this letter, we introduce two visualizations that have proven to be quite useful for us in understanding phasic behaviors of microphone arrays. A time delay versus frequency plot for a window of time is called the time-delay graph, and an ensemble of projected time-delay graphs is called the delayogram. These visualizations are introduced, and simulated and real-speech data examples of both displays are presented. The time-delay graph is compared to the phase transform generalized cross-correlation (GCC-PHAT) for two real-data examples.

Large-aperture microphone arrays have the potential for providing quality speech acquisition from multiple talkers over a large focal area for such applications as teleconferencing and speech-recognition input. The cost of computing is rapidly approaching the point at which these arrays will be practical in many common environments. However, an important issue is the calibration of an array. We discuss procedures to accurately determine both the coordinates in three-dimensions for the position of each microphone and the individual gains of each microphone/microphone channel. For the positions, we found that calibration by direct measurement, using a surveyor's transit, is simply too error prone, time consuming, and difficult for large arrays and is too inaccurate for small arrays. We have also seen that without the careful matching of inexpensive electret microphones, the channel sensitivities vary by as much as 6 dB and performance can be enhanced by suitable compensation. This paper describes new apparatus and techniques for automatic calibration using acoustic signals. Experimental results using both small and large-aperture microphone array systems show the acoustically-determined positions to be more consistent than the ones measured directly. Gain measurements are somewhat more difficult but gains can be found within 1-2 dB with reasonable care.

A large array of microphones is being studied as a possible means of acquiring data in offices, conference rooms, and auditoria without requiring close-talking microphones. An array that surrounds all possible sources has a large aperture and such arrays have attractive properties for accurate spatial resolution and significant signal-to-noise enhancement. For the first time, this paper presents all the details of a real-time, source-location algorithm (LEMSalg) based on time-of-arrival delays derived from a phase transform applied to the generalized cross-power spectrum. It is being used successfully in a representative environment where microphone SNRs are below 0 dB. We have found that many small features are required to make a useful location estimating algorithm work and work well in real-time. We present an experimental evaluation of the current algorithm's performance using data taken with the Huge Microphone Array (HMA) system, which has 448 microphones in a noisy, reverberant environment. Using off-line computation, we also compared the LEMSalg to two alternative methods. The first of these adds local beamforming to the preprocessing of the base algorithm, increasing performance significantly at modest additional computational cost. The second algorithm maximizes the total steered-response power in the same phase transform. While able to derive good position estimates from shorter data runs, this method is two orders of magnitude more computationally expensive and is not yet suitable for real-time use.

We have shown that knowing the orientation of a talker in a large-aperture microphone array system can significantly improve location-estimation and beamforming algorithms. Measurements of a talker in an anechoic chamber have shown significant anisotropy in radiation patterns that may be used to influence the selection, processing, and weighting of microphone signals in such algorithms. We introduce a simple method for determining the orientation of a talker within a large focal area using only acoustic energy data obtained from the array. The mathematical basis for this procedure is presented and computed performance, based solely on acoustical measurements in a real environment, are listed and discussed.

Location estimation is a vital aspect of a real-time large aperture array system. To date, only time-difference-of-arrival (TDOA) based algorithms run sufficiently fast for a real-time system, so we have been using this kind of locator, LEMSalg, in our real-time environment. We present an improved location estimation algorithm. In the new method, microphone-pair selection is made dynamic, rather than fixed a priori. Dynamic selection requires derivation of parameters and criteria. Here we use theoretical analysis, real system statistics and experimental results to obtain: 1) a suitable range for microphone-pair separation; 2) an ideal length for the TDOA vector; 3) an effective TDOA quality assessment criterion. Experimental results based on real speech recorded in a moderately noisy environment at 5 different SNR levels are given to show the performance improvements.

Large-aperture microphone arrays have the potential for providing quality speech acquisition from multiple talkers over a large focal area for such applications as teleconferencing and speech-recognition input. The cost of supporting computing technology is quickly approaching the point in which these arrays will be practical in many common environments. However, an important issue is the calibration of an array that has been instal1ed in some medium-sized to large room. By calibration, we mean the determination of the precise set of coordinates in three-dimensions for the positions of the microphones. Calibration by direct measurement or even by a 'laser transit is simply too error-prone, time-consuming and painful. This paper describes new apparatus and techniques for automatic calibration using acoustic signals. Experimental results using our large-aperture microphone array system show the automatical1y-determined positions to be even more consistent than the ones measured directly. The paradigm for using the apparatus and technique most advantageously is also presented.

Microphone arrays offer the potential of obtaining a high-quality speech signal from a remote talker in a noisy, multi-source environment. In many important applications, sources of interest may be located anywhere within a large focal area and thus it is desirable that an array's performance be uniform over that area. In this paper, simulated and measured results are presented indicating the performances of representative small and large aperture arrays at various points in a large focal area. From these results, it is clear that uniformity over a large focal area is an advantage of large-aperture arrays.

Poor sound pick up by remote microphones in multimedia applications, conference rooms and auditoria has traditionally hampered speech recognition and communication among spatially-separated groups. The problems are reverberation, acoustic noise, and the variability of the radiation pattern of unconstrained talkers. One potential solution that is becoming increasingly practical is to use an array of microphones and sophisticated signal processing. In this paper a brief description of a large, real-time, working system is presented and its measured beamforming performance is compared to what is predicted from a mathematical model. A combination of a synchronized test signal/system and a careful mathematical model results in the performances matching surprisingly well. From this match of theory to practice, we are able to draw some important inferences about future system improvements.

The use of arrays of microphones having a large number of elements (hundreds) is now in place in research laboratories, and will soon be practical for real applications. In rooms of auditorium or conference size, a large number of microphones will virtually always imply that the aperture will be large compared to the focal distance. This requires understanding the volume selectivity of irregular, widely-distributed sets of microphones. In this paper, several pictures of the beamforming performance of large-aperture distributions of microphones are presented. The purpose is to illustrate some of the pitfalls in large-aperture microphone-array design.

This correspondence reports results for a tracking, real-time microphone array as an input to a hidden Markov model based (HMM-based) connected alpha-digits speech recognizer. For a talker in the near field of the array (within 0.5 m), performance approaches that of a close-talking microphone input device.

A major problem for speech recognition systems is relieving the talker of the need to use a close-talking, head-mounted or a deskstand microphone. A likely solution is the use of an array of microphones that can steer itself to the talker and can use a beamforming algorithm to overcome the reduced signal-to-noise ratio due to room acoustics. This paper reports results for a tracking, real-time microphone-array as an input to an HMM-based connected alpha-digits speech recognizer. For a talker in the very near field of the array (within a meter), performance approaches that of a close-talking microphone input device. The effects of both the noise reducing steered array and the use of a maximum a posteriori (MAP) training step are shown to be significant. Here, the array system and the recognizer are described, experiments are presented, and the implications of combining these two systems discussed.

Typically, parameter estimation for a hidden Markov model (HMM) is performed using an expectation-maximization (EM) algorithm with the maximum-likelihood (ML) criterion. The EM algorithm is an iterative scheme that is well-defined and numerically stable, but convergence may require a large number of iterations. For speech recognition systems utilizing large amounts of training material, this results in long training times. This paper presents an incremental estimation approach to speed-up the training of HMMs without any loss of recognition performance. The algorithm selects a subset of data from the training set, updates the model parameters based on the subset, and then iterates the process until convergence of the parameters. The advantage of this approach is a substantial increase in the number of iterations of the EM algorithm per training token, which leads to faster training. In order to achieve reliable estimation from a small fraction of the complete data set at each iteration, two training criteria are studied; ML and maximum a posteriori (MAP) estimation. Experimental results show that the training of the incremental algorithms is substantially faster than the conventional (batch) method and suffers no loss of recognition performance. Furthermore, the incremental MAP based training algorithm improves performance over the batch version.

Armstrong III is a 20 node multi-computer that is currently operational. In addition to a RISC processor, each node contains reconfigurable resources implemented with FPGAs. The in-circuit reprogramability of static RAM based FPGAs allows the computational capabilities of a node to be dynamically matched to the computational requirements of an application. Most reconfigurable computers in existence today rely solely on a large number of FPGAs to perform computations. In contrast, the paper demonstrates the utility of a small number of FPGAs coupled to a RISC processor with a simple interconnect. The article describes a substantive example application that performs HMM training for speech recognition with the reconfigurable platform.

Conventional time-delay estimators exhibit dramatic performance degradations in the presence of multipath signals. This limits their application in reverberant enclosures, particularly when the signal of interest is speech and it may not possible to estimate and compensate for channel effects prior to time-delay estimation. This paper details an alternative approach which reformulates the problem as a linear regression of phase data and then estimates the time-delay through minimization of a robust statistical error measure. The technique is shown to be less susceptible to room reverberation effects. Simulations are performed across a range of source placements and room conditions to illustrate the utility of the proposed time-delay estimation method relative to conventional methods.

The Huge Microphone Array (HMA) project started in February 1994 to design, construct, and test a real-time 512-microphone array system and to develop algorithms for use on it. Analysis of known algorithms showed that signal-processing performance of over 6 Gigaflops would be required; at the same time, there was a need for "portability", i.e., fitting into a small van. These tradeoffs and many others have led to a unique design in both hardware and software. This paper presents the design and its justifications. Performance data for a few important algorithms relative to usage of processing-capability, response latency, and difficulty of programming are discussed.

A method for tracking the positional estimates of multiple talkers in the operating region of an acoustic microphone array is presented. Initial talker location estimates are provided by a time-delay-based localization algorithm. These raw estimates are spatially smoothed by a Kalman filter derived from a set of potential source motion models. Data association techniques based on the estimate clusterings and source trajectories are incorporated to match location observations with individual talkers. Experimental results are presented for array recorded data using multiple talkers in a variety of scenarios.

Armstrong III is a multi node multi-computer designed and built at the Laboratory for Engineering Man/Machine System (LEMS) of Brown University. Each node contains a RISC processor and reconfigurable resources implemented with FPGAs. The primary benefit in using FPGAs is that the resulting hardware is neither rigid nor permanent but is in-circuit reprogrammable. This allows each node to be tailored to the computational requirements of an application. This paper describes the Armstrong III architecture and concludes with a substantive example application that performs HMM Training for speech recognition with the reconfigurable platform.

The linear intersection (LI) estimator, which is a closed-form method for the localization of source positions given sensor array time-delay estimate information, is presented. The LI estimator is shown to be robust and accurate, to closely model the search-based ML estimator, and to outperform a benchmark algorithm. The computational complexity of the LI estimator is suitable for use in real-time microphone-array applications where search-based location algorithms may be infeasible.

For a hidden Markov model (HMM) based speech recognition system it is desirable to combine enhancement of the acoustical signal and statistical representation of model parameters, ensuring both a high quality speech signal and an appropriately trained HMM. In this paper the incremental variant of maximum a posteriori (MAP) estimation is used to adjust the parameters of a talker-independent HMM-based speech recognition system to accurately recognize speech data acquired with a microphone-array. The approach is novel for a microphone-array speech recognition task in that a robust talker-independent model is derived from a baseline system using a relatively small amount of data for training. The results show that (1) ILIAP training significantly improves recognition performance compared to the baseline, and (2) beamforming signal enhancement outperforms single-channel enhancement before and after the adaptive MAP training.

This paper presents a means for predicting the error region associated with a speech-source location estimate obtained from a set of microphones in a room environment. The error predictor presented is derived assuming a specific source-sensor geometry consisting of pairs of closely-spaced sensors for which a delay estimate associated with the potential source has been evaluated. The accuracy of the predictor is evaluated through a set of Monte Carlo simulations and an application of the predictor to microphone-array design in the context of a video-teleconferencing scenario is presented.

Conventional training of a hidden Markov model (HMM) is performed by an expectation-maximization algorithm using a maximum likelihood (ML) criterion. It was reported that, using an incremental variant of maximum a posteriori estimation, substantial speed improvements could be obtained. The approach requires a prior distribution when the training starts, although it is difficult to find an appropriate prior for some cases. This paper presents a new approach for achieving an efficient training of HMM parameters using the standard ML criterion. A prior distribution is not required. The algorithm sequentially selects a subset of data from the training set, updates the parameters from the subset, then iterates until convergence. There is a solid theoretical foundation that ensures a monotone likelihood improvement; thus stable convergence is guaranteed. Experimental results indicate substantially faster convergence than the standard batch training algorithm while holding the same level of recognition performance.

The search for better and more robust performance of speech recognition systems is ongoing. Much of the improvement is likely to come from better acoustic feature analysis. The results from a significant experiment are reported; these show how a warped-DFT analysis outperforms an LPC-cepstral analysis in a significant way, supporting results by other researchers for different recognition tasks. An analysis of nasal-letter performance is used to show the development of the warped-DFT feature analysis.

Genetic algorithms (GAs) are a currently popular method for nonlinear optimization that can be used to provide a solution for the chip partitioning problem. Unfortunately, GAs usually require prohibitively large computation times on current workstations. This paper demonstrates the utility of the Armstrong III architecture by addressing the computational problems associated with partitioning large designs using GAs. An example GA is presented for chip partitioning that runs on Armstrong III. GA computation bottlenecks are identified and hardware implementation strategies are discussed. Results are presented that show the Armstrong III architecture can be adapted to execute a GA in significantly less time than current workstations.

The linear intersection (LI) estimator, a closed-form method for the localization of source positions given only the sensor array time-delay estimate information, is presented. The array is constrained to be composed of 4-element sub-arrays configured in 2 centered orthogonal pairs. A bearing line in 3-space is estimated from each sub-array and potential source locations are found via closest intersection of bearing line pairs. The final location estimate is determined by a probabilistic weighting of these potential locations. The LI estimator is shown to be robust and accurate, to closely model the ML estimator, and to outperform a representative algorithm. The computational complexity of the LI estimator is suitable for use in real-time microphone-array applications.

Continuous density observation hidden Markov models (CD-HMMs) have been shown to perform better than their discrete counterparts. However, because the observation distribution is usually represented with a mixture of multivariate normal densities, the training time for a CD-HMM can be prohibitively long. This paper presents a new approach to speed-up the convergence of CD-HMM training using a stochastic, incremental variant of the EM algorithm. The algorithm randomly selects a subset of data from the training set, updates the model using maximum a posteriori estimation, and then iterates until convergence. Experimental results show that the convergence of this approach is nearly an order of magnitude faster than the standard batch training algorithm. In addition, incremental learning of the model parameters improved recognition performance compared with the batch version.

Describes the design and use of a digital acoustic array signal processing research environment. The system unites the interactive numerical processing capabilities of The MathWorks' MATLAB with multichannel data acquisition/signal processing hardware and an overhead Cartesian robot for accurate sound source placement. In conjunction with additional electronic and electromechanical peripherals, the user-friendly environment facilitates the spatial and temporal analysis and evaluation of acoustic array algorithms. Real-data experiments and their results are provided to demonstrate the flexibility and ease-of-use of the array system.

A spoken language system combines speech recognition, natural language processing and human interface technology. It functions by recognizing the person's words, interpreting the sequence of words to obtain a meaning in terms of the application, and providing an appropriate response back to the user. Potential applications of spoken language systems range from simple tasks, such as retrieving information from an existing database (traffic reports, airline schedules), to interactive problem solving tasks involving complex planning and reasoning (travel planning, traffic routing), to support for multilingual interactions. We examine eight key areas in which basic research is needed to produce spoken language systems: (1) robust speech recognition; (2) automatic training and adaptation; (3) spontaneous speech; (4) dialogue models; (5) natural language response generation; (6) speech synthesis and speech generation; (7) multilingual systems; and (8) interactive multimodal systems. In each area, we identify key research challenges, the infrastructure needed to support research, and the expected benefits. We conclude by reviewing the need for multidisciplinary research, for development of shared corpora and related resources, for computational support and far rapid communication among researchers. The successful development of this technology will increase accessibility of computers to a wide range of users, will facilitate multinational communication and trade, and will create new research specialties and jobs in this rapidly expanding area.>

This paper describes methods of talker clustering and identification based on a "distance" metric between discrete HMM output probabilities. Output probabilities are derived on a tree-based MMI partition of the feature space, rather than the usual vector quantization. The information divergence (relative entropy) between speaker-dependent models is used as a quantitative measure of how much a given talker differs from another talker. An immediate application is talker identification: an unknown speaker may be identified by finding the closest speaker-dependent reference model to a model trained on the unknown speaker's data. Another application is to cluster similar talkers into a group; these may be used to train a HMM model that represents that talker better than a more general model. It is shown that using the model "nearest" a novel talker enhances the performance of a talker-independent speech recognition system.>

Hidden Markov models (HMMs) have been quite successfully applied to speech recognition tasks, but many unsolved problems still remain. HMMs do not directly model all phenomena that might be useful for recognition. This is the case, for example, for duration modeling. Mechanisms are needed to incorporate additional information into an HMM system. This paper presents a maximum a posteriori (MAP) parameter estimation approach for improving the state-duration modeling capability and incorporating a priori knowledge about the word-duration distribution into an HMM. The MAP-based approach is evaluated on a talker-independent, connected alphadigit task for various prior distributions on duration. The results-in terms of both computational complexity and recognition performance-are compared with the results of HMM-based systems trained with the traditional maximum-likelihood criterion.>

A microphone array has the capability of capturing the properties of a significant portion of a talker's radiation pattern. In this paper, potential implications of measuring the radiation pattern of a talker for the recognition and enhancement of speech are discussed. Current applications of microphone arrays entail their installation in small enclosures such as a conference room or an automobile, typically placing a talker in the array's near field. Fitting a nominal acoustic model to a sparse sampling of the radiation pattern yields parameter estimates which should prove useful as features for speech recognition. Parameter-estimation results using fixed-radius sources (loudspeakers) are presented. Other considerations unique to placement of a talker in close proximity to an array are discussed.>

A feature set that captures the dynamics of formant transitions prior to closure in a VCV environment is used to characterize and classify the unvoiced stop consonants. The feature set is derived from a time-varying, data-selective model for the speech signal. Its performance is compared with that of comparable formant data from a standard delta-LPC-based model. The different feature sets are evaluated on a database composed of eight talkers. A 40% reduction in classification error rate is obtained by means of the time-varying model. The performance of three different classifiers is discussed. A novel adaptive algorithm, termed learning vector classifier (LVC) is compared with standard K-means and LVQ2 classifiers. LVC is a supervised learning classifier that improves performance by increasing the resolution of the decision boundaries. Error rates obtained for the three-way (p, t, and k) classification task using LVC and the time-varying analysis are comparable to that of techniques that make use of additional discriminating information contained in the burst. Further improvements are expected when an expanded time-varying feature set is utilized, coupled with information from the burst.>

An explicit-duration hidden Markov model (HMM) algorithm for speech recognition has been proposed that potentially provides a more precise and versatile duration model than the implicit models ordinarily used, but at the cost of increased computation. The authors address the computational issues involved in conventional and explicit-duration HMM training by providing an analysis of the algorithm, suggesting serial enhancements and two efficient parallel implementations, and presenting experimental results on both common network workstations as well as a parallel system.>

A neural network formulation for an HMM (hidden Markov model) is presented, and training using maximum likelihood, maximum mutual information, minimum mean-squared-error (MMSE), and unconstrained MMSE is described. Recognition results are presented for the variously trained models evaluated on a speaker-independent, connected alphadigit speech recognition task. It is concluded that viewing neural networks as HMMs provides a framework for building temporally dependent neural networks, while viewing HMMs as neural networks broadens the class of natural training methods. Despite several drawbacks, performance results indicate that models trained with error-correcting criteria on sufficient amounts of data may do better at discriminating similar sounds.>

A feature set that captures the dynamics of formant transitions is utilized to classify the unvoiced stop consonants. The second formant and its slope are used to characterize the transition between the vowel and the closure in a VCV (vowel-consonant-vowel) environment. The performance of a feature set obtained by means of a time-varying, closed-glottis model for the signal is compared with that of a standard LPC (linear predictive coding) model. The different feature sets are evaluated on a database consisting of eight speakers. A fourfold reduction in the error rate is obtained by means of the more sophisticated model. The performance of three different classifiers is presented. A novel adaptive algorithm, the learning vector classifier, is compared with standard K-means and LVQ2 (learning vector quantization-2) classifiers. Error rates of 5% are obtained for the three-way classification.>

Substantial gains can be achieved by allowing the configuration and fundamental operations of a processor to adapt to a user's program. A method is presented for improving the performance of many computationally intensive tasks by extracting information at compile-time to synthesize new operations that augment the functionality of a core processor. The newly synthesized operations are targeted to RAM-based reconfigurable logic located within the processor. A proof-of-concept system called PLADO, consisting of a C configuration compiler and a hardware platform, is presented. Computation and performance results confirm the concept viability, and demonstrate significant speed-up.>

Stochastic region contraction is applied to an improved field model which includes attenuation due to the source-to-microphone distance, and a new optimal spacing is derived. The performances of equispaced, logarithmically spaced, old-model optimal, and new-model optimal arrays were measured experimentally and are compared to the results predicted from the mathematical model. The shape of the power distribution of the received signal in the near field of the array is verified in general. The differences are due to echoes, background noise, nonideal microphone beam patterns, and other variables that were not included in the model.>

Two data-selective techniques that facilitate formant tracking with greater temporal resolution are presented: a time-varying autoregressive (AR) model and a short window AR model. They are used to analyze the regions of voicing immediately preceding closure in vowel-consonant transitions. Results on real speech data show that these methods detect the rapid change in the formants prior to closure better than standard linear predictive coding methods; hence, they provide important cues as to the nature of the stop. When these cues are used in conjunction with the commonly used ones from the burst, far more accurate labeling of the stops should result.>

An architecture for a neural network that implements a hidden Markov model (HMM) is presented. This HMM net suggests integrating signal preprocessing (such as vector quantization) with the classifier. A minimum mean-squared-error training criterion for the HMM/neural net is presented and compared to maximum-likelihood and maximum-mutual-information criteria. The HMM forward-backward algorithm is shown to be the same as the neural net backpropagation algorithm. The implications of probability constraints on the HMM parameters are discussed. Relaxing these constraints allows negative probabilities, equivalent to inhibitory connections. A probabilistic interpretation is given for a network with negative, and even complex-valued, parameters.>

A Gaussian-model classifier trained by maximum mutual information estimation (MMIE) is compared to one trained by maximum-likelihood estimation (MLE) and to an artificial neural network (ANN) on several classification tasks. Similarity of MMIE and ANN results for uniformly distributed data confirm that the ANN is better than the MLE in some cases due to the ANNs use of an error-correcting training algorithm. When the probability model fits the data well, MLE is better than MMIE if the training data are limited, but they are equal if there are enough data. When the model is a poor fit, MMIE is better than MLE. Training dynamics of MMIE and ANN are shown to be similar under certain assumptions. MMIE seems more susceptible to overtraining and computational difficulties than the ANN. Overall, ANN is the most robust of the classifiers.>

Coarticulation effects and the need to make early decisions require real-time, connected speech recognition systems to use sophisticated final-recognition decision techniques. Attributes such as the ability to form complex decision boundaries in pattern recognition problems make neural networks attractive for performing this final-recognition decision. A combined dynamic programming/neural network approach to connected-speech recognition is evaluated in the context of recognition performance versus a dynamic programming/rule-based expert approach. Discussions of the authors' approach to neural network parameter selection, implementation, and training are included. Results for both the digits and alphadigits vocabularies are given.>

The problem of optimal implementation of the 2-D DFT (discrete Fourier transform) on a very large number of loosely coupled processors is addressed by means of a very accurate, high-level simulation model based on an existing hardware system, called Armstrong. Simulations were run for 2-D DFT sizes of 2*2 to 2048*2048. A comparison of simulation results to timings on the real Armstrong hardware, for up to 32 processors, showed differences of less than 3%. The simulation results were further validated through a simple analytic model for 2-D DFT performance. It is shown that there exists an optimum number of processors for each size DFT and that increasing the number of processors beyond this number actually decreases system performance.>

Experiments comparing artificial neural network (ANN), k-nearest-neighbor (KNN), and Bayes' rule with Gaussian distributions and maximum-likelihood estimation (BGM) classifiers were performed. Classifier error rate as a function of training set size was tested for synthetic data drawn from several different probability distributions. In cases where the true distributions were poorly modeled, ANN was significantly better than BGM. In some cases, ANN was also better than KNN. Similar experiments were performed on a voiced/unvoiced speech classification task. ANN had a lower error rate than KNN or BGM for all training set sizes, although BGM approached the ANN error rate as the training set became larger. It is concluded that there are pattern classification tasks in which an ANN is able to make better use of training data to achieve a lower error rate with a particular size training set.>

The authors show that the solutions of time-dependent difference equations represent a very general class of time-varying models for speech. Kamen's definition of time-dependent poles is introduced. Algorithms are presented which allow the coefficients of the time-dependent difference equation to be derived from the time-dependent poles and vice-versa. Parameter estimation is briefly discussed.>

Speech-source models indicate that event-synchronous feature analysis for a standard real-time speech recognizer (speaker trained using dynamic programming) should provide a strong basis for improved performance. Among the problems which need to be addressed in implementing such a system are those of developing robust event (pitch, burst, etc.) detectors, synchronous-analysis methodologies, more meaningful feature sets, and algorithms for recognition, e.g. dynamic programming. Descriptions of current approaches to these problems for a left-to-right, connected-word recognizer are given. The event-synchronous system's performance, relative to standard asynchronous recognition methodologies, is presented for an easy and a difficult vocabulary. >

An implementation of the two-dimensional (2-D) discrete Fourier transform (DFT) on the Armstrong Multiprocessor System is presented. Armstrong is a loosely coupled parallel processing system made up of a large number of identical processing nodes and is currently operational. The system utilizes a reconfigurable point-to-point network for communication, thus allowing researchers to experiment with different communication topologies for a given application. The 2-D DFT implementation is discussed, and its performance is analyzed under variations in DFT size, network topology, and processing-node count.>

A facility for evaluating a talker-dependent, connected-speech recognition system is described. It is implemented as an independent system and interacts in parallel with a recognizer in real-time. The evaluator includes software for speech acquisition and storage, connected-speech training, data transfer to a recognizer, database queries, and statistical analysis. Important considerations in the design were the human factors of recording, talker and recording-condition variability, and the embedded training paradigm. Automatic statistical analysis is derived via a simple string-alignment algorithm using just the orthography. In order to demonstrate the use of this system, two experiments are described for connected-digit recognition. These results are presented as automatlcally-generated confusion matrices for insertion, substitution and deletion error and individual string alignments.

It is often important that recognition decisions be made nearly instantaneously, i.e., early-decision, in applied connected-speech recognition systems. Therefore, many algorithms, which defer decisions until the end of an utterance, are unsuitable for these systems. This paper describes an early-decision recognizer, which has been operational at LEMS for about two years. For system-development purposes, the recognizer is programmed to reduce the data and make decisions sequentially using a multi-microprocessor implementation. While impractical for a development system, all algorithms are designed for a parallel, real-time, early-decision implementation. Connected-digit performance analysis, obtained from an automatic-evaluation system, is presented for ten male and ten female talkers. The reported evaluation has established a baseline word accuracy of 91.5%.

The analysis of speech for high-quality recognition or for coding is usually done by using a short-time interval for a time-invariant model, where local stationarity is assumed. We propose a general framework for applying a linear, time-varying model to the speech analysis problem. A key feature in our derivation is the assumption that the instantaneous frequency response of the time-varying vocal tract can be represented as a linear combination of appropriately frequency-shifted versions of a single basis function. Both time- and frequency- domain models for the speech signal are derived, Also, a statistic is derived which indicates the transient behavior of the signal and which might be used to arrive at a constant entropy signal analysis methodology. This is a first report of on going theoretical and experimental research.

This paper analyzes a particular parallel architecture which incorporates eight Texas Instruments TMS32010 digital signal processors. The fully built and operational multiprocessor system, named PT8, is described, and several common DSP algorithms are presented which illustrate its characteristics. It is shown that data handling considerations are the major factors in the performance of these algorithms. Suggestions for upgrading this architecture to one which manages data more efficiently are given.

Typically, DUR systems use a head-mounted microphone in order to keep the distance between the mouth and microphone - and therefore the input conditions - constant over time. These devices, unfortunately, also limit user-flexibility and may be quite uncomfortable. A goal of the current work is to ascertain the feasibility of a two-microphone, shoulder-mounted system. The experiment was based on data recorded simultaneously, in PCM, from the reference head-mounted microphone and the experimental system. Results based on machine DUR for two different vocabularies and two combination algorithms are presented. One very simple combination algorithm gave results which compared quite favorably to the performance of the head-mounted transducer.

The inherent complexity of a vocabulary for discrete-utterance recognition (DUR) is qualitatively easy to surmise when the vocabulary size is small. For large vocabularies a quantifying methodology is needed. We propose an operational definition and methodology for empirically deriving a measure of confusability based only on phonetic transcriptions. There are five components - 1) an appropriate (for DUR) set of alphaphonetics and a transcription data-base and methodology, 2) a phonemic distance metric, 3) a mechanism for aligning transcriptions, 4) an utterance-level, pairwise-confusion metric, and 6) a vocabulary confusion measure. These five facets are briefly described, and preliminary results for the first 700 words of the Brown Corpus are presented.

Special architectures are required for real-time response for dynamic programming for speech recognition. One approach for handling these computations, is to use a high-speed, programmable, inexpensive, attached processor. Comparison is made between the performance of the two implementations of such a processor and that of a fast, conventional microprocessor.

One possible approach to achieving talker independence in discrete utterance recognition (DUR) is to classify speech feature vectors by using a talker-independent clustering procedure. There are many possible choices of clustering algorithms. This work studied the characteristics of three clustering procedures, Agglomerative, Basic Isodata, and a 'Biased Mean' modification of Basic Isodata, as applied to speech feature vectors. The feature extractor consisted of a six channel filterbank similar to those used in DUR systems. The speech data was derived from 19 (total) repetitions of a ten word vocabulary, spoken by 16 different talkers. Various distance functions and feature vector representations were employed. Agglomerative clustering did not produce clusters which corresponded to any apparent classification of speech events. The Biased Mean Isodata procedure did not converge, and therefore was not useful. The Basic Isodata algorithm produced clusters which were to varying degrees identifiable with classes of speech sounds. Simple classifiers for three such classes, based on these clusters, would classify feature vectors with 5-10% error rates. Best results were obtained by using feature vectors which consisted of the log filter channel energies. These test results are good enough to encourage further development of cluster-based feature vector classifiers.

A program has been written which can generate banks of FIR filters using optimal-equiripple design. The composite responses of these filterbanks are very flat and have a stopband rejection of approximately 60 dB. Also, for purposes of speech recognition, energy estimates from filterbank output are frequently used, and are derived from oversampled, rectified input. A very substantial reduction in the number of calculations can be made if filter outputs are decimated while introducing a tolerable amount of error. This idea has been experimentally evaluated.

A major drawback of commercially available DUR systems is the inability of the user to easily develop and modify his applications for these devices. For example, it he wants to change the use of the recognizer from a quality control task to an inventory management application, he must either go back to the manufacturer or substantially modify all special application code. Here, concepts for what has come to be called "Automatic Application Generation" (AAG), based upon several years of experience, are presented. Simplicity to the end user, capability to produce complex application structures, and modularity are stressed. Examples are presented, and implementational considerations are discussed.

Performance evaluation of DUR systems has typically consisted of percentage-correct recognition (PCR) for specific vocabularies. This numerical measure is misleading because it presumes that 100% recognition is equally achievable for all vocabularies. This, in fact, is not the case. In this paper, results from an experiment which compared human-listener performance to that of a particular recognition machine will be presented. Three different vocabularies were studied. Preliminary results for normalizing machine performance with respect to the difficulty of a test vocabulary are given. Relevant data from the experiment are included to demonstrate the problem and its potential solution.

In the recent literature, several papers have presented experimental data dealing with performance differences in discrete utterance recognition (DUR), as a function of dynamic programming variables. The theme of the present paper is that, for the most part, these differences are insignificant when viewed in the context of real-time, full-system performance. This theme is supported by the results of several large-scale experiments in which 22 untrained talkers and 2 trained talkers produced some 2676 utterances from two different vocabularies. Results show that the significant variables are the talkers and the vocabulary items themselves; other variables demonstrate strong talker and vocabulary dependencies. As independent variables, their effects are unpredictable.

The Attached Processor for Speech (APS) has been designed to be a low-cost hardware solution for several real-time speech processing problems. Particular emphasis was placed upon the implementation of high-speed discrete utterance recognition algorithms. This paper describes APS, how it was implemented in a minimal amount of off-the-shelf hard-ware, and how it was designed to be efficient for dynamic programming algorithms. The processor executes at a rate of four million instructions per second (4 MIPS), which makes its dynamic programming thruput rate about seventeen times that of a typical conventional 16-bit microprocessor.

A new dynamic programming (DP) formulation, Using the state characteristics of the data to constrain registration, is introduced. It is evaluated in comparison to a standard DP formulation using ten untrained male and female talkers with two separate ten utterance vocabularies. One of these vocabularies was specifically designed to assess the strengths and weaknesses of the analysis system and DP formulations. Results show that the new method is clearly superior to the standard formulation with regard to memory requirements and computational load. While recognition accuracy is only slightly better for one version of the new method, the character of the error is more "sensible" and explainable. This factor is thought to be important for reject control in a discrete utterance recognition system. The parameterization of the new system also provides a better way to marry system specification with system performance.

Advances in integrated circuit technology have made real time digital filtering hardware a viable alternative to more conventional analog approaches. For example, an architecture to be described is currently being used as a dealiasing/interpolating FIR filter for speech acquisition/playback at 10kHz. The intent of the design is flexibility, simplicity, and ultimate low-cost. Flexibility and quantization accuracy are achieved by using a 16-bit microprocessor (Texas Instruments 9900). Simplicity as well as high speed is derived through the use of a hardware arithmetic unit featuring a TRW multiplier or multiplier/accumulator chip. Furthermore, synchronous programming and memory-mapped I/O are used to maximize thruput. With this combination of off-the shelf LSI devices, an FIR digital filter with a multiply-add rate of 160,000/second has been constructed. By design, only the instruction rate of the microprocessor limits the processing speed. Thus, as faster microprocessors become available, it is felt that this architecture will offer an inexpensive, digital alternative for many more signal-processing applications. In this paper, we describe the criteria for design, some of the alternatives, and the resulting hardware/software system.

This report presents results obtained in some experiments on the computer recognition of continuous speech. The experiments deal with two simple languages having vocabularies of 11 and 250 words.

The problems involved in automatic recognition of continuous speech are generally considered very difficult. In an attempt to solve these problems an acoustic processing system has been under development which has stages dedicated to signal analysis, spectral classification, phonemic segmentation, phonemic (steady state) classification, phoneme boundary placement, diadic (transitional) classification, and final phoneme string consolidation. This paper presents the concepts of and some details concerning the first five stages, through phoneme boundary placement. Performance results are also presented.

The Modular Acoustic Processor (MAP), a complex experimental system for automatic derivation of phonemic string output for continuous speech, was first described in April 1974. Many of the new concepts currently in MAP are described. In particular, the problems concerning the diadic segment classification and final string estimation are discussed, and the current solutions given. Results on a large body of continuous speech data, prepared by an automatic evaluation system will also be presented.

The problem considered is how to place records on a secondary storage device to minimize average retrieval time, based on a knowledge of the probability for accessing the records. Theorems are presented for two limiting cases. A numerical example for an intermediate case is also given.

The automatic determination of local similarity between two structured data sets is fundamental to the disciplines of pattern recognition and image processing. A class of algorithms, which may be used to determine similarity in a far more efficient manner than methods currently in use, is introduced in this paper. There may be a saving of computation time of two orders of magnitude or more by adopting this new approach. The problem of translational image registration, used for an example throughout, is discussed and the problems with the most widely used method-correlation explained. Simple implementations of the new algorithms are introduced to motivate the basic idea of their structure. Real data from ITOS-1 satellites are presented to give meaningful empirical justification for theoretical predictions.