A new face recognition algorithm is introduced that is based on finding an optimal mapping between two images and measuring the properties of the mapping to determine the similarity between the two images. This new method is shown to be extremely robust to variations in pose, camera position and lighting for the problem of face recognition especially in an uncontrolled environment such as video surveillance. The technique does not depend on locating or training on facial features and can be applied in general to any object recognition problem. We present recognition results on current databases available to the research community and will demonstrate a real-time version of the software for access control and surveillance applications at the conference.

Introduces the non-negative matrix factorization for language model adaptation. This approach is an alternative to latent semantic analysis based language modeling using singular value decomposition with several benefits. A new method, which does not require an explicit document segmentation of the training corpus is presented as well. This method resulted in a perplexity reduction of 16% on a database of biology lecture transcriptions.

The performance of automatic speaker recognition systems is significantly degraded by acoustic mismatches between training and testing conditions. Such acoustic mismatches are commonly encountered in systems that operate on speech collected over telephone networks, where different handsets and different network routes impose varying convolutional distortions on the speech signal. A new algorithm, the modified-mean cepstral mean normalization with frequency warping (MMCMNFW) method, which improves upon the commonly-employed cepstral mean subtraction method, has been developed. Experimental results on closed-set speaker identification tasks on a channel-corrupted subset of the TIMIT database and on a subset of the NTIMIT database are presented. The new algorithm is shown to offer improved recognition rates over other existing channel normalization methods on these databases.

The performance of speech systems such as speaker recognition degrades drastically when there is mismatch between training and testing conditions, caused by non-linear distortion. This paper describes a technique to estimate and correct such non-linear distortion in speech. The focus is on constrained restoration of degraded speech, that is distortion in the test speech is undone relative to the training speech. Restoration is a two step process-estimation followed by inversion. The non-linearity is estimated in the form of a look-up table by a process of statistical matching using a reference speech template. This statistical matching technique provides a very good estimate of the true non-linear characteristic, and the process is robust, computationally efficient, and universally applicable. Speaker-ID experiments, using artificially corrupted test speech, showed significant improvement in performance after the test speech was 'cleaned' using this technique. The restoration process itself does not introduce appreciable distortion.

We analyse the diversity of information as provided by several modeling approaches for speaker verification. This information is used to facilitate the fusion of the individual results into an overall result that provides advantages in accuracy over the individual models. The modeling methods that are evaluated consist of the neural tree network (NTN), Gaussian mixture model (GMM), hidden Markov model (HMM), and dynamic time warping (DTW). With the exception of DTW, all methods utilize subword-based approaches. The phrase-level scores for each modeling approach are used for combination. Several data fusion methods are evaluated for combining the model results, including the linear and log opinion pool approaches along with voting. The results of the above analysis have been integrated into a system that has been tested with several databases collected within landline and cellular environments. We have found the linear and log opinion pool methods to consistently reduce the error rate from that obtained when the models are need individually.

A common problem in speaker identification systems is that a mismatch in the training and testing conditions sacrifices much performance. We attempt to alleviate this problem by proposing new features that show less variation when speech is corrupted by convolutional noise (channel) and/or additive noise. The conventional feature used is the linear predictive (LP) cepstrum that is derived from an all-pole transfer function which, in turn, achieves a good approximation to the spectral envelope of the speech. A different cepstral feature based on a pole-zero function (called the adaptive component weighted or ACW cepstrum) was previously introduced. We propose four additional new cepstral features based on pole-zero transfer functions. One is an alternative way of doing adaptive component weighting and is called the ACW2 cepstrum. Two others (known as the PFL1 cepstrum and the PFL2 cepstrum) are based on a pole-zero postfilter used in speech enhancement. Finally, an autoregressive moving-average (ARMA) analysis of speech results in a pole-zero transfer function describing the spectral envelope. The cepstrum of this transfer function is the feature. Experiments involving a closed set, text-independent and vector quantizer based speaker identification system are done to compare the various features. The TIMIT and King databases are used. The ACW and PFL1 features are the preferred features, since they do as well or better than the LP cepstrum for all the test conditions. The corresponding spectra show a clear emphasis of the formants and no spectral tilt.

In speaker recognition systems, the adaptive component weighted (ACW) cepstrum has been shown to be more robust than the conventional linear predictive (LP) cepstrum. The ACW cepstrum is derived from a pole-zero transfer function whose denominator is the pth-order LP polynomial A(z). The numerator is a (p-1)th-order polynomial that is up to now found as follows. The roots of A(z) are computed, and the corresponding residues obtained by a partial fraction expansion of 1/A(z) are set to unity. Therefore, the numerator is the sum of all the (p-1)th-order cofactors of A(z). We show that the numerator polynomial is merely the derivative of the denominator polynomial A(z). This greatly speeds up the computation of the numerator polynomial coefficients since it involves a simple scaling of the denominator polynomial coefficients. Root finding is completely eliminated. Since the denominator is guaranteed to be minimum phase and the numerator can be proven to be minimum phase, two separate recursions involving the polynomial coefficients establishes the ACW cepstrum. This new method, which avoids root finding, reduces the computer time significantly and imposes negligible overhead when compared with the approach of finding the LP cepstrum.

A subword-based, text-dependent speaker verification system that embodies the capability of user-selectable passwords (ideally, with no constraints on the choice of vocabulary words or the language) is presented. A novel automatic speech segmentation procedure, called the "blind" segmentation, is also introduced. This algorithm segments speech without any linguistic knowledge and makes the subword modeling of the user-selectable password realizable in the proposed system. The subword modeling is done using the neural tree networks (NTNs). The NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward neural networks. The proposed system also takes advantage of such concepts as the multiple classifier fusion and data resampling to successfully boost the system performance.

A new classifier is described that combines the discriminatory ability of the neural tree network (NTN) with the Gaussian mixture model to create a continuous density neural tree network (CDNTN). The CDNTN is used within a hidden Markov model (HMM), along with a nonparametric state duration model to construct a continuous word spotting system for real time applications. The new word spotting system does not use a general background model, allowing construction of independent models whose performance is independent of the number of models in the recognition system, supporting a direct parallel implementation. Although HMM word spotting systems are shown to provide good performance when sufficient training data is available, for applications where background speech data is not available or only a limited numbers of training tokens are available, the CDNTN word spotting system is shown to out perform comparable HMM systems.

A new algorithm for text-dependent speaker verification is presented. The algorithm uses a set of concatenated neural tree networks (NTNs) trained with subword units for speaker verification. The conventional NTN has been found to provide good performance in text-independent tasks. In the new approach, two types of subword unit are investigated, phone-like units (PLUs) and HMM state-based units (HSU's). The training of the models includes several steps. First, the predetermined password in the training data is segmented into subword units using a hidden Markov model (HMM) based segmentation method. Second, an NTN is trained for each subword unit. The new structure integrates the discriminatory ability of the NTN with the temporal models of the HMM. This new algorithm was evaluated by experiments on a TI isolated-word database, and YOHO database. An improvement of performance was observed over the performance obtained using a conventional HMM.

A new neural tree network (NTN)-based speech recognition system is presented. NTN is a hierarchial classifier that combines the properties of decision trees and feed-forward neural networks. In the sub-word unit-based system, the NTNs model the sub-word speech segments, while the Viterbi algorithm is used for temporal alignment. Durational probability is associated with each sub-word NTN. An iterative algorithm is proposed for training the sub-word NTNs. The sub-word NTN models, as well as the subword segment boundaries within a vocabulary word, are re-estimated. Thus, the proposed system is a homogeneous neural network-based, sub-word unit-based, speech recognition system. Furthermore, embedded within this word model paradigm, multiple NTNs are trained for each subword segment and their output decisions are combined or fused to yield improved performance. The proposed discriminatory training-based system did not perform favourably as compared to a hidden Markov model-based system. The paradigm presented in this paper can be argued to represent a class of discriminatory training-based, homogeneous (versus hybrid), sub-word unit-based, speech recognition systems. Hence, the results reported here can be generalized to other similar systems.

Various linear predictive (LP) analysis methods are studied and compared from the points of view of robustness to noise and of application to speaker identification. The key to the success of the LP techniques is in separating the vocal tract information from the pitch information present in a speech signal even under noisy conditions. In addition to considering the conventional, one-shot weighted least-squares methods, the authors propose three other approaches with the above point as a motivation. The first is an iterative approach that leads to the weighted least absolute value solution. The second is an extension of the one-shot least-squares approach and achieves an iterative update of the weights. The update is a function of the residual and is based on minimizing a Mahalanobis distance. Third, the weighted total least-squares formulation is considered. A study of the deviations in the LP parameters is done when noise (white Gaussian and impulsive) is added to the speech. It is revealed that the most robust method depends on the type of noise. Closed-set speaker identification experiments with 20 speakers are conducted using a vector quantizer classifier trained on clean speech. The relative performance of the various LP approaches depends on the type of speech material used for testing.>

In this paper we introduce a new set of features that provides improved performance for speaker identification. This feature set is referred to as the adaptive component weighting (ACW) cepstral coefficients. The ACW scheme modifies the linear predictive (LP) spectral components (resonances) so as to emphasize the formant structure by attenuating the broad-bandwidth spectral components. Such components are found to introduce undesired variability in the LP spectra of speech signals due to environmental factors. The ACW cepstral coefficients represent an adaptively weighted version of the LP cepstrum. The adaptation results in deemphasizing the irrelevant variations of the LP cepstral coefficients on a frame-by-frame basis. Experiments are presented using the San Diego portion of the King database. The ACW cepstrum is shown to offer improved speaker identification performance as compared to other common methods of cepstral weighting.>

A modified neural tree network (NTN) is examined for use in text independent speaker identification. The NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward neural networks. The modified NTN uses discriminant learning to partition feature space as opposed to the more common clustering approaches, such as vector quantization. The modified NTN also uses forward pruning to avoid overfitting the training data. The modified NTN is evaluated for both closed and open set speaker identification experiments using the TIMIT database. The performance of the modified NTN is compared to that of vector quantization classifiers. The results presented show the modified NTN to provide comparable performance to the vector quantization classifier for closed set speaker identification while providing improved performance for the open set problem.>

Speaker identification (SI) and speech recognition systems display a strong sensitivity to ambient noise conditions. It is known that the performance of these systems is maximized when trained under conditions that approximate the testing environment. However these systems generally do not perform well when there is no noise or when the level or characteristics of the noise changes. A new speech enhancement technique is presented that, when used with a standard cepstrum-based SI system, significantly reduces the sensitivity of the SI system to mismatches in the noise environment between training and testing.>

A new set of features is introduced that has been found to improve the performance of automatic speaker identification systems, The new set of features is referred to as the adaptive component weighting (ACW) cepstral coefficients. The new features emphasize the formant structure of the speech spectrum while attenuating the broad-bandwidth spectral components. The attenuated components correspond to the variations in spectral tilt of transmission and recording environment, and other characteristics that are irrelevant to speaker identification. The resulting ACW spectrum introduces zeros into the usual all-pole linear prediction (LP) spectrum. This is equivalent to applying a finite impulse response (FIR) filter that normalizes the narrow-band modes of the spectrum. Unlike existing fixed cepstral weighting schemes, the ACW cepstrum provides an adaptively weighted version of the LP cepstrum. The adaptation results in deemphasizing the irrelevant variations of the LP cepstral coefficients on a frame-by-frame basis. The ACW features are evaluated for text-independent speaker identification and are shown to yield improved performance.>

An evaluation of various classifiers for text-independent speaker recognition is presented. In addition, a new classifier is examined for this application. The new classifier is called the modified neural tree network (MNTN). The MNTN is a hierarchical classifier that combines the properties of decision trees and feedforward neural networks. The MNTN differs from the standard NTN in both the new learning rule used and the pruning criteria. The MNTN is evaluated for several speaker recognition experiments. These include closed- and open-set speaker identification and speaker verification. The database used is a subset of the TIMIT database consisting of 38 speakers from the same dialect region. The MNTN is compared with nearest neighbor classifiers, full-search, and tree-structured vector quantization (VQ) classifiers, multilayer perceptrons (MLPs), and decision trees. For closed-set speaker identification experiments, the full-search VQ classifier and MNTN demonstrate comparable performance. Both methods perform significantly better than the other classifiers for this task. The MNTN and full-search VQ classifiers are also compared for several speaker verification and open-set speaker-identification experiments. The MNTN is found to perform better than full-search VQ classifiers for both of these applications. In addition to matching or exceeding the performance of the VQ classifier for these applications, the MNTN also provides a logarithmic saving for retrieval.

The goal of this special issue is to present a representative set of current research papers that address the topic of neural networks in speech processing. The application areas addressed include speech analysis, synthesis, recognition and understanding. Due to the large volume of current research in these areas, the authors make the usual disclaimer and apologize for any work that may not be represented in the limited space allocated here. However, the included papers provide a fairly broad overview of each area, as well as citations to the related literature.>

A pattern classification method called neural tree networks (NTNs) is presented. The NTN consists of neural networks connected in a tree architecture. The neural networks are used to recursively partition the feature space into subregions. Each terminal subregion is assigned a class label which depends on the training data routed to it by the neural networks. The NTN is grown by a learning algorithm, as opposed to multilayer perceptrons (MLPs), where the architecture must be specified before learning can begin. A heuristic learning algorithm based on minimizing the L1 norm of the error is used to grow the NTN. It is shown that this method has better performance in terms of minimizing the number of classification errors than the squared error minimization method used in backpropagation. An optimal pruning algorithm is given to enhance the generalization of the NTN. Simulation results are presented on Boolean function learning tasks and a speaker independent vowel recognition task. The NTN compares favorably to both neural networks and decision trees.>

It is shown that beamforming with microphone arrays can be used to significantly reduce the effects of noise and reverberation. The Frost algorithm and a new vectorized algorithm called the row-action projection (RAP) method are shown to yield approximately 11 dB of interference suppression. Computer simulations demonstrate the performance of the beamforming techniques for various locations of interference sources. The scenarios used for the computer simulations consist of a speech signal arriving normal to the array contaminated with speech interference from an unknown direction off the normal. The direction of the desired speaker is taken to the normal to the array for the simulation; however, it could be at any specified 'look direction'.>

A novel method of characterizing speech signals is presented in which the signal is modeled in terms of the mean radian frequency, the amplitude modulating function and the frequency modulating function of each frequency domain component (i.e., formants). This model defines the component structure and represents the local characteristics of the component and its bandwidth. A vowel classification task is presented where a distance metric of the relative modulation functions is used. This technique has been found to be robust to inter- and intraspeaker variability.>

An adaptive technique which determines the spectral coefficients of a time-varying signal is presented. The algorithm performs a least-squares decomposition of the signal onto a high-resolution nonharmonic Fourier basis. The spectral estimate is updated sample by sample and produces a decomposition with very good localization in both the time and frequency domains. This yields a distribution of signal intensities in the time-frequency plane which tracks the instantaneous frequency and local bandwidth of the signal. The detection of tones spaced closer than expected by the uncertainty distance (super-resolution) is shown by computer simulation. Numerical results demonstrate the super-resolving capability and fast tracking rate as well as the stability in the presence of additive noise. Potential applications of this method include tracking of frequency hopping communications, FM demodulation, and formant tracking in speech signals.>

A new iterative algorithm for image restoration and point spread function (PSF) estimation is presented. The method initially restores the image with the expectation maximization (EM) algorithm. The resulting estimate is then significantly improved by using the adaptive row action projection (RAP) algorithm which is based on the theory of projection onto convex sets (POCS). The PSF is re-estimated using a least squares technique. Computer simulations show the method's competitiveness in restoring degraded images and estimating the PSF from noisy blurred images with unknown PSF.>

A fast, stable adaptive algorithm, the row-action projection (RAP) algorithm, that is amenable to a wide range of signal processing applications is presented. Many problems in signal processing can be formulated as a regression parameter estimation problem, yielding a system of linear equations which is generally inconsistent and ill-conditioned. Problems formulated in this way include autoregressive speech modeling, spectrum estimation, array processing, and channel equalization. The algorithm iteratively estimates the unknown coefficients by using each equation of the system more than once. Simulation results are presented that show the stable performance of the RAP algorithm.>

The application of row action projections (RAPs) to the problem of acoustic echo cancellation is discussed. It is shown how RAPs, a subset of projections onto convex sets (POCSs) and a generalization of LMS, can be used to further improve the tracking ability of subband acoustic echo canceller (SBAECs). In addition, the RAP algorithm has a useful complexity versus speed-of-convergence tradeoff. That is, RAP can use additional computational resources to speed convergence. This is used to dynamically allocate computational resources among the subband in such a way as to improve the perceptual performance of the SBAEC. The feasibility of this method is demonstrated by sizing it for AT&T's WE DSP16A digital signal processor.>

The recovery of depth from defocused images is formulated as a 3-D image restoration problem. A defocused image is modeled as the combinatorial outcome of the depths and intensities of the volume elements (voxels) of an opaque 3-D object. A large depth-of-field image is used to constrain the intensities of the voxels. The depths of voxels are estimated from a highly defocused image by using simulated annealing to solve a constrained optimization problem. It is concluded that the method provides a framework for high-resolution depth recovery from defocused images. The method is computationally-intensive; however, it is amenable to parallel processing and is well suited for small field-of-interest applications.>

A method of encoding monochrome video sequences is presented. This method is applicable to slow-moving scenes with a zoom factor close to unity. It incorporates a perceptual model for the detection and tracking of image blocks perceived to be in apparent motion. The perceptual model is a threshold model based on contrast. Weber fractions in the spatiotemporal domain are used to set the threshold for the detection of motion. The threshold is determined on a scene-adaptive basis. The information content of the scene is derived from the spectral distribution of image data and the masking characteristics of the human visual system. The perceptual model is used in the block-classification process, which in turn determines the coding strategy used for each block type. The discrete cosine transform is used for encoding image blocks which have changed beyond a perceptual threshold. A comparison of the perceptually encoded image sequence with that using no perceptual model indicates substantially improved quality and performance of the perceptual encoder for compression ratios up to 100:1.>

In this paper we introduce a new method of determining the multi-pulse excitation used in multipulse LPC introduced by Atal and Remde [1]. The new method finds that multi-pulse excitation sequence whose short time Fourier spectrum is the closest to the spectrum of the residual from LPC analysis. A Linear Programming method is used to determine the optimal pulse locations and pulse amplitudes. In this preliminary study, experimental results have shown that this Linear Programming technique provides an excitation sequence with a better short time Fourier spectrum and is more robust to environment noise than previous techniques.

In this paper a new approach to spectral analysis is presented. The method consists of two stages of processing. The first stage is used to reduce the variance of the noise. This stage of processing corresponds to many standard spectral estimation processes which involve averaging of one kind or another. This averaging stage is realized in this paper by using an adaptive line enhancer (ALE). The output of the ALE is a spectral estimate with reduced noise variance but a coarse frequency resolution. The second stage of processing is used to improve the frequency resolution of this coarse spectral estimate. The frequency resolution enhancement process is accomplished by a template matching method. The method uses Linear Programming to search for the best fit of(\sin f)/ffunctions to the coarse spectral estimate. The position and scale factors of the(\sin f)/ffunctions then yield the frequencies and amplitudes of the underlying sinusoids. The numerical results indicate that this second stage of processing gives a significantly better spectral estimate in terms of frequency accuracy (one tone) and resolution (two tone).

A new method is presented for low bit encoding. The approach taken is similar to that of LPC analysis in that an autoregressive model of the vocal tract is used. The estimate of the model is made using a least absolute value (LAV) criteria. That is the sum of the absolute values (L1norm) of the error is minimized. This approach is compared with the usual least squares (L2norm) methods i.e. covariance and autocorrelation [1]. The minimum L1estimate was obtained using the simplex method of linear programming [2,3]. It is well known that the L1norm estimate is highly robust to statistical outliers [4]. The robust nature of the minimum L1algorithm can be interrupted as an expectation that the residual will consist of a series of impulses. This expectation is of course valid for speech as demonstrated by the improved performance provided by multipulse LPC [5]. One of the disadvantages of multipulse LPC is the need for windowing [6]. This bandwidth doubling is not present in the L1norm method.