• Analysis of speech signals for automatic estimation of the language spoken. • Automatic speech recognition, speaker verification, and language identification are compared. • What distinguishes different spoken languages is discussed. • Utility of methods is noted in terms of performance, using accuracy, complexity, and cost as measures. • Approaches include: phonotactics, use of intonation, mel-frequency cepstral coefficients, neural networks. • Major components of neural systems (CNN, RNN, Transformer) are discussed. Identification of the language used in spoken utterances is useful for multiple applications, e.g., assist in directing or automating telephone calls, or selecting which language-specific speech recognizer to use. This paper reviews modern methods of automatic language identification. It examines what information in speech helps to distinguish among languages, and extends these ideas to dialect estimation as well. As approaches to recognize languages often share much in common with both automatic speech recognition and speaker verification, these three processes are compared. Many methods are drawn from pattern recognition research in other areas, such as image and text recognition. This paper notes how speech is different from most other signals to recognize, and how language identification differs from other speech applications. While it is mainly addressed to readers who are not experts in speech processing (as detailed algorithms, readily found in the cited literature, are omitted here), the presentation covers a wide discussion useful to experts too.

A review of techniques to identify speakers from their voices is presented, noting strengths and weaknesses of various methods. Similar acoustic analysis has been often used for both speech and speaker recognition, despite the two tasks being quite different. Speaker biometrics from voice is far more indirect and subtle than the estimation of phoneme sequences for automatic speech recognition from periodic evaluations of the spectral envelope of the vocal tract output. Speech signals are discussed from the point of view of how to recognize their textual content versus estimating other aspects of speakers. Common speech analysis methods such as filter banks, linear prediction, and mel-frequency cepstrum are examined. Approaches such as hidden Markov models, i-vectors, and artificial neural networks are shown to be useful for multiple speech applications. Focus is on how various types of networks can accomplish automatic speaker verification (ASV). Suggestions to improve these methods are made.

A review of techniques to improve distorted speech is presented, noting the strengths and weaknesses of common methods. Speech signals are discussed from the point of view of which features should be preserved to retain both naturalness and intelligibility. Enhancement methods range from classical spectral subtraction and Wiener filtering to recent deep neural network approaches. The difficulty of finding objective acoustic measures that approximate perceptual speech quality is discussed. Suggestions to improve these methods are made.

Interest in affective computing is burgeoning, in great part due to its role in emerging affective human-computer interfaces (HCI). To date, the majority of existing research on automated emotion analysis has relied on data collected in controlled environments. With the rise of HCI applications on mobile devices, however, so-called “in-the-wild” settings have posed a serious threat for emotion recognition systems, particularly those based on voice. In this case, environmental factors such as ambient noise and reverberation severely hamper system performance. In this paper, we quantify the detrimental effects that the environment has on emotion recognition and explore the benefits achievable with speech enhancement. Moreover, we propose a modulation spectral feature pooling scheme that is shown to outperform a state-of-the-art benchmark system for environment-robust prediction of spontaneous arousal and valence emotional primitives. Experiments on an environment-corrupted version of the RECOLA dataset of spontaneous interactions show the proposed feature pooling scheme, combined with speech enhancement, outperforming the benchmark across different noise-only, reverberation-only and noise-plus-reverberation conditions. Additional tests with the SEWA database show the benefits of the proposed method for in-the-wild applications.

In this paper, the i-vector framework is investigated as an intrusive quality measure for noisy and enhanced speech. While widely used across numerous speech applications, the potential of using i-vectors to summarize the quality of a speech recording has been overlooked. This paper aims to fill this gap. We show that the i-vector framework is well-suited for assessing speech signal quality, surpassing well-established instrumental measures such as the Perceptual Evaluation of Speech Quality (PESQ) and Perceptual Objective Listening Quality Analysis (POLQA). Three datasets are used in our experiments. First, the TIMIT database is used to train the i-vector extractor on clean speech. To evaluate the proposed method, the noisy speech corpus (NOIZEUS) and the evaluation set of the 2014 IEEE REVERB challenge are used with both containing subjective ratings of perceived quality. Correlations with mean opinion scores (MOS) as high as 0.90 are achieved.

In this paper, automatic speaker verification using normal and whispered speech is explored. Typically, for speaker verification systems with varying vocal effort inputs, standard solutions such as feature mapping or addition of data during parameter estimation (training) and enrollment stages result in a trade-off between accuracy gains with whispered test data and accuracy losses (up to 70% in equal error rate, EER) with normal test data. To overcome this shortcoming, this paper proposes two innovations. First, we show the complementarity of features derived from AM-FM models over conventional mel-frequency cepstral coefficients, thus signalling the importance of instantaneous phase information for whispered speech speaker verification. Next, two fusion schemes are explored: score- and feature-level fusion. Overall, we show that gains as high as 30% and 84% in EER can be achieved for normal and whispered speech, respectively, using featurelevel fusion.

In this paper, we introduce a document-specific context probabilistic latent semantic analysis (DCPLSA) model for speech recognition. This is an extension of a CPLSA model [1] where the probability of word is conditioned only on topics. The CPLSA model uses the bigram counts that are the number of appearances of the bigrams in the corpus. These counts are the sum of the bigram counts in different documents where they could appear to describe different topics. We encounter this problem in the CPLSA model and introduce the document-specific CPLSA model (DCPLSA) where the probability of a word is conditioned on both topic and document. We carried out experiments on a continuous speech recognition (CSR) task using the Wall Street Journal (WSJ) corpus and have seen that the proposed DCPLSA approach yields significant reduction in both perplexity and word error rate (WER) measurements over the other approaches used in the literature.

In this paper, we introduce a novel method of smoothing language models (LM) based on the semantic information found in ontologies that is especially adapted for limited-resources language modeling. We exploit the latent knowledge of language that is deeply encoded within ontologies. As such, this work examines the potential of using the semantic and syntactic relations between words from the WordNet ontology to generate new plausible contexts for unseen events to simulate a larger corpus. These unseen events are then mixed-up with a baseline Witten-Bell(WB) LM in order to improve its performance both in terms of language model perplexity and automatic speech recognition word error rates. Results indicate a significant reduction in the perplexity of the language model (up to 9.85% relative) all the while reducing word error rate in a statistically significant manner compared to both the original WB LM and baseline Kneser-Ney smoothed language model on the Wall Street Journal-based Continuous Speech Recognition Phase II corpus.

This work presents a noise spectrum estimator based on the Gaussian mixture model (GMM)-based speech presence probability (SPP) for robust speech recognition. Estimated noise spectrum is then used to compute a subband a posteriori signal-to-noise ratio (SNR). A sigmoid shape weighting rule is formed based on this subband a posteriori SNR to enhance the speech spectrum in the auditory domain, which is used in the Mel-frequency cepstral coefficient (MFCC) framework for robust feature, denoted here as Robust MFCC (RMFCC) extraction. The performance of the GMM-SPP noise spectrum estimator-based RMFCC feature extractor is evaluated in the context of speech recognition on the AURORA-4 continuous speech recognition task. For comparison we incorporate six existing noise estimation methods into this auditory domain spectrum enhancement framework. The ETSI advanced frontend (ETSI-AFE), power normalized cepstral coefficients (PNCC), and robust compressive gammachirp cepstral coefficients (RCGCC) are also considered for comparison purposes. Experimental speech recognition results show that, in terms of word accuracy, RMFCC provides an average relative improvements of 8.1%, 6.9% and 6.6% over RCGCC, ETSI-AFE, and PNCC, respectively. With GMM-SPP -based noise estimation method an average relative improvement of 3.6% is obtained over other six noise estimation methods in terms of word recognition accuracy.

We propose a document-based Dirichlet class language model (DDCLM) for speech recognition using document-based n-gram events. In this model, the class is conditioned on the immediate history context and the document, and the word is conditioned on the the class and the document in the original DCLM model. In the DCLM model, the class information was obtained from the (n-1) history words of n-gram events of a training corpus. Here, the model uses the count of the n-grams, which are the number of appearances of the n-grams in the corpus. These counts are the sum of the n-gram counts in different documents where they could appear to describe different topics. Therefore, the n-gram counts of the corpus may not yield the proper class information for the histories. We encounter this problem in the DCLM model and propose a DDCLM model that overcomes the above problem by finding the class information for the document-based history context using the document-based n-gram events. We carried out experiments on a continuous speech recognition (CSR) task using the Wall Street Journal (WSJ) corpus and have seen that the proposed approach shows significant perplexity and word error rate (WER) reductions over the other approach.

This paper presents regularized minimum variance distortion-less response (MVDR)-based cepstral features for robust continuous speech recognition. The mel-frequency cepstral coefficient (MFCC) features, widely used in speech recognition tasks, are usually computed from a direct spectrum estimate, that is, the squared magnitude of the discrete Fourier transform (DFT) of speech frames. Direct spectrum estimation methods (also known as nonparametric estimators) perform poorly under noisy and adverse conditions. To reduce this performance drop we propose to increase robustness of the speech recognition system by extracting more robust features based on the regularized MVDR technique. The proposed method, when evaluated on the AURORA-4 speech recognition task, provides an average relative improvement in word accuracy of 11.3%, 6.1%, and 5.2% over the conventional MFCC, PLP, MVDR and PMVDR-based MFCC features, respectively.

Subjective speech quality assessment depends on listener “quality” opinions after hearing a particular test speech stimulus. Subjective scores are given based on a perception and quality judgment process that is unique to a particular listener. These processes are postulated to be dependent on the listener's internal reference of what good and bad quality sounds like, as well as their mental and emotional states. To overcome this variability, subjective listening tests often average scores over several listeners. In this paper, we use electroencephalography (EEG) and self-assessment tools to investigate the neural and affective correlates of speech quality perception of reverberant speech, with the goal of obtaining new insights into human speech quality perception in complex listening environments. We show that EEG event related potentials (ERP) are a useful tool to monitor the conscious stages of neural-processing during a speech quality assessment task. Significant correlations were obtained between the so-called P300 ERP component and the reverberation time of the room, as well as between the P300 peak amplitude and emotional self-assessment ratings. These insights could lead to more effective ways of characterizing room acoustics for improved speech quality and intelligibility.

MFCC (Mel Frequency Cepstral Coefficients) and PLP (Perceptual linear prediction coefficients) or RASTA-PLP have demonstrated good results whether when they are used in combination with prosodic features as suprasegmental (long-term) information or when used stand-alone as segmental (short-time) information. MFCC and PLP feature parameterization aims to represent the speech parameters in a way similar to how sound is perceived by humans. However, MFCC and PLP are usually computed from a Hamming-windowed periodogram spectrum estimate that is characterized by large variance. In this paper we study the effect of averaging spectral estimates obtained using a set of orthogonal tapers (windows) on emotion recognition performance. The multitaper MFCC and PLP are examined separately as short-time information vectors modeled using Gaussian mixture models (GMMs). When tested on the FAU AIBO spontaneous emotion corpus, a relative improvement ranging from 2.2% to 3.9% for both MFCC and PLP systems is achieved by multiple windowed spectral features compared to single windowed ones.

We propose a novel context-based probabilistic latent semantic analysis (PLSA) language model for speech recognition. In this model, the topic is conditioned on the immediate history context and the document in the original PLSA model. This allows computing all the possible bigram probabilities of the seen history context using the model. It properly computes the topic probability of an unseen document for each history context present in the document. We compare our approach with a recently proposed unsmoothed bigram PLSA model where only the seen bigram probabilities are calculated, which causes computing the incorrect topic probability for the present history context of the unseen document. The proposed model requires a significantly less amount of computation time and memory space requirements than the unsmoothed bigram PLSA model. We carried out experiments on a continuous speech recognition (CSR) task using theWall Street Journal (WSJ) corpus. The proposed approach shows significant reduction in both perplexity and word error rate (WER) measurements over the other approach.

In this paper, automatic speaker verification and gender detection using whispered speech is explored. Whispered speech, despite its reduced perceptibility, has been shown to convey relevant speaker identity and gender information. This study compares the performance of a GMM-UBM speaker verification system trained with normal and whispered speech under different matched and mismatched conditions, and describes the benefits of adaptation in a speaking-style independent model to handle both vocal efforts. It is shown that performance improvements can be achieved by using speaking-style and gender dependent models, as well as by adding features based on the AM-FM signal representation. Moreover, the AM-FM based features showed to be more discriminative than classical MFCCs for whispered speech gender detection. Experimental results suggest that whispered speech carries sufficient information for reliable automatic speaker identification.

The goal of speech emotion recognition (SER) is to identify the emotional or physical state of a human being from his or her voice. One of the most important things in a SER task is to extract and select relevant speech features with which most emotions could be recognized. In this paper, we present a smoothed nonlinear energy operator (SNEO)-based amplitude modulation cepstral coefficients (AMCC) feature for recognizing emotions from speech signals. SNEO estimates the energy required to produce the AM-FM signal, and then the estimated energy is separated into its amplitude and frequency components using an energy separation algorithm (ESA). AMCC features are obtained by first decomposing a speech signal using a C-channel gammatone filterbank, computing the AM power spectrum, and taking a discrete cosine transform (DCT) of the root compressed AM power spectrum. Conventional MFCC (Mel-frequency cepstral coefficients) and Mel-warped DFT (discrete Fourier transform) spectrum based cepstral coefficients (MWDCC) features are used for comparing the recognition performances of the proposed features. Emotion recognition experiments are conducted on the FAU AIBO spontaneous emotion corpus. It is observed from the experimental results that the AMCC features provide a relative improvement of approximately 3.5% over the baseline MFCC.

In this paper, we present two robust feature extractors that use a regularized minimum variance distortionless response (RMVDR) spectrum estimator instead of the discrete Fourier transform-based direct spectrum estimator, used in many front-ends including the conventional MFCC, for estimating the speech power spectrum. Direct spectrum estimators, e.g., single tapered periodogram, have high variance and they perform poorly under noisy and adverse conditions. RMVDR spectrum estimator has low spectral variance and are robust to mismatch conditions. Based on RMVDR spectrum estimator two robust feature extractors, robust RMVDR cepstral coefficients (RRMCC) and normalized RMVDR cepstral coefficients (NRMCC), are proposed that incorporate an auditory domain spectrum enhancement (ASE) method and a medium duration power bias subtraction (MDPBS) technique, respectively, for enhancement of the speech spectrum. Experimental speech recognition results are conducted on the AURORA-4 corpus and performances are compared with the MFCC, PLP, MVDR-MFCC, RMVDR-MFCC, PMVDR, ETSI advancement front-end (ETSI-AFE), PNCC, CFCC, and the robust feature extractor (RFE) of [6]. Experimental results demonstrate that the proposed robust feature extractors outperformed the other robust front-ends in terms of percentage word accuracy on the AURORA-4 large vocabulary continuous speech recognition (LVCSR) task under different mismatch conditions.

Accuracy of speaker verification is high under controlled condi-tions but falls off rapidly in the presence of interfering sounds. This is because spectral features, such as Mel-frequency cep-stral coefficients (MFCCs), are sensitive to additive noise. MFCCs are a particular realization of warped-frequency rep-resentation with low-frequency focus. But there are several alternative, potentially more robust, warped-frequency repre-sentations. We provide an experimental comparison of five warped-frequency features. They use exactly the same fre-quency warping function, the same number of coefficients and postprocessing, but differ in their internal computations. The compared variants are (1) conventional MFCCs from discrete Fourier transform (DFT), followed by Mel-scaled filterbank, (2) MFCCs via direct warping of DFT, followed by linear-scale fil-terbank, (3) warped linear prediction features, (4) perceptual minimum variance distortionless features and (5) recently pro-posed sparse Mel-scale histogram features. Experiments car-ried out on a subset of the SRE 10 corpus using a scaled-down i-vector system indicate that direct DFT warping outperforms conventional MFCCs in most of the cases. Index Terms: speaker recognition, noise, frequency warping 1.

As a pattern recognition application, automatic speech recognition (ASR) requires the extraction of useful features from its input signal, speech. To help determine relevance, human speech production and acoustic aspects of speech perception are reviewed, to identify acoustic elements likely to be most important for ASR. Common methods of estimating useful aspects of speech spectral envelopes are reviewed, from the point of view of efficiency and reliability in mismatched conditions. Because many speech inputs for ASR have noise and channel degradations, ways to improve robustness in speech parameterization are analyzed. While the main focus in ASR is to obtain spectral envelope measures, human speech communication efficiently exploits the manipulation of one's vocal-cord vibration rate [fundamental frequency (F0)], and so F0 extraction and its integration into ASR are also reviewed. For the acoustic analysis reviewed here for ASR, this work presents modern methods as well as future perspectives on important aspects of speech information processing.

The articles in this special issue are devoted to the topic of speech information processing, key technologicial and systems theories, and applications for its use.

This paper presents a noise tracking and estimation algorithm for highly non-stationary noises using the Bayesian on-line spectral change point detection (BOSCPD) technique. In BOSCPD, the local minima search window update technique of minima controlled recursive averaging (MCRA) algorithm is made a function of spectral change point detection. The novelty of this algorithm is that it can detect the rapid changes instantly and quickly update the non-stationary noise estimate compared to the MCRA-based algorithms. The BOSCPD algorithm shows improvement in objective quality measures in terms of higher SNR and lower output distortion scores for speech enhancement. It is also tested to track and compensate for rapidly varying noises in on-line automatic speech recognition (ASR) using the Aurora 2 speech database. The simulation results show significant improvement in recognition accuracy compared to the baseline MCRA technique.

We introduce novel language model (LM) adaptation approaches using the latent Dirichlet allocation (LDA) model. Observed n-grams in the training set are assigned to topics using soft and hard clustering. In soft clustering, each n-gram is assigned to topics such that the total count of that n-gram for all topics is equal to the global count of that n-gram in the training set. Here, the normalized topic weights of the n-gram are multiplied by the global n-gram count to form the topic n-gram count for the respective topics. In hard clustering, each n-gram is assigned to a single topic with the maximum fraction of the global n-gram count for the corresponding topic. Here, the topic is selected using the maximum topic weight for the n-gram. The topic n-gram count LMs are created using the respective topic n-gram counts and adapted by using the topic weights of a development test set. We compute the average of the confidence measures: the probability of word given topic and the probability of topic given word. The average is taken over the words in the n-grams and the development test set to form the topic weights of the n-grams and the development test set respectively. Our approaches show better performance over some traditional approaches using the WSJ corpus.

This paper studies the low-variance multi-taper mel-frequency cepstral coefficient (MFCC) features in the state-of-the-art speaker verification. The MFCC features are usually computed using a Hamming-windowed DFT spectrum. Windowing reduces the bias of the spectrum but variance remains high. Recently, low-variance multi-taper MFCC features were studied in speaker verification with promising preliminary results on the NIST 2002 SRE data using a simple GMM-UBM recognizer. In this study our goal is to validate those findings using a up-to-date i-vector classifier on the latest NIST 2010 SRE data. Our experiment on the telephone (det5) and microphone speech (det1, det2, det3 and det4) indicate that the multi-taper approaches perform better than the conventional Hamming window technique.

This paper presents a system that allows the user to interact by speech with a Radio-Frequency IDentification (RFID) network working in a highly noisy environment. A new dialog framework is proposed in order to give the human operators the ability to communicate with the system in a more natural fashion. This is achieved by the implementation of the Artificial Intelligence Markup Language which, allows the system to respond to close-to natural language queries by means of pattern matching. Besides this, and in order to deal with highly noisy and changing environments, an online signal subspace enhancement technique based on the Variance of the Reconstruction Error of Karhunen-Love Transform (VRE-KLT) is proposed. Both of the dialog naturalness and speech enhancement efficiency are evaluated in real-life situations by using the Microsoft Speech API framework.

This paper proposes an efficient codebook design for tree-structured vector quantization (TSVQ), which is embedded in nature. We modify two speech coding standards by replacing their original quantizers for line spectral frequencies (LSF's) and/or Fourier magnitudes quantization with TSVQ-based quantizers. The modified coders are fine-granular bit-rate scalable with gradual change in quality for the synthetic speech.

In this paper we propose a k-NN/SASH phoneme classification algorithm that competes favourably with state-ofthe-art methods. We apply a similarity search algorithm (SASH) that has been used successfully for classification of high dimensional texts and images. Unlike other search algorithms, the computational time of SASH is not affected by the dimensionality of the data. Therefore, we generate fixed-length but high-dimensional feature vectors for phonemes using their underlying frames and those of their boundaries. The k-NN/SASH phoneme classifier is fast, efficient, and could achieve a classification rate of 79.2% for the TIMIT test database. Finally, we apply this algorithm to rescore phoneme lattices, generated by the GMMHMM monophone recognizer for both context-independent and context-dependent tasks. In both cases, thek-NN/SASH classifier leads to improvements in the recognition rate. Index Terms: phoneme classification, nonparametric density estimation, lattice rescoring

Suppression of speech components after perceptual speech enhancement (SE) lowers the noise masking threshold (NMT) level of the enhanced signal. This may re-introduce noise components that are initially masked but not processed by the denoising filter, thereby, favoring the emergence of musical noise. This paper presents a modified perceptual speech enhancement algorithm based on a perceptually motivated weighting factor to effectively suppress the background noise without introducing much distortion in the enhanced signal using the perceptual speech enhancement methods. The performance of the proposed enhancement algorithm is evaluated by the Segmental SNR and Perceptual Evaluation of Speech Quality (PESQ) measures under various noisy environments and yields better results compared to the perceptual speech enhancement methods.

This paper examines the use of a new filtered minima-controlled recursive averaging (FMCRA) noise estimation technique as a robust front-end processing to improve the performance of a distributed speech recognition (DSR) system in noisy environments. The noisy speech is enhanced by using a two-stage framework in order to simultaneously address the inefficiency of the voice activity detector (VAD) and to remedy the inadequacies of MCRA. The performance evaluation carried out on the Aurora 2 task showed that the inclusion of FMCRA in the front-end side leads to a significant improvement in DSR accuracy.

In this paper we investigate a non-parametric classification of English phonemes in speaker-independent continuous speech. We employ the ldquovotingrdquo k-nearest neighbour (k-NN) classifier, a powerful technique in pattern recognition problems, along with a new representation of phonemes for the speech recognition task. We also exploit the idea behind ldquoapproximaterdquo k-NN that results in a very fast way of computing the k approximate closest neighbours of each data point. Comparing the recognition performance of the proposed method with the HMM-based recognizer of HTK toolkit reveals that the k-NN-based recognizer outperforms its counterpart. In addition, incorporating the ldquoapproximaterdquo nearest neighbour search instead of the ldquoexactrdquo one results in completing the training step much faster than the HMM-based system, and the testing step with a comparable computational time. We also reduced the amount of the training data by applying a pattern recognition technique, called ldquothinningrdquo algorithm. The outcome was a considerable reduction in the k-NN search space and hence the execution time, and also a slight increase in the recognition performance.

It has been shown that speech spectrograms can be read by trained experts. In this work, we regard the speech spectrogram image as a written text in some unknown language and perform segmentation in order to capture the energy associated with each formant. We propose an algorithm based on Mathematical Morphology operators and mainly on the watershed transform. The result is robust segmentation for wideband speech spectrograms that can be later used for automatic speech recognition. We show results of experimental runs for different phoneme classes.

A new, likelihood-based non-uniform allocation of Gaussian kernels in scalar (feature) dimension is proposed to compress complex, Gaussian mixture-based, continuous density HMMs into computationally efficient, small footprint models. Different from the objective of the previously proposed Kullback-Leibler divergence-based (KLD-based) allocation (Li et al., 2005), which is to make a better representation of the original model, the objective of the likelihood-based approach is to make the current compressed model be a better representation of the training data. It is implemented based on the unequal likelihood contributions of different features with uniform representation resolutions. Our experiments on the resource management database show that likelihood-based allocation outperforms uniform allocation and KLD-based non-uniform allocation due to its better representation of the training data.

We present in this paper a signal subspace-based approach for enhancing a noisy signal. This algorithm is based on a principal component analysis (PCA) in which the optimal subspace selection is provided by a variance of the reconstruction error (VRE) criterion. This choice overcomes many limitations encountered with other selection criteria, like over-estimation of the signal subspace or the need for empirical parameters. We have also extended our subspace algorithm to take into account the case of colored and babble noise. The performance evaluation, which is made on the Aurora database, measures improvements in the distributed speech recognition of noisy signals corrupted by different types of additive noises. Our algorithm succeeds in improving the recognition of noisy speech in all noisy conditions.

In distributed speech recognition, vector quantization is used to reduce the number of bits for coding speech features at the user end in order to save energy for transmitting speech feature streams to remote recognizers and reduce data traffic congestion. We notice that the overall bit rate of the transmitted feature streams could be further reduced by not sending redundant frames that can be interpolated at the remote server from received frames. Interpolation introduces errors and may degrade speech recognition. This paper investigates the methods of selecting frames for transmission and the effect of interpolation on recognition. Experiments on a large vocabulary recognizer show that with spline interpolation, the overall frame rate for transmission can be reduced by about 50% with a relative increase in word error rate less than 5.2% for clean and noisy speech.

In any classification task the confusion error, in general, is proportional to the number of classes. This is mainly due to sharing of some common attributes (feature vectors) among different classes. This, in many cases, leads to a serious problem, in the sense that, the classifier itself may be biased towards a specific class or a subset of classes. An ideal classifier is not expected to have any such bias. If we assume that, for a given pair of models and their corresponding training data, the log-likelihoods are distributed normally, the bias of any of these models may be visualized in the likelihood-space as an overlap between Gaussian likelihoods of different models (classes). In this paper, we propose a discriminant measure, using a product of Gaussian likelihoods, to estimate the amount of bias. By adjusting the complexity of the models, we show that this bias can be neutralized and a better classification accuracy can be achieved. Presently, the experiments are carried out on the OGLMLTS telephone speech corpus on a language identification task. The results show that a better classification accuracy can be achieved without any degradation in the performance of any of the individual classes.

Voiced-unvoiced-silence (V/UV/S) classification of speech sounds is important in automatic speech/speaker recognition, speech segmentation, speech signal compression, and speech analysis. Training-based classifications suffer from lack of training databases or degrade when training and test statistics mismatch due to variances in speakers, languages, talking styles, noise, transmission channels, etc. This paper proposes a novel voiced-unvoiced-silence classification based on unsupervised learning. The class-dependent statistics (feature means, covariance matrices, and occurrence frequencies of voiced, unvoiced, and silence classes) needed for the classification are estimated directly from the signal to be classified via Gaussian mixture models and the expectation maximization algorithm. The classification is evaluated using NTIMIT, and the results are encouraging: V/UV/S classification accuracy is greater than 91.15%, and voice activity detection accuracy is greater than 97.45%.

We present in this paper a subspace approach for enhancing a noisy speech signal. The original algorithm for model identification from which we have derived our method has been used in the field of fault detection and diagnosis. This algorithm is based on principal component analysis in which the optimal subspace selection is provided by a variance of the reconstruction error (VRE) criterion. This choice overcomes many limitations encountered with other selection criteria, like overestimation of the signal subspace or the need for empirical parameters. We have also extended our subspace algorithm to take into account the case of colored and babble noise. The performance evaluation, which is made on the Aurora database shows that our method provides a higher noise reduction and a lower signal distortion than existing enhancement methods. Our algorithm succeeds in enhancing the noisy speech in all noisy conditions without introducing artifacts such as “musical noise”. Index Terms: speech enhancement, model identification, signal subspace, principal component analysis, colored noise

In this letter, we derive the theoretical complex cepstrum (TCC) of the discrete cosine transform (DCT) and warped DCT (WDCT) filters. Using these derivations, we intend to develop an analytic model of the warped discrete cosine transform cepstrum (WDCTC), which was recently introduced as a speech processing feature. In our derivation, we start with the filter bank structure for the DCT, where each basis is represented by a finite impulse response (FIR) filter. The WDCT filter bank is obtained by substituting z-1in the DCT filter bank with a first-order all-pass filter. Using the filter bank structures, we first derive the transfer functions for the DCT and WDCT, and subsequently, the TCC for each filter is computed. We analyze the DCT and WDCT filter transfer functions and the TCC by illustrating the corresponding pole-zero maps and cepstral sequences. Moreover, we also use the derived TCC expressions to compute the cepstral sequence for a synthetic vowel /aa/ where the observations on the theoretical cepstrum corroborate well with our practical findings

The mismatch between system training and operating conditions can seriously deteriorate the performance of automatic speech recognition (ASR) systems. Various techniques have been proposed to solve this problem in a specified speech environment. Employment of these techniques often involves modification on the ASR system structure. In this paper, we propose an environment-independent (EI) ASR model parameter adaptation approach based on Bayesian parametric representation (BPR), which is able to adapt ASR models to new environments without changing the structure of an ASR system. The parameter set of BPR is optimized by a maximum joint likelihood criterion which is consistent with that of the hidden Markov model (HMM)-based ASR model through an independent expectation-maximization (EM) procedure. Variations of the proposed approach are investigated in the experiments designed in two different speech environments: one is the noisy environment provided by the AURORA 2 database, and the other is the network environment provided by the NTIMIT database. Performances of the proposed EI ASR model compensation approach are compared to those of the cepstral mean normalization (CMN) approach, which is one of the standard techniques for additive noise compensation. The experimental results show that performances of ASR models in different speech environments are significantly improved after being adapted by the proposed BPR model compensation approach

Knowledge about lip and glottal reflection coefficients during phonation is needed to eliminate their distortion effects on the estimates of vocal-tract area functions and glottal waves from vowel sounds. Direct measurements of these coefficients at human mouths are difficult. This paper presents a method for estimating them from vowel sounds. The estimation encounters an ill-defined inverse problem: the number of unknowns is greater than the number of constraints, and non-unique solutions exist for a sound. To overcome this problem, this paper uses a vowel sound produced by a subject whose vocal-tract area function (VTAF) for the sound is known. The estimates of the lip and the glottal reflection coefficients are determined as those that lead to a VTAF solution most similar to the known VTAF for the sound. The lip and the glottal reflection coefficients obtained for/a/and/i/ are presented.

Abstract A Longer-sized sub-word unit is known to be a better candi-date in the development of a continuous speech recognition sys-tem. However, the basic problem with such units is the data spar-sity. To overcome this problem, researchers have tried to com-bine longer-sized sub-word unit models with phoneme models. Inthis paper, we have considered only frequently occurring syllablesand VC (Vowel + Consonant) units, and phone-sized units (mono-phones and triphones) for the development of a continuous speechrecognition system. Insuch a case, even for a single pronunciationof a word, there can be multiple representational baseforms in thelexicon, each with different-sized units. We show that a consid-erable improvement in recognition performance can be achievedif the baseforms are selected properly. Out of all possible base-forms for a given word in the lexicon, the baseform that maxi-mizes the acoustic likelihood, for possible sub-word unit concate-nations to make a word, alone is considered. In the baseline sys-tems’ word-lexicon, like pure monophone or triphone-based sys-tems, since only the acoustically weaker baseforms are replacedby baseforms with longer-sized units, the resultant performance isguaranteed to be better than that of baseline systems. The prelim-inary experiments carried out on the TIMIT speech corpus showa considerable improvement in the recognition performance overa pure monophone/triphone-based systems when the larger-sizedunits are combined using proper selection of baseforms.

The subspaces representing two different vowel classes may have a large common subspace due to speaker variability, noise and coarticulation. We use common principal component (CPC) and its extension i.e., partial-CPC (pCPC) to obtain a specific subspace for each vowel which is insensitive to variations. We perform CPC analysis on the covariance matrices of the vowels. An input vector from an unknown vowel is classified based on the maximum length of its projection on the specific subspaces. We have chosen 18D mel-frequency cepstral coefficients as a feature in our recognition task. The specific subspace is treated as a transformation matrix which enhances the vowel-specific information in the feature vector and, in turn, increases SNR. Experiments were performed on vowels extracted from a multiple speaker set taken from different dialect regions. The results are encouraging in the context of a speaker-independent framework. Visual analysis of the vowel basis spectra provides useful and interesting information by highlighting the importance of different frequency regions.

Cepstral mean normalization (CMN) has proved to be a simple noise robust feature processing technique. In its computation, the log-energy feature or C0 is treated in the same way as other cepstral coefficients. Mean normalization is not an effective way to remove effects of additive noise for the log-energy feature. We propose a log-energy dynamic range normalization (ERN) algorithm which normalizes log-energy sequences of an utterance to a target dynamic range. The AURORA 2.0 database together with HTK speech recognition toolkits are used to evaluate the proposed algorithm. The proposed algorithm improves the recognition result by 30.83% over the reference front-end algorithm in clean-condition training. It is superior to the result of CMN, which is 19.30%. It is also confirmed that this technique can be combined with CMN to achieve a 46.33% performance gain. The proposed algorithm is fairly simple and it only requires a very small extra computation load.

This paper addresses the problem of speaker characterization in the speaker-dependent speech recognition problem. Speaker Adaptation and Normalization techniques are designed to reduce the mismatch introduced by inter-speaker variability. Yet there is another source of mismatch introduced by intra-speaker variability. Indeed, the speaking style of a speaker depends on the nature of the speech uttered. The framework of this paper is speaker-dependent isolated-word recognition on an embedded engine. The limited computational and memory loads of this engine reduce the possible techniques for normalization. The proposed solution uses a speaker profile trained on dictation data and exported to the embedded engine. In this framework we study the portability of a task-dependent speaker profile from dictation task to command task. Experiments have been conducted on a Scansoft 255 speakers database. We show that the portability results in a loss of efficiency due to the nature of the considered tasks.

Speech signals convey information from many sources, but not all information sources are relevant to describe speaker identity. In fact, speech is affected by spurious events, artifacts (mouth breath, lip clicks), and noise (channel and background). Such unwanted information sources are shared by speakers and do not contribute in distinguishing between them. Furthermore, in most cases, training data are collected from different environments and it is of great importance that such data convey relevant joint information. This paper discusses a method for removing unwanted information in order to build more robust models. Two criteria are used to extract relevant information from the speech signal: the first criterion, which we call self-information criterion, is used to extract relevant information from data collected from a given environment; the second is called joint information criterion, and it is used when collected data are from different environments. Both criteria originate from information theory. Simulations on telephone speech have revealed the high efficency of the method.

Abstract In this paper, we continue our investigation of the warpeddiscrete cosine transform cepstrum (WDCTC), which wasearlier introduced as a new speech processing feature [1].Here, we study the statistical properties of the WDCTC andcompare them with the mel-frequency cepstral coefficients(MFCC). We report some interesting properties of the WD-CTC when compared to the MFCC: its statistical distrib-ution is more Gaussian-like with lower variance, it obtainsbettervowel clusterseparability,it formstightervowelclus-ters and generates better codebooks. Further, we employthe WDCTC and MFCC features in a 5-vowel recognitiontask using Vector Quantization (VQ) and 1-Nearest Neigh-bour(1-NN)as classifiers. Inourexperiments,the WDCTCconsistently outperforms the MFCC. 1. Introduction We recently introduced the warped discrete cosine trans-form cepstrum (WDCTC) as a new speech processing fea-ture and demonstrated its better performance than the mel-frequency cepstral coefficients (MFCC) in a vowel recog-nition and speaker-identification task [1]. The WDCTC hasshown good promise as a speech processing feature and weare encouraged to further investigate the WDCTC featureand its statistical properties.Alargevolumeoftrainingdataisrequiredto buildspea-ker-independentspeechrecognitionsystems. Onetechniqueof reducing the data size is clustering the data and choos-ing a reasonable number of representative feature vectorsto form codebooks [2]. Hence, codebook techniques arevery relevant and practical to speech recognition systems.We form WDCTC and MFCC codebooks using a k-meansclustering algorithm and compare the codebook statisticsfor clean and noisy vowels using the coefficient of varianceand overlap ratio (defined later). Our experiment demon-strates that the WDCTC codebooks represent the underly-ing vowel data better than MFCC.In order to compare the classification capability of thefeatures, the WDCTC and MFCC are employed in a 5-vowel recognition task. Vector quantization (VQ) and 1-nearestneighbor(1-NN,[2])are usedas classifiersandtheirrecognition performance is reported. We also investigatethe clean and noisy vowel clusters formed by WDCTC andMFCC features and present the average separability of thevowel classes.

We investigate the ASR performance of speech transmitted over a noisy Bluetooth RF channel. Bluetooth shares its transmission channel with IEEE 802.11-based devices. Despite Bluetooth's frequency hopping scheme, our investigation shows that Bluetooth packet loss rates may reach up to 38% in unfavorable 802.11 interference conditions, and as Bluetooth uses a CVSD (continuous variable slope delta modulation) codec with syllabic companding, these packet losses not only manifest themselves as segments of missing speech upon CVSD decoding, but also as incorrect scaling of subsequent successfully received voice packets as CVSD step-size information is also lost. We investigate the effects of these degradations on the ASR performance of Bluetooth speech, and accordingly propose alternative CVSD decoder schemes employing insertion-based techniques for compensating for these effects. Results show that our proposed techniques improve ASR performance considerably while requiring only minor modifications to the current Bluetooth receiver.

The paper presents a method to compensate Mel-frequency cepstral coefficients (MFCCs) for a HMM-based speech recognition system evolving under telephone-channel degradations. The technique we propose is based on the combination of the Karhonen-Loeve transform (KLT) and genetic algorithms (GA). The idea consists of projecting the band-limited MFCCs onto a subspace generated by the genetically optimized KLT principal axes. Experiments show a clear improvement when the method is applied to the NTIMIT telephone speech database. Word recognition results obtained on the HTK toolkit platform using N-mixture triphone models and a bigram language model are presented and discussed.

Speech field is affected by accidental structures such as spurious events and artifacts (breath mouth, lip clicks etc). Because the whole utterance is used during training and the identification process, these factors may represent confusable acoustic classes which do not contribute to the performance of the speaker recognition system. In this paper we propose a new approach for extracting as much essential information as possible from the acoustic data in order to estimate more robust speaker models. To this end, we make use of the Asymptotic Equipartition Property (AEP) originated from the information theory to generate a reduced feature subspace termed ”typical set”. Results on the Spidre corpus show that the method leads to an appreciable improvement (averaging 10 % gain in performance) over the baseline system.

Automatic speech recognition (ASR) model adaptation is important to many real-life ASR applications due to the variability of speech. The differences of speaker, bandwidth, context, channel and et al. between speech databases of initial ASR models and application data can be major obstacles to the effectiveness of ASR models. ASR models, therefore, need to be adapted to the application environments. Maximum Likelihood Linear Regression (MLLR) is a popular model-based method mainly used for speaker adaptation. This paper proposes a feature-based statistical Data Mapping (SDM) approach, which is more flexible than MLLR in various applications, such as different bandwidth and context. Experimental results on the TIMIT database show that ASR models adapted by the SDM approach have improved accuracy.

The mismatch between system training and operat-ing conditions often has negative influences on automatic speech recognition (ASR) systems. Noise in the operat-ing environments is commonly encountered. ASR model adaptation is an important way to enhance the system per-formance in noisy environments. This paper proposes a feature-based statistical data mapping (SDM) approach for robust noisy digit recognition. The recognition tasks are carried out on the AURORA 2 database. Compared to other model adaptation methods such as MLLR, the SDM approach has more robust performances. 1.

In speech recognition systems, feature extraction can be achieved in two steps: parameter extraction and feature transformation. Feature transformation is an important step. It can concentrate the energy distributions of a speech signal onto fewer dimensions than those of parameter extraction and thus reduce the dimensionality of the system. Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA) are the two popular feature transformation methods. This paper investigates their performances in dimensionality reduction tasks in continuous speech recognition systems. A new type of feature transformation, LP transformation, is proposed and its performance is compared to those of LDA and PCA transformations.

This paper examines how people communicate with computers using speech. Automatic speech recognition (ASR) transforms speech into text, while automatic speech synthesis [or text-to-speech (TTS)] performs the reverse task. ASR has been largely developed based on speech coding theory, while simulating certain spectral analyses performed by the ear. Typically, a Fourier transform is employed, but following the auditory Bark scale and simplifying the spectral representation with a decorrelation into cepstral coefficients. Current ASR provides good accuracy and performance on limited practical tasks, but exploits only the most rudimentary knowledge about human production and perception phenomena. The popular mathematical model called the hidden Markov model (HMM) is examined; first-order HMMs are efficient but ignore long-range correlations in actual speech. Common language models use a time window of three successive words in their syntactic-semantic analysis. Speech synthesis is the automatic generation of a speech waveform, typically from an input text. As with ASR, TTS starts from a database of information previously established by analysis of much training data, both speech and text. Previously analyzed speech is stored in small units in the database, for concatenation in the proper sequence at runtime. TTS systems first perform text processing, including "letter-to-sound" conversion, to generate the phonetic transcription. Intonation must be properly specified to approximate the naturalness of human speech. Modern synthesizers using large databases of stored spectral patterns or waveforms output highly intelligible synthetic speech, but naturalness remains to be improved.

In this paper, a multi-stream paradigm is proposed to improve the performance of automatic speech recognition (ASR) systems. Our goal in this paper is to improve the performance of the HMM-based ASR systems by exploiting some features that characterize speech sounds based on the auditory system and one based on the Fourier power spectrum. It was found that combining the classical MFCCs with some auditory-based acoustic distinctive cues and the main peaks of the spectrum of a speech signal using a multi-stream paradigm leads to an improvement in the recognition performance. The Hidden Markov Model Toolkit (HTK) was used throughout our experiments to test the use of the new multi-stream feature vector. A series of experiments on speaker-independent continuous-speech recognition have been carried out using a subset of the large read-speech corpus TIMIT. Using such multi-stream paradigm, N-mixture mono-/tri-phone models and a bigram language model, we found that the word error rate was decreased by about 4.01%.

ABSTRACTIn this paper, the problem of robust large-vocabulary continuous-speech recognition (CSR) in the presence of highly interferingcar noise has been considered. Our approach is based on thenoise reduction of the parameters that we use for recognition,that is, the Mel-based cepstral coefficients. This is achieved bythe use of a Multilayer Perceptron (MLP) network for noise re-duction in the cepstral domain in order to get less-variant pa-rameters. Then, the obtained enhanced features are refined viathe Karhunen-Lo`eve Transform (KLT) implemented using thePrincipal Component Analysis (PCA). Experiments show thatthe use of the enhanced parameters using such an approach in-creases the recognition rate of the CSR process in highly inter-fering car noise environments. The HTK Hidden Markov ModelToolkit was used throughout our experiments. Results show thatthe proposed hybrid technique when included in the front-endof an HTK-based CSR system, outperforms that of the conven-tional recognition process based on either a KLT- or an MLP-based preprocessing recognition in severe interfering car noiseenvironments for a wide range of SNRs varying from 16 dB to-4 dB using a noisy version of the TIMIT database.1. INTRODUCTIONThe performance of existing CSR systems, whose designs arepredicated on relatively noise-free conditions, degrades rapidlyin the presence of a high level of adverse conditions. Several ap-proaches have been studied for achieving noise robustness [1, 2].In this paper, we focus on optimizing the performance of a CSRsystem by choosing a suitable distortion measure. The idea ofa robust distance measure is to extract relevant features fromspeech signals which must be insensitive to degradations of thespeech signal due to interfering noise or distortions. Many ap-proaches [3] have been used to extract relevant features froma speech signal. Cepstral parameters are well suited to speechrecognition due to their compact orthogonality. Unfortunately,cepstral features are highly sensitive to noise. It was shown in[4] that cepstral distributions for clean data are well behaved andapproximately normal, but in the presence of noise, their profilesare changed significantly and this consequently degrades the per-formance of an CSR system. However, the cepstrum coefficientshave the additional advantage that one can derive from them aset of parameters which are invariant to any fixed frequency-response distortion introduced by either the adverse environ-ments or thetransmission channels. Severalapproaches toobtaina new set of robust parameters were introduced in [5, 6, 7].In this paper, we propose a novel robust CSR system to be usedin car noisy environments. Our approach for noise reduction isapplied in the cepstral domain. It is based on the application ofa combination of the Karhunen-Lo`eve Transform (KLT) and aConnectionist approach. Each of these two approaches has beensuccessfully used in both speech enhancement and recognitionprocesses. We show in this paper through experiments on highlynoisy data that a cepstral noise reduction can be obtained us-ing such an approach and consequently an improvement of therecognition performance.This paper will be organized into the following sections. In sec-tion 2 we describe the basis of the MLP network and the PCAapproaches that will be used to describe our proposed hybridPCA-MLP approach. Then, we proceed in section 3 with the de-scription of the database, the platform used in our experimentsand the evaluation of the proposed MLP-PCA-based recognizerin a noisy car environment and the comparison of such a recog-nizer to both the MLP- and the PCA-based recognizers in orderto evaluate its performance. Finally, in section 5 we concludeand discuss our results.2. PROPOSED ENHANCEMENT APPROACH2.1. Multilayer Perceptron NetworkAs mentioned above, the first step that has been proposed to im-prove the performance of the CSRprocess in highly noisy caren-vironments in the cepstral domain is the use of a multilayer per-ceptron (MLP) network. The fact that the noise and the speechsignal are combined in a nonlinear way in the cepstral domainmotivated us to choose the MLP, since it can approximate the re-quired nonlinear function to some extent [6, 7]. The input of theMLP is the noisy MFCC vector

Reports the progress of the large-vocabulary French-speech vocal dictation studies at INRS-Te/spl acute/le/spl acute/com. To evaluate such progress, the hidden Markov model (HMM) based recognizer of INRS is used. This recognizer, which represents each phone using HMMs, uses context-dependent phone modeling and n-gram statistics in order to cope with both coarticulation and phonological phenomena, respectively. A series of experiments on speaker-independent continuous-speech recognition have been carried out using a subset of the large read-speech French-language corpus, BREF, containing recordings of texts selected from the French newspaper Le Monde. We show through experiments that using a lexical graph that ignores the language model states and homophone distinctions and postponing the application of such knowledge to a post-processor simplifies the recognition process while keeping its high accuracy. The word recognition rate, using gender-dependent vector quantization (VQ) models, a 20,000-word pronunciation variants-based lexicon and a bigram model estimated using Le Monde text data, was found to be 91.62% for males and 90.98% for females.

Automatic recognition of sequences of spoken digits (e.g., telephone or credit card numbers) can be accomplished with excellent accuracy, even in speaker‐independent applications over telephone links. However, even such relatively simple recognition tasks suffer decreased performance in adverse conditions, such as significant background noise or fading on portable telephone channels. If one further imposes significant limitations on the computing resources to be dedicated to a recognition task, then robust, limited‐resource speech recognition remains a suitable challenge, even for a vocabulary as simple as the digits. Since connected‐digit recognition over telephone lines is a very practical application, the amount of computer resources needed for a given level of recognition accuracy was investigated for different levels and types of acoustic noise. Rather than use a traditional hidden Markov model approach with cepstral analysis, which is computationally intensive and does not always work well under adv...

We show that the concept of voiced-unvoiced (VU) classification of speech sounds can be incorporated not only in speech analysis or speech enhancement processes, but also can be useful for recognition processes. That is, the incorporation of such a classification in a continuous speech recognition (CSR) system not only improves its performance in low SNR environments, but also limits the time and the necessary memory to carry out the process of the recognition. The proposed V-U classification of the speech sounds has two principal functions: (1) it allows the enhancement of the voiced and unvoiced parts of speech separately; (2) it limits the Viterbi (1967) search space, and consequently the process of recognition can be carried out in real time without degrading the performance of the system. We prove via experiments that such a system outperforms the baseline HTK when a V-U decision is included in both front- and far-end of the HTK-based recognizer.

The first motivation for using Gaussian mixture models for text-independent speaker identification is based on the observation that a linear combination of Gaussian basis functions is capable of representing a large class of sample distributions. While this technique gives generally good results, little is known about which specific part of a speech signal best identifies a speaker. This contribution suggests a procedure, based on the Jensen divergence measure, to automatically extract from the input speech signal the part that best contributes to identify a speaker. Experiments conducted using the Spidre database indicate a significant improvement in the performance of the speaker recognition system.

A new cepstrum-based channel compensation method is proposed for speaker verification over the telephone network. The method consists of intra-frame and interframe cepstral processing. For the former, a poleremoved cepstrum is derived, where the LP poles with frequency higher than a certain threshold are removed. For the latter, we introduce a novel way of cepstral mean subtraction called differential-partial cepstral mean subtraction (DPCMS). The main idea is that the cepstral mean of clean speech is not necessarily zero and that the cepstral difference between clean and channel-corrupted speech is mainly contributed by the channel effects on LP poles within a certain frequency range. A speaker verification system based on radial basis function networks was used to evaluate the proposed approach. Clean speech was used to train the networks and telephone speech was used to evaluate their performance. Experimental results show that the proposed method reduces verification error rate sig...

Adaptation to the user is of great importance in a language training system since the user, making unfamiliar, socially unacceptable sounds, must be made to feel self-confident in order to obtain best results. The Fluency project aims at creating a pronunciation trainer for foreign language learning. It uses automatic speech recognition (CMU’s SPHINX II) and is guided by basic principles in foreign language learning research. We describe efforts in this project to make the interface adaptable concerning learning strategies and “golden voices”..

The focus of a continuous speech recognition process is to match an input signal with a set of words or sentences according to some optimality criteria. The first step of this process is parameterization, whose major task is data reduction by converting the input signal into parameters while preserving virtually all of the speech signal information dealing with the text message. This contribution presents a detailed analysis of a widely used set of parameters, the mel frequency cepstral coefficients (MFCCs), and suggests a new parameterization approach taking into account the whole energy zone in the spectrum. Results obtained with the proposed new coefficients give a confidence interval about their use in a large-vocabulary speaker-independent continuous-speech recognition system.

The objective of this work is to allow the INRS continuous-speech recognizer to process accurately new words and incorporate them into the vocabulary. Until now only a few new-word detectors have been reported, all of them defining an acoustic filler model different from the models used to represent vocabulary words. In this paper, we define several designs using, unlike other researchers, strictly-lexical fillers and a unique process to perform speech recognition, new-word detection and new-word phonetic transcription. Moreover, we propose here four different types of language models differing in the way they use the limited information we gathered on new words. The best combinations are found to be different from the ones we obtained for keyword spotting.

In this paper, a novel feature vector based on both mel frequency cepstral coefficients (MFCCs) and a mel-based nonlinear discrete-time energy operator (MDEO) is proposed to be used as the input of an HMM-based automatic continuous speech recognition (ACSR) system. Our goal is to improve the performance of such a recognizer using the new feature vector. Experiments show that the use of the new feature vector increases the recognition rate of the ACSR system. The HTK hidden Markov model toolkit was used throughout. Experiments were done on both the TIMIT and NTIMIT databases. For the TIMIT database, when the MDEO was included in the feature vector to test a multi-speaker ACSR system, we found that the error rate decreased by about 9.51%. On the other hand, for NTIMIT, the MDEO deteriorates the performance of the recognizer. That is, the new feature vector is useful for clean speech but not for telephone speech.

In this paper, the problem of robust speech recognition has been considered. Our approach is based on the noise reduction of the parameters that we use for recognition, that is, the Mel-based cepstral coefficients. A Temporal-Correlation-Based Recurrent Multilayer Neural Network (TCRMNN) for noise reduction in the cepstral domain is used in order to get less-variant parameters to be useful for robust recognition in noisy environments. Experiments show that the use of the enhanced parameters using such an approach increases the recognition rate of the continuous speech recognition (CSR) process. The HTK Hidden Markov Model Toolkit was used throughout. Experiments were done on a noisy version of the TIMIT database. With such a pre-processing noise reduction technique in the front-end of the HTK-based continuous speech recognition system (CSR) system, improvements in the recognition accuracy of about 17.77% and 18.58% using single mixture monophones and triphones, respectively, have been obtained at a moderate SNR of 20 dB.

In this paper, the implementation of a robust front-end to be used for a large-vocabulary Continuous Speech Recognition (CSR) system based on a Voiced-Unvoiced (V-U) decision has been addressed. Our approach is based on the separation of the speech signal into voiced and unvoiced components. Consequently, speech enhancement can be achieved through processing of the voiced and the unvoiced components separately. Enhancement of the voiced component is performed using an adaptive comb filtering, whereas the unvoiced component is enhanced using the modified spectral subtraction approach. We proved via experiments that the proposed CSR system is robust in additive noisy environments (SNR down to 0 dB).

Our goal is to design an accurate keyword spotter that can deal with any size of keyword set, since the size actually required in a wide range of applications is large (number of airports, number of names in a directory, etc.). This justifies the choice of an architecture based on a large-vocabulary continuous-speech recognizer. In a previous paper we introduced the use of strictly-lexical subword fillers for keyword spotting based on the INRS large-vocabulary continuous-speech recognizer showing that they are, when compared to acoustic fillers, a good compromise between memory and time consumption, keyword choice freedom and task-independence training on one hand and accuracy on the other hand. We propose here two new high-performance designs of individual strictly-lexical subword fillers that perform, this time, better than their acoustic counterparts while still keeping the mentioned advantages.

Modification of speech is a subject of major interest today, with numerous applications including text to speech synthesis. The basic mechanisms behind this process often consist of pitch-scale and time-scale modifications of speech. While giving generally good results, it remains in most of the cases that the same speaker can be associated with the original signal and its modified version, which limits the use of these techniques in some applications where disguising voices is necessary. These paper presents an approach to increase the possibilities of speech modifications while preserving most of the speech quality of the original signal.

A new algorithm is developed for constructing a transcription graph in large vocabulary continuous speech recognition. The transcription graph satisfies lexical constraints and encodes all high scoring phonetic transcriptions. By building a transcription graph in parallel with the word graph, we use high-level knowledge sources to constrain the paths of phonetic transcriptions such that they are created only if needed for building the word graph. This not only speeds up the search of the lexicon, but also limits the memory requirement in our system.

This paper describes a new word clustering approach for statistical language modeling. The classification criteria used by our approach is the contextual word similarity used in a simplified clustering algorithm. This clustering technique was tested on the INRS speech recognizer using the spontaneous English corpora, ATIS. Automatic word classification increases the word accuracy rate by 8.6% with a perplexity reduction about of 6.9%.

The high concentration of energy in the low frequency range observed for most speech spectra is considered a nuisance because it makes less relevant the energy of the signal at middle and high frequencies in many speech analysis algorithms. This problem is often partially solved by filtering the input signal with a single-zero high pass filter. This technique, known as pre-emphasis, is not used in some speech recognition systems because it can increase the noise energy at high frequencies, which in some cases degrades the system's performance. A new algorithm, to avoid the ill effects of pre-emphasis, is presented in this paper. It is shown that this approach can increase the recognition rate.

In this paper we present a new automatic male/female classication method based on the location in the frequency domain of the rst 2 formants.This classication is based on a new automatic formant extraction which i s faster than a peak picking technique.Gender-dependent acoustic-phonetic models stemming from this classication are used in the INRS Continuous speech recognition system with ATIS corpora.An improvement of 14% is obtained with these models in comparison to the baseline speaker-independent system.

We present new developments in our approach to the search problem in very large vocabulary speech recognition. A transcription graph, which encodes all high scoring phonetic transcriptions which satisfy lexical constraints, is built so as to completely separate the search of the lexicon from the construction of the word graph. This enables us to limit the computational burden of searching in a way which is essentially independent of the size of the language model.

Most previous acoustic analysis of speech has examined data from speakers who carefully pronounce their speech, usually by reading prepared texts. Natural spontaneous or conversational speech differs from careful or read speech, especially concerning hesitation phenomena and variable speaking rates. We examine and model global speaking rate, how it varies for both fluent and disfluent spontaneous speech, in terms of the linguistic content of the utterances. Speakers tend to maintain a fixed speaking rate during most utterances, but often adopt a faster or slower rate, depending on the cognitive load (i.e., slowing down when having to make unanticipated choices, or accelerating when repeating some words). Such a model can find application in automatic speech synthesis and recognition, because most synthesizers maintain a constant (and unnatural) speaking rate and most recognizers are not capable of adapting their templates or probabilistic models to reflect global changes in speaking rate.

We present a new multi-pass approach to the search problem in speech recognition. It builds on the 1-phone lookahead property which enables all of the time alignment for the first forward pass to be performed in pre-processing stage. This feature allows us to view the first forward pass a classical graph search problem and to apply powerful searching techniques that have not previously been used in speech recognition.>

A primary difference between spontaneous speech and read speech concerns the use of false starts, where a speaker interrupts the flow of speech to restart an utterance. We define 'complex restarts' as those involving deletion and/or insertion of word(s), as opposed to simple restarts which just repeat words (or parts of a word). The acoustic and distributional aspects of such complex restarts in a widely-used speech database were examined here. Identifying the nature and extent of the restart in such cases could improve the performance of an automatic speech recognizer, by allowing the elimination from consideration of some hypotheses based on spectral analysis. Results are given here which allow identification of many complex restarts.>

We present an algorithm to generate wideband speech from a narrow band version of the same. The main body of the algorithm is a statistical recovery function (SRF), which predicts the highband spectrum based solely on the narrowband spectrum. The performance of the algorithm has been measured both in terms of spectral distortion and spectral signal-to-noise ratio (SNR). We obtained a 3 dB gain in SNR for the reconstructed wideband speech as compared to the narrowband speech.>

Algorithms for encoding speech with good intelligence and naturalness at very low rates are studied. Naturalness is retained by encoding accurately the speech excitation information from an LPC (linear predictive coding) model. A glottal ARX (autoregressive with exogenous input) technique is used to model the speech signal for high quality. A large reduction in coding rate is achieved through short-term temporal compression of the speech and vector quantization. Application of traditional vector quantization to the temporal decomposition output is discussed, with consideration of distortion measures and codebook generation. Based on properties of short-term temporal decomposition, finite-state vector quantization is introduced to further decrease the coding rate. A problem associated with this technique, estimation of a state transition matrix with incomplete data, is treated. The general result is that practical coders operating in a range of 450-600 b/s with a delay of about 200 ms and natural-sounding output speech can be designed.>

A new class of A* algorithms for Viterbi phonetic decoding subject to lexical constraints is presented. This type of algorithm can be made to run substantially faster than the Viterbi algorithm in an isolated word recognizer having a vocabulary of 1600 words. In addition, multiple recognition hypotheses can be generated on demand and the search can be constrained in respect conditions on phone durations in such a way that computational requirements are substantially reduced. Results are presented on a 60000-word recognition task.>

The authors have assessed the possibility of modeling phone trajectories to accomplish speech recognition. This approach has been considered as one of the ways to model context-dependency in speech recognition based on the acoustic variability of phones in the current database. A hybrid segmental learning vector quantization/hidden Markov model (SLVQ/HMM) system has been developed and evaluated on a telephone speech database. The authors have obtained 85.27% correct phrase recognition with SLVQ alone. By combining the likelihoods issued by SLVQ and by HMM, the authors have obtained 94.5% correct phrase recognition, a small improvement over that obtained with HMM alone.>

Both filled and unfilled (silent) hesitation types of pauses in a widely used speech database were examined for both unintended and intended pauses. A distinction is made between grammatical pauses (at major syntactic boundaries) and ungrammatical ones (within minor syntactic phrases). While unfilled pauses cannot be reliably thus separated based on silence duration alone, grammatical pauses tended to be longer. In the prepausal word before ungrammatical pauses, there were few continuation rises in fundamental frequency (F0), whereas 70% of the grammatical pauses were accompanied by a prior F0 rise. Identifying the syntactic function of such pauses could improve the performance of an automatic speech recognizer, by eliminating from consideration some hypotheses based on spectral analysis. Results are given which could allow simple identification of most filled and unfilled pauses and their syntactic function.>

We view the speech enhancement task in two aspects: reduction of the perceptual noise level in degraded speech and reconstruction of the degraded information, which may result in improvement of speech intelligibility. We are also very interested in noiseindependent speech enhancement where test noise environments could differ in intensity from those of algorithm development. To this end, we have developed in this paper an algorithm called Noise-Independent Statistical Spectral Mapping (NISSM) to estimate a speech enhancement Wiener filter. NISSM consists of a noise-resistant transformation, which converts noisy speech to a set of noise-resistant features, and a spectral mapping function, which maps the features to autoregressive spectra of clean speech. We will show that the proposed algorithm effectively reduces noise intensity. When the noise intensity of training differs from that of testing, NISSM outperforms significantly a conventional spectral mapping. The algorithm operates frame-by-frame and is designed for real-time application. The noise interference could be stationary or non-stationary white noise with variable intensity. I.

We present an algorithm to generate wideband speech from a narrow band version of the same. The main body of the algorithm is a statistical recovery function (SRF), which predicts the highband spectrum based solely on the narrowband spectrum. The performance of the algorithm has been measured both in terms of spectral distortion and spectral signal-to-noise ratio (SNR). We obtained a 3 dB gain in SNR for the reconstructed wideband speech as compared to the narrowband speech. >

A primary difference between spontaneous speech and read speech concerns the use of false starts, where a speaker interrupts the flow of speech to restart his or her utterance. A study examined the acoustic aspects of such restarts in a widely-used speech database, examining approximately 1000 utterances, about 10% of which contained a restart. Identifying the type of restart in such cases could improve the performance of an automatic speech recognizer, by eliminating from consideration some hypotheses based on spectral analysis. Results showed that simple restarts (those without inserted or substituted words) could be distinguished acoustically, via an analysis of duration, FO (fundamental frequency) and spectral detail in the neighborhood of a pause. This would allow automatic identification of such restarts with an accuracy exceeding 80%, while keeping false alarms to below 10%. (Author/SR) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** PERMISSION TO REPRODUCE THIS IATERIAL HAS BEEN GRANTED BY

The problem of enhancing speech degraded by stationary and nonstationary additive white noise is addressed. The authors have explored a new idea, enhancing speech based on auditory evidence, for this traditional topic. Distinguishing different objectives for heavy and light noise interference, two related algorithms have been developed. For speech degraded by heavy noise, the improvement in signal-to-noise ratio (SNR) is as high as 12 dB; for lightly noisy speech, the improvement is modest and decreases as the SNR of the noisy speech increases. Quantizing noise is used to assess the capacity of reducing nonstationary noise with the algorithms; a significant reduction of such noise and an improvement in speech quality have been achieved. The advantages of the proposed algorithms for speech enhancement include no need for prior knowledge of the noise and only a modest computational requirement.>

Minimum duration constraints and energy thresholds for phonemes were used to increase the recognition accuracy of an 86000-word speaker-trained isolated word recognizer. Minimum duration constraints force the phoneme models to map to acoustic segments longer than the duration minima for the phonemes. Such constraints result in significant lowering of likelihoods of many incorrect word choices, improving the accuracy of acoustic recognition and recognition with the language model. The phoneme models were also improved by correcting the segmentation of the phonemes in the training set. During training, the boundaries between phonemes are not marked accurately. Energy is used to correct these boundaries. Application of an energy threshold improves the segment boundaries between stops and sonorants (vowels, liquids, and glides), between fricatives and sonorants, between affricates and sonorants. and between breath noise and sonorants. On two speakers, the overall reduction in errors using minimum durations and energy thresholds is from 27.3% to 23.1% for acoustic recognition and from 14.3% to 8.8% with the language model.>

The authors present a new class of A* algorithms for Viterbi phonetic decoding subject to lexical constraints. They show that this type of algorithm can be made to run substantially faster than the Viterbi algorithm in an isolated word recognizer having a vocabulary of 1600 words and that it runs very quickly on a 60000-word recognition task. In addition, multiple recognition hypotheses can be generated on demand and the search can be constrained to respect conditions on phone durations in such a way that computational requirements are substantially reduced.>

We present a new stochastic model for the energy and duration of phone segments which takes account of the speech rate, the loudness of the signal and the effects of stress and pre-pausal lengthening and we show how the block Viterbi decoding algorithm can be used to integrate it with phone-based HMM speech recognizers. The model has been implemented on an isolated-word data-base and a preliminary experiment gives a modest improvement in word recognition accuracy.

An ARX model with glottal excitation is used to achieve an accurate and economical frequency compression of speech, and a short-term temporal decomposition is used to do temporal compression. Quantization properties of the new set of coefficients and of the side information are studied through vector quantization. Moreover, use is made of the sequential structure of the vectors produced by a short-term temporal decomposition to decrease further the coding bit rate. As an application of the above techniques, a 450-b/s vocoder with a 200-ms delay is described.>

The authors demonstrate theoretically that an ARX model with glottal excitation can take into account segmentation error, but this introduces a perturbation of model parameters. A practical investigation of parameter sensitivity due to the error is reported. The parameter sensitivity can be viewed in terms of the robustness of the estimation process. The authors present a robust estimation procedure to decrease the sensitivity without an increase in computation.>

A global, context-sensitive parsing procedure that aids a large-vocabulary isolated-word speech recognition system is described. Presented with a long sequence of word candidates, the parser identifies likely erroneous words on the basis of a syntactic analysis of the preceding words. The parser suggests likely locations in the word sequence for punctuation marks such as sentence-final periods. It is not as powerful as semantic trigram language models, but it requires much less memory and training. One significant advantage is that it exploits sentence structure well beyond the three-word limit of trigram models.>

A system to improve the intelligibility of noisy speech through the use of vector quantization of linear predictive coding (LPC) spectra and a distance measure involving formants is described. Based on experiments using the system on natural speech degraded by additive white noise, the approach appears to be a promising way to transform noisy speech into more intelligible signals. As the noise corruption increases (and SRN decreases), the output speech becomes more distorted in terms of spectral jumps and mismatches, but remains free of noise. Good intelligibility remains as low as 9 dB SNR, although the speech is unnatural due to the LPC synthesis.>

In automatic synthesis of speech from English text, the quality of the output speech is highly dependent upon realistic intonation patterns. Most synthesizers have difficulty obtaining sufficient linguistic information from an input text to specify intonation properly. The syntactic structure of the text often specifies where a speaker should pause and which words to stress. However, the problem of parsing natural English is as yet unsolved. The problem is further complicated in systems which may wish to limit memory space and access time by minimizing dictionary size. In most other references, the parsing problem is only mentioned in passing, or parsing occurs on a local basis, ignoring important syntactic structures that encompass the entire sentence. The system described here recognizes function words and uses a set of syntactic constraints to estimate which words are likely to form phrases. This paper is the first to report on parsing details specifically for synthesis, while using only a small dictionary (of about 300 words).

Toward the objective of higher quality French synthetic speech by rule. 181 phonetically-diverse words spoken at three speaking rates were analyzed for formant patterns and durations. Examples of formant undershoot of vowel targets due to coarticulation or increased speaking rate occurred only when large formant excursions were needed between vowels and adjacent consonants. Increased rate led to faster formant transitions, but not, in general, to changes in the extent of the transitions. Undershoot occurred more often in semivowels, especially at the fast speaking rate. Formant transitions were largely independent of the number and position of syllables in a word, although vowel duration was much longer in a word-final syllable than word-internally.

From these studies, it is possible to model the fundamental frequency (F0) contours for sentences which exhibit a comprehensive range of linguistic communicative functions, and thus extend the understanding of the role of F0from simple declarative sentences to a class which more nearly typifies the utterances found in nornsl discourse. Any attempt to characterize discourse-length utterances must account for these factors, and it is felt that the use of language functions within a speech act framework provides an insightful means to account for these F0phenomena.