Probabilistic linear discriminant analysis (PLDA) is a popular tool in speaker recognition. PLDA parameters are estimated from labeled data, and domain shift causes performance degradation. Since obtaining a labeled dataset for target domain is costly, we propose an unsupervised domain adaptation of PLDA which does not require clustering or alignment. Our proposed method relies on discriminative training of PLDA using pseudo target/non-target labels. Segment pairs extracted from the same utterance that is likely to contain a single speaker are labeled as target, and pairs from randomly sampled utterances that are likely to belong to different speakers are labeled as non-target. We applied this approach to Neural PLDA. Our approach significantly improved the performance of the out-of-domain PLDA on the target domain with a relatively small amount of unlabeled data, performing on par with a baseline adaptation approach on SRE18 full-full scenario while outperforming it on full-10s and 10s-10s cases.

Abstract Sequence to sequence models (seq2seq) require a large amount of labelled training data to learn the mapping between the input and output. A large set of misspelled words together with their corrections is needed to train a seq2seq spelling correction system. Low‐resource languages such as Turkish usually lack such large annotated datasets. Although misspelling‐reference pairs can be synthesized with a random procedure, the generated dataset may not well match to genuine human‐made misspellings. This might degrade the performance in realistic test scenarios. In this paper, we propose a novel procedure to automatically introduce human‐like misspellings to legitimate words in Turkish language. Generated human‐like misspellings are used to improve the performance of a seq2seq spelling correction system. The proposed system consists of two separate models; a misspelling generator and a spelling corrector. The generator is trained using a relatively small number of human‐made misspellings and their manual corrections. Reference words and their misspellings are used as inputs and outputs of the generator, respectively. As a result, it is trained to add realistic spelling errors to the valid words. Training data of the spelling corrector is augmented by the generator's human‐like misspellings. In the experiments, we observe that the data augmentation significantly improves the spelling correction performance. Our proposed method yields 5% absolute improvement over the state‐of‐the‐art Turkish spelling correction systems in a test set which contains human‐made misspellings from Twitter messages.

This paper presents a method of language model adaptation for call-center conversations using automatic speech recognition (ASR) transcripts and their confidence scores. The goal is to select the optimal adaptation set by estimating the recognition errors and minimizing the adaptation language model (LM) perplexity. ASR transcripts are ranked with respect to their confidence scores and adaptation data selection is done iteratively by filtering the most reliable transcript set that minimizes the LM perplexity. Model adaptation is then carried out by interpolating the selected adaptation LM with the baseline in-domain LM. We have evaluated our approach on agent speech of real call-center conversations and experiments show that 4% relative word error rate reduction is achieved with the proposed approach.

Cepstral mean subtraction (CMS) is a well-known feature domain channel compensation technique employed to eliminate the effects of convolutive channel distortion. However, as the authors use in log-spectral mean subtraction (LSMS), the compensation might be applied in spectral domain before the filter-bank analysis with a higher-frequency resolution. LSMS can also be combined with CMS to further improve the recognition performance. In this study, the authors compare the performances of LSMS and CMS methods using a multi-channel, text-dependent single utterance speaker recognition database. In the experiments, the authors observe that LSMS outperforms CMS especially in the high false acceptance region. Moreover, the accuracy is further improved when the methods are combined together. With the combination, the authors achieve 15.5% relative reduction in equal error rate for no score normalisation and 9.4% for test normalisation cases when compared with the baseline CMS experiment.

Automatic speech recognition accuracies of call-canter conversations are still below intended levels due to harsh conditions such as channel distortions, external noises, coarticulated speech, etc. Agglutinative and free word order nature of Turkish degrades the recognition performances further; therefore the usage of confidence measures (CMs) is inevitable to retrieve correct information from the calls. In this paper, two conversational CMs, namely speech overlap ratio and opposite party energy level, are proposed, and tested together with single-channel confidence measures on Turkish stereo call center recordings. Experimental results show that conversational CMs improve the rating accuracies of the utterances with respect to their recognition rates.

Voice conversion enables generation of a desired speaker's voice from audio recordings of another speaker. In this paper, we focus on a music application and describe the first steps towards generating voices of music celebrities using conventional voice conversion techniques. Specifically, rap singing transformations from English to Spanish are performed using parallel training material in English. Weighted codebook mapping based voice conversion with two different alignment methods and temporal smoothing of the transformation filter are employed. The first aligner uses a HMM trained for each source recording to force-align the corresponding target recording. The second aligner employs speaker-independent HMMs trained from a large number of speakers. Additionally, a smoothing step is devised to reduce discontinuities and to improve performance. The results of subjective evaluations indicate that both aligners perform equivalently well. The proposed smoothing technique improves both similarity to target singer and quality significantly regardless of the alignment method.

Synthesizing desired emotions using concatenative algorithms relies on collection of large databases. This paper focuses on the development and assessment of a simple algorithm to interpolate the intended vocal effort in existing databases in order to create new databases with intermediate levels of vocal effort. Three diphone databases in German with soft, modal, and loud voice qualities are processed with a spectral interpolation algorithm. A listening test is performed to evaluate the intended vocal effort in the original databases as well as the interpolated ones. The results show that the interpolation algorithm can create the intended intermediate levels of vocal effort given the original databases independent of the language background of the subjects. 1.

This study and transformation of the vocal tract spectrum and the pitch contour. The first method (selective pre-emphasis) relies on band-pass filtering to perform vocal tract transformation. The second method (segmental pitch contour model) focuses on a more detailed modeling of pitch contours. Both methods are utilized in the design of a voice conversion algorithm based on codebook mapping. We compare them with existing vocal tract and pitch contour transformation methods and acoustic feature transplantations in subjective tests. The performance of the selective pre-emphasis based method is similar to the methods used in our previous work at higher sampling rates with a lower prediction order. The results also indicate that the segmental pitch contour model improves voice conversion performance. 1 proposes two new methods for detailed modeling 1.

Perception of speaker identity is an important characteristic of the human auditory system. This paper describes a subjective test for the investigation of the relevance of four acoustic features in this process: vocal tract, pitch, duration, and energy. PSOLA based methods provide the framework for the transplantations of these acoustic features between two speakers. The test database consists of different combinations of transplantation outputs obtained from a database of 8 speakers. Subjective decisions on speaker similarity indicate that the vocal tract is the most relevant feature for single feature transplantations. Pitch and duration possess similar significance whereas the energy is the least important acoustic feature. Vocal tract + pitch + duration transplantation results in the highest similarity to the target speaker. Vocal tract + pitch, vocal tract + duration + energy and vocal tract + duration transplantations also yield convincing results in transformation of the perceived speaker identity.

In this study, we describe and evaluate a subjective test to determine the importance of subbands in perceiving the speaker identity. Pairs of sound files have been generated where the first sound file is the full-band version of a word and the second file is a subband of the same word in the first part and a subband of a different word in the second part of the experimental work. Words in each pair are uttered by either the same speaker or different speakers. S ubjects were asked to score each pair of sound files they listened to according to the similarity of the speakers uttering each word. Also a consistency score is calculated for each subject to render the statistics more reliable. The experiment demonstrates that the second subband (1034 Hz-1895 Hz range) is the most important subband for speaker identification.

A new voice conversion method that improves the quality of the voice conversion output at higher sampling rates is proposed. Speaker Transformation Algorithm Using Segmental Codebooks (STASC) is modified to process source and target speech spectra in different subbands. The new method ensures better conversion at sampling rates above 16KHz. Discrete Wavelet Transform (DWT) is employed for subband decomposition to estimate the speech spectrum better with higher resolution. Faster voice conversion is achieved since the computational complexity decreases at a lower sampling rate. A Voice Conversion System (VCS) is implemented using the proposed algorithm with necessary tools. The performance of the proposed method is demonstrated by both subjective listening tests and applications to film dubbing and looping. In ABX listening tests, the listeners preferred the subband based output by 92.1% as compared to the full-band based output.

A classification system that aims to recognize the presence of sounds from different sources is described. The type of audio signals considered are speech, music, noise and silence. Appropriate subband processing is applied for the characterization of each sound source. The algorithm operates in four steps to classify the contents of a given audio signal. The acoustical parameters and statistical measures to be used in the classification process are obtained via an off-line training procedure. In the silence and onset detection stages, we aim to label the starting and finishing instants of the acoustical events present in the audio signal. Acoustical parameters of the given signal are extracted, analysis of variance and classification using the LBG algorithm is carried out by generating codebooks of acoustical vectors. Experimental work is carded out on a database that contains mixtures of human speech, music, noise and silence. The experiments demonstrate that the system achieves 88% classification success on the average when sounds from different sources are presented non-simultaneously.

Automatic speech recognition systems are prone to errors when there are confusable words in the dictionary. Even human beings sometimes make errors when they have to choose between words like "fix" and "six". The situation is worse for telephone conversation. Most unvoiced phonemes with low energy are lost in the background noise. In this paper, a new approach to the solution of this problem is proposed. The idea is to create a new language with words that are orthogonal to each other in the acoustic space. The suggestion in this paper is to limit the vocabulary of the new language to include only a few very essential words (i.e., digits, yes-no, etc.). In such a case the users may have the option of learning 10-20 words in the new language and get better service in return, which may be preferred by some part of the population.

Traditional maximum likelihood estimation of hidden Markov model parameters aims at maximizing the overall probability across the training tokens of a given speech unit. As such, it disregards any interaction or biases across the models in the training procedure. Often, the resulting model parameters do not result in minimum error classification in the training set. A new selective training method is proposed that controls the influence of outliers in the training data on the generated models. The resulting models are shown to possess feature statistics which are more clearly separated for confusable patterns. The proposed selective training procedure is used for hidden Markov model training, with application to foreign accent classification, language identification, and speech recognition using the E-set alphabet. The resulting error rates are measurably improved over traditional forward-backward training under open test conditions. The proposed method is similar in terms of its goal to maximum mutual information estimation training, however it requires less computation, and the convergence properties of maximum likelihood estimation are retained in the new formulation.

This paper presents several improvements to our voice conversion system which we refer to as speaker transformation algorithm using segmental codebooks (STASC). First, a new concept, sentence HMM, is introduced for the alignment of speech waveforms sharing the same text. This alignment technique allows reliable and high resolution mapping between two speech waveforms. In addition, it is observed that energy and speaking rate differences between two speakers are not constant across all phonemes. Therefore a codebook based duration and energy scaling algorithm is proposed. Finally, a more detailed pitch modification is introduced that takes into account pitch range differences between source and target speakers in addition to mean pitch level differences. The proposed changes made a significant impact on the quality of transformed speech. Subjective listening tests showed that intelligibility is maintained at the same level as natural speech after the speaker transformation.

This paper presents a novel algorithm which generates three-dimensional face point trajectories for a given speech file with or without its text. The proposed algorithm first employs an off-line training phase. In this phase, recorded face point trajectories along with their speech data and phonetic labels are used to generate phonetic codebooks. These codebooks consist of both acoustic and visual features. Acoustics are represented by line spectral frequencies (LSF), and face points are represented with their principal components (PC). During the synthesis stage, speech input is rated in terms of its similarity to the codebook entries. Based on the similarity, each codebook entry is assigned a weighting coefficient. If the phonetic information about the test speech is available, this is utilized in restricting the codebook search to only several codebook entries which are visually closest to the current phoneme (a visual phoneme similarity matrix is generated for this purpose). Then t...

In maximum likelihood (ML) estimation of hidden Markov models (HMMs) for speech recognition, the criterion is to maximize the total probability across the training data for a particular speech unit, such as a word, monophone, diphone, or triphone. Since each unit model is trained separately, such a strategy can often lead to biases among decision boundaries of the generated model set. In this correspondence, we propose a new technique to minimize the total number of misclassifications in the training data set by adjusting the decision boundaries between HMM pairs. The proposed algorithm is shown to reduce the error rate in a number of speech recognition tasks such as accent detection, language identification, and confusable word pair discrimination. The technique is also attractive because it is simple to implement and the improvement in performance is achieved without any added complexity in the decoding phase.

In this study, the frequency characteristics of foreign accented speech is investigated. Experiments are conducted to discover the relative significance of different resonant frequencies and frequency bands in terms of their accent discrimination ability. It is shown that second and third formants are more important than other resonant frequencies. A filter bank analysis of accented speech supports this statement, where the 1500-2500 Hz range was shown to be the most significant frequency range in discriminating accented speech. Based on these results, a new frequency scale is proposed in place of the commonly used Mel-scale to extract the cepstrum coefficients from the speech signal. The proposed scale results in better performance for the problems of accent classification and language identification.

This paper presents a new scheme for developing a voice conversion system that modifies the utterance of a source speaker to sound like speech from a target speaker. We refer to the method as Speaker Transformation Algorithm using Segmental Codebooks (STASC). Two new methods are described to perform the transformation of vocal tract and glottal excitation characteristics across speakers. In addition, the source speaker's general prosodic characteristics are modified using time-scale and pitch-scale modification algorithms. Informal listening tests suggest that convincing voice conversion is achieved while maintaining high speech quality. The performance of the proposed system is also evaluated on a standard Gaussian mixture model based speaker identification system, and the results show that the transformed speech is assigned higher likelihood by the target speaker model when compared to the source model. 1 Introduction There has been a considerable amount of research effort directed...

Speech signals are often degraded by additive interference over single-channel communication systems. For stationary and well defined noise sources, effective solutions exist. However, it is often difficult to formulate a model for nonstationary and speechlike noise sources such as crosstalk or multispeaker babble, which exist in real scenarios. In this paper, we propose a solution to this problem under the assumption that we have access to the clean speech signal prior to transmission. A novel method for tracking transmission noise characteristics is described. Based on this noise estimate, a new speech enhancement technique is proposed. The enhancement method is evaluated for multispeaker babble noise, and shown to substantially improve both the quality and intelligibility of the processed speech signal.

We propose two methods to improve HMM speech recognition performance. The first method employs an adjustment in the training stage, whereas the second method employs it in the scoring stage. It is well known that a speech recognition system performance increases when the amount of labeled training data is large. However, due to factors such as inaccurate phonetic labeling, end-point detection, and voiced-unvoiced decisions, the labeling procedure can be prone to errors. We propose a selective hidden Markov model (HMM) training procedure in order to reduce the adverse influence of atypical training data on the generated models. To demonstrate its usefulness, selective training is applied to the problem of accent classification, resulting in a 9.4% improvement in classification error rate. The second goal is to improve HMM scoring performance. The objective of HMM training algorithms is to maximize the probability over the training tokens for each model. However, this does not guarantee a minimized error rate across the entire model set. Typically, biases in the confusion matrices can be observed. We propose a method for estimating the bias from input training data, and incorporating it into the general scoring algorithm. Using this technique, a 9.8% improvement is achieved in accent classification error rate.

We propose three new adaptive noise suppression algorithms for enhancing noise-corrupted speech: smoothed spectral subtraction (SSS), vector quantization of line spectral frequencies (VQ-LSF), and modified Wiener filtering (MWF). SSS is an improved version of the well-known spectra subtraction algorithm, while the other two methods are based on generalised Wiener filtering. We have compared these three algorithms with each other and with spectral subtraction on both simulated noise and actual car noise. All three proposed methods perform substantially better than spectral subtraction, primarily because of the absence of any musical noise artifacts in the processed speech. Listening tests showed preference for MWF and SSS over VQ-LSF. Also, MWF provides a much higher mean opinion score (MOS) than does spectral subtraction. Finally, VQ-LSF provides a relatively good spectral match to the clean speech, and may, therefore, be better suited for speech recognition.

Speaker accent is an important issue in the formulation of robust speaker independent recognition systems. Knowledge gained from a reliable accent classification approach could improve overall recognition performance. In this paper, a new algorithm is proposed for foreign accent classification of American English. A series of experimental studies are considered which focus on establishing how speech production is varied to convey accent. The proposed method uses a source generator framework, recently proposed for analysis and recognition of speech under stress [5]. An accent sensitive database is established using speakers of American English with foreign language accents. An initial version of the classification algorithm classified speaker accent from among four different accents with an accuracy of 81.5% in the case of unknown text, and 88.9% assuming known text. Finally, it is shown that as ascent sensitive word count increases, the ability to correctly classify accent also increases, achieving an overall classification rate of 92% among four accent classes.

Research has shown that degrading acoustic background noise influences speech quality across phoneme classes in a nonuniform manner. This results in variable quality performance of many speech enhancement algorithms in noisy environments. A phoneme classification procedure is proposed which directs single-channel constrained speech enhancement. The procedure performs broad phoneme class partitioning of noisy speech frames using a continuous mixture hidden Markov model recognizer in conjunction with a perceptually motivated cost-based decision process. Once noisy speech frames are identified, iterative speech enhancement based on all-pole parameter estimation with inter- and intra-frame spectral constraints is employed. The phoneme class-directed enhancement algorithm is evaluated using TIMIT speech data and shown to result in substantial improvement in objective speech quality over a range of signal-to-noise ratios and individual phoneme classes.>

The introduction of acoustic background distortion into speech causes recognition algorithms to fail. In order to improve the environmental robustness of speech recognition in adverse conditions, a novel constrained-iterative feature-estimation algorithm is considered and shown to produce improved feature characterization in a variety of actual noise conditions. In addition, an objective measure based MAP estimator is formulated as a means of predicting changes in robust recognition performance at the speech feature extraction stage. The four measures considered include (i) NIST SNR; (ii) Itakura-Saito log-likelihood; (iii) log-area-ratio; (iv) the weighted-spectral slope measure. A continuous distribution, monophone based, hidden Markov model recognition algorithm is used for objective measure based MAP estimator analysis and recognition evaluation. Evaluations were based on speech data from the Credit Card corpus (CC-DATA). It is shown that feature enhancement provides a consistent level of recognition improvement for broadband, and low-frequency colored noise sources. As the stationarity assumption for a given noise source breaks down, the ability of feature enhancement to improve recognition performance decreases. Finally, the log-likelihood based MAP estimator was found to be the best predictor of recognition performance, while the NIST SNR based MAP estimator was found to be poorest recognition predictor across the 27 noise conditions considered.>

It is known that degrading acoustic noise influences speech quality across phoneme classes in a non-uniform manner. This results in variable quality performance for many speech enhancement algorithms in noisy environments. To address this, a hidden-Markov-mode phoneme classification procedure is proposed which directs single channel speech enhancement across individual phoneme classes. The procedure performs broad phoneme class partitioning of noisy speech frames using a continuous-mixture hidden-Markov-model recognizer in conjunction with a cost based decision process. Cost functions are assigned which weigh errors between phoneme classes that are perceptually different (e.g., vowels versus fricatives, etc.). Once noisy speech frames are partitioned, iterative speech enhancement based on all-pole parameter estimation with inter and intra-frame spectral constraints (Auto:I,LSP:T) is employed. The phoneme class directed enhancement algorithm is evaluated using TIMIT speech data, and shown to result in substantial improvement in objective speech quality over a range of signal-to-noise ratios and individual phoneme classes. The algorithm is also shown to possess consistent quality improvement in a speaker independent scenario.>