De-identification of data used for automatic speech recognition modeling is a critical component in protecting privacy, especially in the medical domain.However, simply removing all personally identifiable information (PII) from end-to-end model training data leads to a significant performance degradation in particular for the recognition of names, dates, locations, and words from similar categories.We propose and evaluate a two-step method for partially recovering this loss.First, PII is identified, and each occurrence is replaced with a random word sequence of the same category.Then, corresponding audio is produced via text-to-speech or by splicing together matching audio fragments extracted from the corpus.These artificial audio/label pairs, together with speaker turns from the original data without PII, are used to train models.We evaluate the performance of this method on in-house data of medical conversations and observe a recovery of almost the entire performance degradation in the general word error rate while still maintaining a strong diarization performance.Our main focus is the improvement of recall and precision in the recognition of PII-related words.Depending on the PII category, between 50% -90% of the performance degradation can be recovered using our proposed method.

This paper investigates the impact of sub space based techniques for acoustic modeling in automatic speech recognition (ASR). There are many well known approaches to subspace based speaker adaptation which represent sources of variability as a projection within a low dimensional subspace. A new approach to acoustic modeling in ASR, referred to as the subspace based Gaussian mixture model (SGMM), represents phonetic variability as a set of projections applied at the state level in a hidden Markov model (HMM) based acoustic model. The impact of the SGMM in modeling these intrinsic sources of variability is evaluated for a continuous speech recognition (CSR) task. The SGMM is shown to provide an 18% reduction in word error rate (WER) for speaker independent (SI) ASR relative to the continuous density HMM (CDHMM) in the resource management CSR domain. The SI performance obtained from SGMM also represents a 5% reduction in WER relative to subspace based speaker adaption in an unsupervised speaker adaptation scenario.

Techniques are presented for detecting phoneme level mispronunciations in utterances obtained from a population of impaired children speakers. The intended application of these approaches is to use the resulting confidence measures to provide feedback to patients concerning the quality of pronunciations in utterances arising within interactive speech therapy sessions. The pronunciation verification scenario involves presenting utterances of known words to a phonetic decoder and generating confusion networks from the resulting phone lattices. Confidence measures are derived from the posterior probabilities obtained from the confusion networks. Phoneme level mispronunciation detection performance was significantly improved with respect to a baseline system by optimizing acoustic models and pronunciation models in the phonetic decoder and applying a nonlinear mapping to the confusion network posteriors.

This paper deals with the interaction between progressive model adaptation and score normalization strategies which are used for reducing the variation in likelihood ratio scores in making speaker verification decisions. This issue is important in establishing robust decision thresholds for practical speaker verification systems. An adaptive score normalization method is proposed that is designed to reduce the drift in likelihood ratio scores that occurs when speaker models are adapted. This method is investigated and compared with other more well know score normalization methods in the context of a joint factor analysis speaker verification approach. All approaches are evaluated on the progressive adaptation track in the NIST 2005 text independent speaker verification evaluation plan.

This paper presents a study of confidence measure based techniques for detecting phoneme level mispronunciations in utterances from impaired children with neuromuscular disorders. Several different adaptation scenarios are investigated to determine the effects of mismatched speaker characteristics and mismatched task domain on the ability to verify the phoneme level pronunciations. These techniques are evaluated in the context of a speech corpus where utterances were elicited from children in interactive speech therapy sessions involving a multimodal game-like environment. Results are presented in terms of phone detection charateristics where, for example, equal error rates of as low as 16.2 % were obtained for detecting instances where phonemes were deleted by impaired speakers. 1.

This paper addresses the issue of speaker variability and session variability in text-independent Gaussian mixture model (GMM)-based speaker verification. A speaker model adaptation procedure is proposed which is based on a joint factor analysis approach to speaker verification. It is shown in this paper that this approach facilitates the implementation of a progressive unsupervised adaptation strategy which is able to produce an improved model of speaker identity while minimizing the influence of channel variability. The paper also deals with the interaction between this model adaptation approach and score normalization strategies which act to reduce the variation in likelihood ratio scores. This issue is particularly important in establishing decision thresholds in practical speaker verification systems since the variability of likelihood ratio scores can increase as a result of progressive model adaptation. These adaptation methods have been evaluated under the adaptation paradigm defined under the NIST 2005 Speaker Recognition Evaluation Plan, which is based on conversation sides derived from telephone speech utterances. It was found that when target speaker models were trained from a single conversation, an equal error rate (EER) of 4.5% was obtained under the NIST unsupervised speaker adaptation scenario.