This paper presents a complete framework for articulatory inversion based on jump Markov linear systems (JMLS). In the model, the acoustic measurements and the position of each articulator are considered as observable measurement and continuous-valued hidden state of the system, respectively, and discrete regimes of the system are represented by the use of a discrete-valued hidden modal state. Articulatory inversion based on JMLS involves learning the model parameter set of the system and making inference about the state (position of each articulator) of the system using acoustic measurements. Iterative learning algorithms based on maximum-likelihood (ML) and maximum a posteriori (MAP) criteria are proposed to learn the model parameter set of the JMLS. It is shown that the learning procedure of the JMLS is a generalized version of hidden Markov model (HMM) training when both acoustic and articulatory data are given. In this paper, it is shown that the MAP-based learning algorithm improves modeling performance of the system and gives significantly better results compared to ML. The inference stage of the proposed algorithm is based on an interacting multiple models (IMM) approach, and done online (filtering), and/or offline (smoothing). Formulas are provided for IMM-based JMLS smoothing. It is shown that smoothing significantly improves the performance of articulatory inversion compared to filtering. Several experiments are conducted with the MOCHA database to show the performance of the proposed method. Comparison of the performance of the proposed method with the ones given in the literature shows that the proposed method improves the performance of state space approaches, making state space approaches comparable to the best published results.

This paper presents a detailed framework for Gaussian mixture model (GMM)-based articulatory inversion equipped with special postprocessing smoothers, and with the capability to perform audio-visual information fusion. The effects of different acoustic features on the GMM inversion performance are investigated and it is shown that the integration of various types of acoustic (and visual) features improves the performance of the articulatory inversion process. Dynamic Kalman smoothers are proposed to adapt the cutoff frequency of the smoother to data and noise characteristics; Kalman smoothers also enable the incorporation of auxiliary information such as phonetic transcriptions to improve articulatory estimation. Two types of dynamic Kalman smoothers are introduced: global Kalman (GK) and phoneme-based Kalman (PBK). The same dynamic model is used for all phonemes in the GK smoother; it is shown that GK improves the performance of articulatory inversion better than the conventional low-pass (LP) smoother. However, the PBK smoother, which uses one dynamic model for each phoneme, gives significantly better results than the GK smoother. Different methodologies to fuse the audio and visual information are examined. A novel modified late fusion algorithm, designed to consider the observability degree of the articulators, is shown to give better results than either the early or the late fusion methods. Extensive experimental studies are conducted with the MOCHA database to illustrate the performance gains obtained by the proposed algorithms. The average RMS error and correlation coefficient between the true (measured) and the estimated articulatory trajectories are 1.227 mm and 0.868 using audiovisual information fusion and GK smoothing, and 1.199 mm and 0.876 using audiovisual information fusion together with PBK smoothing based on a phonetic transcription of the utterance.

A feature of minimizing images of submodular binary Markov random field energies is introduced. Using this novel feature, the collection of minimizing images of levels of higher order, monotonically levelable multi label MRF energies is shown to constitute a monotone collection. This implies that these minimizing binary images can be combined to give minimizing images of the multi label MRF energies. Thanks to the graph cuts framework, the mentioned class of binary MRF energies is known to be minimized by maximum flow computations on appropriately constructed graphs. With the aid of these developments an exact and efficient algorithm to minimize monotonically levelable multi label MRF energies of any order, which is composed of a series of maximum flow computations, is proposed and an application of the proposed algorithm to image denoising is given.

This work examines the utility of formant frequencies and their energies in acoustic-to-articulatory inversion. For this purpose, formant frequencies and formant spectral amplitudes are automatically estimated from audio, and are treated as observations for the purpose of estimating electromagnetic articulography (EMA) coil positions. A mixture Gaussian regression model with mel-frequency cepstral (MFCC) observations is modified by using formants and energies to either replace or augment the MFCC observation vector. The augmented observation results in 3.4 % lower RMS error, and 2.7 % higher correlation coefficient, than the baseline MFCC observation. Improvement is especially good for plosive consonants, possibly because formant tracking provides information about the acoustic resonances that would be otherwise unavailable during plosive closure and release. Index Terms: acoustic-to-articulatory inversion, formant tracking, GMM regression

In this paper, we propose a new approach for dynamic speech spectrum representation and tracking vocal tract resonance (VTR) frequencies. The method involves representing the spectral density of the speech signals as a mixture of Gaussians with unknown number of components for which time-varying Dirichlet process mixture model (DPM) is utilized. In the resulting representation, the number of formants is allowed to vary in time. The paper first presents an analysis on the continuity of the formants in the spectrum during the speech utterance. The analysis is based on a new state space representation of concatenated tube model. We show that the number of formants which appear in the spectrum is directly related to the location of the constriction of the vocal tract (i.e., the location of the excitation). Moreover, the disappearance of the formants in the spectrum is explained by “uncontrollable modes” of the state space model. Under the assumption of existence of varying number of formants in the spectrum, we propose the use of a DPM model based multi-target tracking algorithm for tracking unknown number of formants. The tracking algorithm defines a hierarchical Bayesian model for the unknown formant states and the inference is done via Rao–Blackwellized particle filter.

This paper presents a systematic framework for accurate estimation of vocal tract resonances (formants) using neither training data nor a phonetic transcription. In the proposed method, the speech signal is segmented in voiced and unvoiced parts and the resonance frequencies of the vocal tract are estimated by dynamic programming and further processed by using Kalman filtering/smoothing for each part. The performance of the proposed method is compared with three different methods which are baseline, WaveSurfer and MSR. The proposed method reduces the overall vocal tract resonances (for F1, F2 and F3) estimation error rate by 35%, 39.6% and 2.74% over the baseline, WaveSurfer and MSR methods respectively.

This paper describes recent work towards development of new corpora and tools for Turkish speech research. This effort represents an on-going collaboration between the Center for

This paper presents a comparison of objective functions for optimally combining different speaker identification systems. The comparison is based on the classification performance of the resultant multiple classifier system (MCS). The objective functions considered are; classification figure of merit (CFM), mean square error (MSE) and cross entropy (CE). In all three methods, the outputs of individual classifiers assumed to be the posterior probabilities of each speaker and linear combination of the output vectors are considered. CFM seeks to maximize the difference between the output value of the speaker and the output values of all other incorrect speakers. On the other hand, MSE and CE compare the outputs with some ideal vectors where the output of the correct speaker is set to one and the others are zero. The experimental results are also compared with the averaging method where the combination is not optimized. Our simulation experiments on four different sets of speakers show that CFM performs better compared to the other objective functions.

In this paper, we propose a novel rank-based classifier combination scheme under uncertainty for speaker identification. The combination is based on a heuristic method that uses Dempster-Shafer theory of evidence under some conditions. The method is based on the extraction of first and R/sup th/ level ranking statistics. Using these statistics, the pattern classes are clustered into model sets where the classes in these sets share set specific properties. Some of these model sets are used to reflect the strengths and weaknesses where some others carry class dependent ranking statistics of the corresponding classifier. The experiments conducted on the Polycost database have shown that the proposed approach is more effective compared to some other rank-based combination schemes.

Presents the work done to improve the recognition rate in an isolated word recognition problem with single utterance training. The negative effect of errors (due to insufficient training data) in estimated model parameters is compensated by fusing the information obtained from HMM evaluation and those generated for the word length and voicing at the beginning and end of the word. A Bayesian causal tree structure is developed to accomplish the fusion. The final decision is made on one of the three candidates which are most likely according to HMM evaluation. The reliability of the HMM ordering is improved by applying variance flooring.

This study presents a theoretical investigation of the rank-based multiple classifier decision problem for closed-set pattern classification. The case with classifier raw outputs in the form of candidate class rankings is considered and formulated as a discrete optimization problem with the objective function being the total probability of correct decision. The problem has a global optimum solution but is of prohibitive dimensionality. We present a partitioning formalism under which this dimensionality can be reduced by incorporating our prior knowledge about the problem domain and the structure of the training data. The formalism can effectively explain a number of rank-based combination approaches successfully used in the literature, one of which is discussed.

This paper introduces the use of genetics-based algorithm in the reduction of 24 parameter set (i.e the base set) to a 5,6,7,8 or 10 parameter set, for each speaker in text-independent speaker identification. The feature selection is done by finding the best features that discriminates a person from his/her two closest neighbors. The experimental results show that there is approximately 5% increase in the recognition rate when the reduced set of parameters are used. Also the amount of calculation necessary for speaker recognition using the reduced set of features is much less than the amount of calculation required using the complete feature set in the testing phase. Hence it is more desirable to use the subset of the complete feature set found using the genetic algorithm suggested.

In this paper, a fixed point variable bit-rate (VBR) mixed excitation linear predictive coding (MELP/sup TM/) vocoder is presented. The VBR-MELP vocoder is also implemented on a TMS320C54x and it achieves virtually indistinguishable federal standard MELP quality at bit-rates between 1.0 to 1.6 kb/s. The backbone of VBR-MELP vocoder is similar to that of federal standard MELP. It utilizes a novel sub-band based voice activity detector in the back-end of encoder to discriminate background noise from speech activity. Since proposed detector uses only parameters extracted in the encoder, its computational complexity is very low.

In this study, closed set, text-independent speaker identification is considered and the problem of improving the reliability of the decisions made by available algorithms is addressed. The work presented is based on the idea of combining the evidence from different algorithms or decision strategies to improve the recognition performance and the reliability. For this purpose, the models generated by a single algorithm for 17 speakers from the SPIDRE database are considered and a matrix of speaker-to-model fitness values is processed by two different decision strategies. Ideas from the mathematical theory of evidence are applied to combine the decisions produced by these two strategies to generate a better decision on the speaker identity. The combined decision show an improved degree of correctness hence suggesting a promising way of combining the decisions from partially successful algorithms.