In this study, we evaluate and compare two different approaches for multilingual phone recognition in code-switched and non-code-switched scenarios. First approach is a front-end Language Identification (LID)-switched to a monolingual phone recognizer (LID-Mono), trained individually on each of the languages present in multilingual dataset. In the second approach, a common multilingual phone-set derived from the International Phonetic Alphabet (IPA) transcription of the multilingual dataset is used to develop a Multilingual Phone Recognition System (Multi-PRS). The bilingual code-switching experiments are conducted using Kannada and Urdu languages. In the first approach, LID is performed using the state-of-the-art i-vectors. Both monolingual and multilingual phone recognition systems are trained using Deep Neural Networks. The performance of LID-Mono and Multi-PRS approaches are compared and analysed in detail. It is found that the performance of Multi-PRS approach is superior compared to more conventional LID-Mono approach in both code-switched and non-code-switched scenarios. For code-switched speech, the effect of length of segments (that are used to perform LID) on the performance of LID-Mono system is studied by varying the window size from 500 ms to 5.0 s, and full utterance. The LID-Mono approach heavily depends on the accuracy of the LID system and the LID errors cannot be recovered. But, the Multi-PRS system by virtue of not having to do a front-end LID switching and designed based on the common multilingual phone-set derived from several languages, is not constrained by the accuracy of the LID system, and hence performs effectively on code-switched and non-code-switched speech, offering low Phone Error Rates than the LID-Mono system.

We address the problem of efficient acoustic-model refinement (continuous retraining) using semi-supervised and active learning for a low resource Indian language, wherein the low resource constraints are having i) a small labeled corpus from which to train a baseline `seed' acoustic model and ii) a large training corpus without orthographic labeling or from which to perform a data selection for manual labeling at low costs. The proposed semi-supervised learning decodes the unlabeled large training corpus using the seed model and through various protocols, selects the decoded utterances with high reliability using confidence levels (that correlate to the WER of the decoded utterances) and iterative bootstrapping. The proposed active learning protocol uses confidence level based metric to select the decoded utterances from the large unlabeled corpus for further labeling. The semi-supervised learning protocols can offer a WER reduction, from a poorly trained seed model, by as much as 50% of the best WER-reduction realizable from the seed model's WER, if the large corpus were labeled and used for acoustic-model training. The active learning protocols allow that only 60% of the entire training corpus be manually labeled, to reach the same performance as the entire data.

We address the problem of automatic segmentation of the unit database in unit-selection based TTS and propose template based forced alignment segmentation in the one-pass dynamic programming (DP) framework with several variants: i) multi-template representation derived by modified K-means (MKM) algorithm, ii) context-independent and context-dependent templates for reduced multi-template representation, iii) segmental K-means algorithm with MKM modeling of phone classes, as a template-based equivalent of the conventional embedded re-estimation procedure for HMM based modeling and segmentation, that is typical for deriving unit-databases for TTS (e.g. EHMM in Festival). We first benchmark the performance of the proposed segmentation framework on TIMIT database for phonetic segmentation given the availability of phonetic labeling ground truth in TIMIT. We then apply the proposed template based segmentation algorithms for syllabic Indian language TTS, and benchmark the proposed segmentation using objective measures based on spectral distortions (SD) obtained on time-aligned speech utterances and compare it with other recent segmentation approaches, namely the group-delay (GD) based semiautomatic method, Hybrid method, EHMM, HMM and SKM-HMM and show that the proposed template based approaches offer comparable and better spectral distortions, validating their ability to provide accurate high-resolution segmentation of the unit-database.

We address the problem of audio indexing for a class of special-case scenarios, where it is required to index an audio stream into 2 classes, namely, a target class and a background class, as arising, say in, audio surveillance and machine diagnostics. With the emphasis on dealing with limited training exemplars defining the target class in these scenarios, we propose a 2-class `audio verification' framework, where the target and background classes are modeled by GMMs and the indexing is done via a sliding window based detection. We characterize the performance of the system in terms of ROCs, EERs and visual detection plots for a set of 2 target classes and 4 background classes from a surveillance audio database and show the viability of such a system in practical applications. We highlight the robustness of the system to high levels of background-class using visual detection plots of continuous audio streams at SNRs ranging from 30 dB down to -20 dB.

We address the problem of audio analytics with respect to efficient modeling of audio classes and continuous decoding of audio stream to automatically segment and label the audio stream as required in audio indexing. We propose the use of left-to-right HMMs and ergodic HMMs to respectively model definite and indefinite duration audio classes and Viterbi decoding using these HMMs with non-emitting states for continuous decoding of audio streams. We quantify the decoding performance using detection and false-alarm rates and show that the proposed HMM based modeling and Viterbi decoding can have high decoding accuracies with average (%Hit, %False-alarm) of (79.2%, 1.6%), which are significantly better than VQ, GMM and Template based decoding, indicating the viability of the proposed modeling and decoding technique for practical surveillance audio analytics.

We consider the class of segment quantizers used for low to ultralow rate speech coding, namely, vector quantization (VQ), matrix quantization (MQ), variable-length segment quantization (VLSQ) and two more recent unit-selection based segment quantization algorithms which represent an important shift in using large unclustered continuous codebooks in contrast to the conventional clustered codebooks of VQ, MQ and VLSQ. We examine the advantage, if any, in this shift from small clustered codebooks of VQ, MQ and VLSQ ( 10-14 bits/segment), to the larger continuous unit databases ( 16-18 bits/segment) in the unit-selection framework, by comparing their rate-distortion curves. We show that while the conventional VQ, MQ have higher distortions and VLSQ satuarates in its distortion reduction, unit-selection algorithms provide lower distortions and steeper reductions at marginally low increase in bit-rates and justifies exploring their potential further.

We propose a novel framework for continuous speech recognition (CSR) based on non-parametric acoustic modeling using multiple phoneme templates set in a modified one-pass DP decoding algorithm, in contrast to the conventional HMM acoustic models set in Viterbi decoding. We particularly emphasis the 'selectivity' property of templates as set in the proposed modified one-pass DP decoding algorithm and explore various contextual definitions of the templates and their relative performances for a range of small vocabulary tasks with TIMIT database using only acoustic models. Based on this, we show that the proposed framework based on phoneme template modeling is a viable means for CSR with potential for interesting issues in acoustic modeling and decoding strategies, particularly in the paradigmatically novel framework of model-free CSR.

The problem of the effect of accent on the performance of Automatic Speech Recognition (ASR) systems is well known. In this paper, we study the effect of accent variability on the performance of the Indian English ASR task. We evaluate the test vocabularies on HMMs trained on (a) Accent specific training data (b) Accent pooled training data which combines all the accent specific training data (c) Accent pooled training data of reduced size matching the size of the accent specific training data. We demonstrate that the accent pooled training set performs the best on phonetically rich isolated word recognition task. But the accent specific HMMs perform better than the reduced accent pooled HMMs, indicating a possible approach of using a first stage accent identification to choose the correct accent trained HMMs for further recognition.

In this paper, we first analyze an algorithm proposed recently by Lee and Cox, 2002, which attempts to perform a segment unit-selection for segmental quantization and show how it intrinsically suffers from several sub-optimalities. We then propose a generalized unit-selection algorithm for ultra low bit-rate segment quantization based on a modified one-pass dynamic programming algorithm. We show that this proposed algorithm is exactly optimal for both fixed and variable-length segments and also how it solves the sub-optimalities of the Lee-Cox-2002 algorithm. From rate-distortion curves from a very large continuous speech multi-speaker database, we show that our algorithm has a significantly superior performance than the algorithm of Lee and Cox by achieving considerably lower spectral distortions (up to 3 dB lower distortions) as well as much lower bit-rates for a given distortion over a range of unit database sizes.

We propose a variable-text text-dependent speaker-recognition system based on the one-pass dynamic programming algorithm. The key feature of our proposed algorithm is its ability to use multiple templates for each of the words which form the "password" text. The use of multiple templates allows the proposed system to capture the idiosyncratic intra-speaker variability of a word, resulting in significant improvement in the performance. Our algorithm also uses inter-word silence templates to handle continuous speech input. Application of the proposed algorithm to two 100-speaker speaker-recognition systems, namely, closed-set speaker-identification (CSI) and speaker-verification (SV), delivers 100% speaker identification accuracy and a speaker-verification EER of 0.09%. The use of multiple templates (in comparison to a single template) enhances the CSI performance from 94% to 100% and the SV EER from 1.6% to 0.09%. Front-end noise suppression enables our systems to deliver robust performance in up to -10 dB car noise

Recently, we established the equivalence of an ergodic HMM (EHMM) to a parallel sub-word recognition (PSWR) framework for language identification (LID). The states of EHMM correspond to acoustic units of a language and its state-transitions represent the bigram language model of unit sequences. We consider two alternatives to represent the state-observation densities of EHMM, namely, the Gaussian mixture model (GMM) and hidden Markov model (HMM). We present a segmental K-means algorithm for the training of both these types of EHMM (EHMM of GMM and EHMM of HMM) and compare their performance on a 6 language LID task in the OGI-TS database. EHMM of GMM has a performance comparable to PSWR and superior to EHMM of HMM; we provide reasons for the performance difference between EHMM(G) and EHMM(H), and identify ways of enhancing the performance of EHMM(H) which is a novel and powerful architecture, ideal for spoken language modeling.

Segment vocoders play a special role in very low bitrate speech coding to achieve intelligible speech at bitrates of /spl sim/300 bits/sec. We explore the definition and use of automatically derived units for segment quantization in segment vocoders. We consider three automatic segmentation techniques, namely, spectral transition measures (STM), maximum-likelihood (ML) segmentation (unconstrained) and duration-constrained ML segmentation, towards defining diphone-like and phone-like units. We show that the ML segmentations realize phone-like units which are significantly better than those obtained by STM in terms of match accuracy with TIMIT phone segmentation as well as actual vocoder performance measured in terms of segmental SNR. Moreover, the phone-like units of ML segmentations also outperform the diphone-like units obtained using STM in early vocoders. We also show that the segment vocoder can operate at very high intelligibility when used in a single-speaker mode.

Parallel sub-word recognition (PSWR) is a new model that has been proposed for language identification (LID) which does not need elaborate phonetic labeling of the speech data in a foreign language. The new approach performs a front-end tokenization in terms of sub-word units which are designed by automatic segmentation, segment clustering and segment HMM modeling. We develop PSWR based LID in a framework similar to the parallel phone recognition (PPR) approach in the literature. This includes a front-end tokenizer and a back-end language model, for each language to be identified. Considering various combinations of the statistical evaluation scores, it is found that PSWR can perform as well as PPR, even with broad acoustic sub-word tokenization, thus making it an efficient alternative to the PPR system.

Automatic segmentation of speech is an important problem that is useful in speech recognition, synthesis and coding. We explore in this paper, the robust parameter set, weighting function and distance measure for reliable segmentation of noisy speech. It is found that the MFCC parameters, successful in speech recognition. holds the best promise for robust segmentation also. We also explored a variety of symmetric and asymmetric weighting lifters. from which it is found that a symmetric lifter of the form 1 + A sin1/2(πn/L), 0 ≤ n ≤ L − 1, for MFCC dimension L, is most effective. With regard to distance measure, the direct L2norm is found adequate.

Previously a modified K-means algorithm for vector quantization design has been proposed where the codevector updating step is as follows: new codevector=current codevector+scale factor (new centroid-current codevector). This algorithm uses a fixed value for the scale factor. In this paper, we propose the use of a variable scale factor which is a function of the iteration number. For the vector quantization of image data, we show that it offers faster convergence than the modified K-means algorithm with a fixed scale factor, without affecting the optimality of the codebook.

In this work, we consider two fast nearest-neighbor search methods based on the projections of Voronoi regions, namely, the box-search method and the cell-partition search method. We provide their comprehensive study in the contest of vector quantization encoding. We show that the use of principal component transformation reduces the complexity of Voronoi-projection based search significantly for data with high degree of correlation across their components.

The authors present an efficient algorithm for fast nearest-neighbour search in multidimensional space under a so called approximation-elimination framework. The algorithm is based on an approximation procedure which selects codevectors for distance computation in the close proximity of the test vector and eliminates codevectors using the triangle inequality based elimination. The algorithm is studied in the context of vector quantization of speech and compared with related algorithms proposed earlier. It is shown to be more efficient in terms of reducing the main search complexity, overhead costs and storage.>

Recently, C.D. Bei and R.M. Gray (1985) used a partial distance search algorithm that reduces the computational complexity of the minimum distortion encoding for vector quantization. The effect of ordering the codevectors on the computational complexity of the algorithm is studied. It is shown that the computational complexity of this algorithm can be reduced further by ordering the codevectors according to the sizes of their corresponding clusters.>