The periodic opening and closing of the vibrating vocal folds changes the production system continuously during the production of voiced speech. The subglottal and supraglottal cavities have distinct structure and impedance. A coupling and decoupling of these cavities affects effective length of the vocal tract which in turn affects its impedance. This can be studied based on the shift in the location of dominant resonant frequencies (DRFs) of the instantaneous system response obtained using zero time windowing (ZTW) method. The change in the impedance of the vocal tract is reflected in the bandwidth of spectral resonances. Bandwidth values estimated using the group delay function at dominant resonance locations, approximated by the Hilbert envelope of the numerator of group delay (HNGD) function. An estimate of this change is made by comparing the bandwidths in closed glottal phase of the glottal cycle for the oral and nasal tracts. A study of the relative difference in the bandwidth of vowels and nasals in speech is attempted to compare the impedance of the respective tracts. The study examines VC/CV segments in English for speakers in the TIMIT database. The results show that the nasal segments exhibit higher bandwidth values compared to vowels.

Most of the applications in speech use mel-frequency spectral coefficients (MFSC) as features as they match the human perceptual mechanism, where the emphasis is given to vocal tract characteristics. But in accent classification, mel-scale distribution of filters may not always be the best representations, e.g., pitch accented languages where the emphasis should be on vocal source information too. Motivated by this, we use end-to-end classification of accents directly from waveforms which will reduce the effort of designing features specific to each corpus. The convolution neural network (CNN) model architecture is designed in such a way that the initial layers exhibit similar operation as in MFSC by initializing the weights using time approximate of MFSC. The entire network along with initial layers is trained to learn accent classification. We observed that learning directly from waveform improved the performance of accent classification when compared to CNN trained on hand-engineered features by 10.94% UAR on the test dataset of common voice corpus. Analyzing the filters after learning, we observed changes in distribution and bandwidths of center frequencies. We further observed the importance of appropriately initializing CNN filters.

Various representations have been developed for acoustic scene classifications (ASC) task using spectral information. However, there is a wide gap in dealing with acoustic scene representations. In this paper, we propose to use a single frequency filtering (SFF) approach, which provides good temporal and spectral resolution at each instant. Single-frequency filtering cepstral coefficients (SFFCC) with deep neural network (DNN) model as the classifier is used for the experimentation on DCASE 2019 and DCASE 2018 Task 1, development data of subtasks A and B. From the conducted experiments on the development datasets, the usage of the SFFCC features significantly improved ASC performance. This approach has got 35thteam rank out of 46 submissions to the corresponding DCASE 2019 Task 1A challenge with a 52.6% classification accuracy on the evaluation dataset. Also, the effect of raw waveforms taken as features for ASC using DNNs was observed.

A fundamental essence for emotional speech analysis towards emotion recognition is a good database. Database collected from natural scenarios consists of spontaneous emotions, but there are several issues in collection of such database. Other than the privacy and legal related concerns, there is no control over environment at the background. As it is difficult to collect data from natural scenarios, many research groups have collected data from semi-natural or designed procedures. In this paper, a new emotional speech database named IIIT-H TEMD (International Institute of Information Technology-Hyderabad Telugu Emotional Database) is collected using designed drama situations from actors and non-actors. Utterances are manually annotated using a hybrid strategy by giving the context to one of the listeners. As some of the data collection studies in the literature recommend for actors, analysis of actors data versus non-actors data is carried out for their significance. The total size of the dataset is about 5 hours, which makes it an useful resource for the emotional speech analysis.

Majority of the text modeling techniques yield only point-estimates of document embeddings and lack in capturing the uncertainty of the estimates. These uncertainties give a notion of how well the embeddings represent a document. We present Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model that learns to represent documents in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. Additionally, in the proposed Bayesian SMM, we address a commonly encountered problem of intractability that appears during variational inference in mixed-logit models. We also present a generative Gaussian linear classifier for topic identification that exploits the uncertainty in document embeddings. Our intrinsic evaluation using perplexity measure shows that the proposed Bayesian SMM fits the unseen test data better as compared to the state-of-the-art neural variational document model on (Fisher) speech and (20Newsgroups) text corpora. Our topic identification experiments show that the proposed systems are robust to over-fitting on unseen test data. The topic ID results show that the proposed model outperforms state-of-the-art unsupervised topic models and achieve comparable results to the state-of-the-art fully supervised discriminative models.

For voiced speech, significant excitation of the vocal tract system is due to sequence of impulse-like events in the vocal fold vibration. Significant excitations take place around the instants of glottal closure (GCIs), which are often referred as epochs. In literature, most of the epoch extraction algorithms rely on estimation of the excitation source, by attempting to remove the vocal tract system characteristics. Performance of these algorithms depends on the effectiveness of excitation source estimation methods. Algorithms such as zero frequency filtering (ZFF) and speech event detection using the residual excitation and a mean-based signal (SEDREAMS) detect epochs from the smoothened speech signal by exploiting temporal cues. But these algorithms rely on the value of pitch period. The objective of this paper is to develop an algorithm which does not require any apriori information of pitch period, and can be implemented in real-time applications. This algorithm exploits the harmonic property of sequence of impulse-like excitations to obtain the local pitch period, and temporal cues in the filtered signal to obtain epoch locations.

The most prevalent multilingual Text-to-Speech (TTS) synthesis systems encounter an unnatural speaker shift at the language boundaries. This is observed when they are employed for code-mixed TTS synthesis. For the very fact that the collection of polyglot speech is non-trivial, many alternative approaches have been in focus. Cross-Lingual Voice Conversion (CLVC) has been one of those to generate speech with desired speaker and language identities. Our aim in this paper is to design a light-weighted CLVC framework between a pair of Mandarin-English speakers. CLVC is challenging when compared to traditional Voice Conversion (VC) because of its nature of accommodating unaligned corpus from the source and target speakers. We thus focus on generating a parallel corpus for CLVC and bridging the gap between speakers and languages. We perform a text-independent voice conversion with a three-layered conventional Neural Network (NN) for this purpose. The main contributions include i) Source similarity in both training and conversion stages of CLVC, ii) generation of a parallel corpus and iii) text independent and transcription free CLVC. We exploit two variants of a Neural Network in the proposed framework, i) an autoencoder to enable the source similarity and generation of parallel corpus, ii) a traditional DNN for feature mapping between the source and target. The subjective and objective evaluations show that the proposed method is indeed capable of performing a CLVC with an auto-encoded speech.

In this work, we explore the combination of MFCC and RCC features for Indian Language identification (LID) task. MFCC represents the vocal tract information while RCC represents the excitation source information, which are complementary to each other. We use Deep neural network (DNN) framework for end to end learning. To combine these two information we uses different fusion techniques such as Min, Max, Average and Min-Max for decision level fusion. We perform experiments on 13 Indian languages data set. We compare our results with state of art I-vector based approach. Min-max based fusion approach gives 9.64% ERR compare to 10.18% of 39D I-vector based approach.

This paper investigates the application of unsupervised acoustic unit discovery for topic identification (topic ID) of spoken audio documents. The acoustic unit discovery method is based on a non-parametric Bayesian phone-loop model that segments a speech utterance into phone-like categories. The discovered phone-like (acoustic) units are further fed into the conventional topic ID framework. Using multilingual bottleneck features for the acoustic unit discovery, we show that the proposed method outperforms other systems that are based on cross-lingual phoneme recognizer.

In this paper, we analyze the performance of various sequence to sequence neural networks on the task of grapheme to phoneme (G2P) conversion. G2P is a very important component in applications like text-to-speech, automatic speech recognition etc,. Because the number of graphemes that a word consists of and the corresponding number of phonemes are different, they are first aligned and then mapped. With the recent advent of sequence to sequence neural networks, the alignment step can be skipped allowing us to directly map the input and output sequences. Although the sequence to sequence neural nets have been applied for this task very recently, there are some questions concerning the architecture that need to be addressed. We show in this paper that, complex recurrent neural network units (like long-short term memory cells) may not be required to achieve good performance on this task. Instead simple recurrent neural networks (RNN) will suffice. We also show that the encoder can be a uni-directional RNN as opposed to the usually preferred bi-directional RNN. Further, our experiments reveal that encoder-decoder models with soft-alignment outperforms fixed vector context counterpart. The results demonstrate that with very few parameters we can indeed achieve comparable performance to much more complicated architectures.

In this paper, we investigate the usage of a continuous representation based approach of the feature vector derived from input text to predict the phone durations in a Text to Speech(TTS) system. We pose the problem of predicting the duration as a data driven statistical transformation from the input text onto the feature space. First we present a method to map both the categorical and numeric features that are typically used into a continuous numeric representation and then model it as a form of Matrix Factorization to improve the representation. The proposed system is evaluated based on Root Mean Squared Error(RMSE) as the objective measure and Mean Opinion Score(MOS) as the subjective measure. We find that the system performs on par with the state of the art duration modeling systems both subjectively and objectively.

Speech-laugh is a speech-synchronous form of laughter that often occurs in natural conversation. However, there are deviations in features of speech-laugh when compared with laughter and neutral speech individually. The objective of this study is to analyse the excitation source features to capture the deviations between laughter and speech-laughs in voiced regions. The features used in this analysis are based on instantaneous fundamental frequency and strength of excitation (β) at epochs. Modified zero frequency filtering (ZFF) method is used to extract the features. Kullback-Leibler (KL) distances obtained show that there are deviations in excitation source features which can be exploited to develop a method to discriminate speech-laughs from laughter. Experimental results show that features used are robust and speaker independent in discriminating speech-laughs from laughter. Results showing deviations of laughter and speech-laughs from neutral speech were also presented.

Studies on the emotion recognition task indicate that there is confusion in discrimination among higher activation states like ‘anger’ and ‘happy’. In this study, features related to excitation source of speech are examined for discriminating ‘anger’ and ‘happy’ emotions. The objective is to explore the features which are independent of lexical content, language, channel and speaker. The features like strength of excitation from zero frequency filtering method and spectral band magnitude energies from short-time spectral analysis are used. Experimental results show that these features can discriminate ‘anger’ and ‘happy’ emotion states to a good extent. Index Terms: Emotion recognition, zero frequency filtering method, KL distance measure.

Reliable acoustic-phonetic (AP) information derived from the speech signal can be used to detect and correct errors in the output of a phone recognizer. In this paper, limited acoustic-phonetic information derived primarily by processing the excitation source information in the speech signal is used to improve the performance of detection of manner of articulation from a baseline phone recognition system. A context-independent HMM-based monophone system without any language information is used as the baseline system for this purpose. The performance of the phone recognizer in terms of its ability to detect the manners of articulation is studied. The errors in the hypothesis of the manner of articulation of phones are corrected using AP information such as voicing, voice bar and frication. It is shown that significant improvement can be achieved by using simple or limited AP information.

The motivation for this study is the need for careful analysis of aperiodicity of the excitation component in expressive voices. The paper proposes analysis methods which can preserve the excitation information corresponding to sequence of impulse-like excitation with variable strengths. To analyze the details of the excitation source characteristics, the epochs and the strength of the excitation at the epochs are obtained using the output of an ideal zero-frequency digital resonator. The vocal tract system characteristics are derived from the signal between two successive epochs using the numerator of the group delay function. The spectrogram of the zero-frequency filtered signal and the group delay spectrum correspond to characteristics of the excitation and the vocal tract system, respectively. Decomposition of the speech signal into these two components bring out the features of excitation and vocal tract system, which can be used to explain the perception of expressive voices in terms of features of aperiodicity, pitch, harmonics and sub-harmonics. The decomposition method is illustrated using examples from linguistically significant glottalized sounds (glottal stops and ejectives), singing voices and Noh voice.

For voiced speech, the most significant excitation takes place around the instant of glottal closure. Glottal closure instants (GCI) information is useful for accurate speech analysis. In particular accurate spectrum analysis is performed by considering the speech in the intervals of glottal closure. In this paper we propose an approach for detection of GCI by exploring Bessel feature, and the use of AM-FM signal. Using appropriate range of Bessel coefficients, the narrow band, band limited signal is obtained for the given signal.The bandlimited signal is considered as AM-FM signal. The signal is band limited for 0300 Hz to remove effect of formants. Amplitude envelope (AE) function of the AM-FM signal model has been estimated by the discrete energy separation algorithm (DESA). The performance of the method is demonstrated using CMU-Arctic database. The corresponding electro-glottograph (EGG) signals are used as a reference for the validation of the detected GCI locations.

This paper examines the Lombard effect on the excitation fea-tures in speech production. These features correspond mostly to the acoustic features at subsegmental (< pitch period) level. The instantaneous fundamental frequency F0 (i.e., pitch), the strength of excitation at the instants of significant excitation and a loudness measure reflecting the sharpness of the impulse-like excitation around epochs are used to represent the excitation features at the subsegmental level. The Lombard effect influ-ences the pitch and the loudness. The extent of Lombard effect on speech depends on the nature and level (or intensity) of the external feedback that causes the Lombard effect. Index Terms: Lombard effect, excitation source, loudness 1.

We study the performance of support vector machine based classifiers in acoustic model combination for recognition of context dependent sub word units of speech in multiple languages. In acoustic model combination, the data for similar sub word units across languages are shared to train acoustic models for multilingual speech. Sharing of data across languages leads to an increase in the number of training examples for a subword unit common to the languages. It may also lead to increase in the variability of the data for a subword unit. In This work, we study the effect of data sharing on the classification accuracy and complexity of acoustic models built using support vector machines. We compare the performance of multilingual acoustic models with that of monolingual acoustic models in the recognition of a large number of consonant-vowel units in the broadcast news corpus of three Indian languages.

Detection of vowel onset points (VOPs) is important for spotting subword units in continuous speech. For consonant-vowel (CV) utterances, VOP is the instant at which the consonant part ends and the vowel part begins. Accurate detection of VOPs is important for recognition of CV units in continuous speech. In this paper, we propose an approach for detection of VOPs using autoassociative neural network (AANN) models. A pair of AANN models are trained for each CV class to capture the characteristics of speech signal in the consonant and vowel regions of that class. The trained AANN models are then used to detect VOPs in continuous speech. The results of studies show that the proposed approach leads to significantly less number of spurious hypotheses.

We address the issues in recognition of a large number of subword units of speech with high confusability among several units. Evidence available from the classification models trained with a limited number of training examples may not be strong to correctly recognize the subword units. We present a constraint satisfaction neural network model that can be used to enhance the evidence for a particular unit with the supporting evidence available for a subset of units confusable with that unit. We demonstrate the enhancement of evidence by the proposed model in recognition of utterances of 145 consonant-vowel units.

In this paper, we address the issues in recognition of a large number of subword units of speech with high confusability among several units. Evidence available from the classification models trained with a limited number of training examples may not be strong to correctly recognize the subword units. We present a constraint satisfaction neural network model that can be used to enhance the evidence for a particular unit with the supporting evidence available for a subset of units confusable with the unit. We demonstrate the enhancement of evidence by the proposed model in recognition of utterances of 145 consonant-vowel units.

In this paper, we present a method to combine evidence from multiple classifiers to recognize a large number of subword units of speech using small size training data sets. Grouping criteria based on phonetic description are considered, to build multiple modular networks for recognition of the large number of units. Nonlinear compression of feature vectors is carried out to obtain reduced dimensional patterns, and multiple classifiers are trained separately using the uncompressed feature vectors and compressed feature vectors. Evidence from multiple classifiers at different stages in the recognition system is combined using the sum rule. Effectiveness of the proposed method is demonstrated for recognition of isolated utterances of 145 consonant-vowel units of speech.

In this paper, we consider approaches for linear and nonlinear compression of feature vectors for recognition of utterances of syllable-like units in Indian languages. The distribution capturing ability of an autoassociative neural network model is exploited to derive the components for compressing the feature vectors. The nonlinear compression is accomplished by a five layer autoassociative neural network model. Linear compression is realized by principal component analysis. Both linear and nonlinear compressions are performed on each subgroup of the sound units separately. The results show that it is indeed possible to compress the feature vectors from 50 to 19 dimension without affecting the performance of the classifier.