Transformers are popular sequence-to-sequence models but have large number of parameters and high compute requirements. As an initiative to reduce the energy demand by Transformer models and to get better Transformer models for edge devices, we propose a light weight Transformer in this paper. We attempt to reduce the compute and carbon footprint of the original Transformer architecture by incorporating architectural modifications. The proposed modifications reduce the Transformer parameters by 42.7 % relative to the original Transformer of same depth and width. Our automatic speech recognition experiments on LibriSpeech, SPGISpeech and GigaSpeech datasets show that the proposed light weight Transformer has negligible ASR performance degradation. The compute requirements also reduce by 23% relative to the original Transformer of same depth and width. We also show that the proposed Lite ASR Transformer has acceptable convergence and also the latency is 20% lesser relative to the original Transformer of same depth and width.

This paper presents the data collection efforts, statistics and preparation involved in creating the 2333-hour INX-SpeakerHub, an Indian multilingual speaker identification dataset. It has legally collected speech from approximately 11,000 Indian native speakers in 10 different Indian languages. Until now, the VoxCeleb dataset (1+2) has been the most popular corpus used in many current state-of-theart systems. Therefore, VoxCeleb based speaker embedding extractors are often used by default even for Indian language-based speech applications. However, the proportion of Indian language data in VoxCeleb is quite less and might lead to subpar performance in speech tasks involving Indian languages. India is a country with 22 official languages and is home to 1.43 billion people. So creating a dataset like INX-SpeakerHub to build speaker embedding extractors for the Indian languages is of great interest. As our analysis shows, the VoxCeleb dataset has higher % equal error rate (% EER) in comparison to the INX-SpeakerHub for the speaker verification task in Indian languages. Further, we also analyse the speaker verification performance across different language families as well as on unseen languages.

One of the most popular speaker embeddings is x-vectors, which are obtained from an architecture that gradually builds a larger temporal context with layers. In this paper, we propose to derive speaker embeddings from Transformer’s encoder trained for speaker classification. Self-attention, on which Transformer’s encoder is built, attends to all the features over the entire utterance and might be more suitable in capturing the speaker characteristics in an utterance. We refer to the speaker embeddings obtained from the proposed speaker classification model as s-vectors to emphasize that they are obtained from an architecture that heavily relies on self-attention. Through experiments, we demonstrate that s-vectors perform better than x-vectors. In addition to the s-vectors, we also propose a new architecture based on Transformer’s encoder for speaker verification as a replacement for speaker verification based on conventional probabilistic linear discriminant analysis (PLDA). This architecture is inspired by the next sentence prediction task of bidirectional encoder representations from Transformers (BERT), and we feed the s-vectors of two utterances to verify whether they belong to the same speaker. We name this architecture the Transformer encoder speaker authenticator (TESA). Our experiments show that the performance of s-vectors with TESA is better than s-vectors with conventional PLDA-based speaker verification.

Code-switching (CS) refers to (inter/intra-word) switching between multiple languages in a single conversation. In multilingual countries like India, CS occurs very often in everyday speech, resulting in a new breed of languages in urban regions like Hinglish (Hindi-English), Tanglish (Tamil-English), etc. Research in Indic CS speech recognition is primarily affected by insufficient data. In this paper, we investigate methods to deal with such very low resource scenarios. Recently, Transformers have shown promising results on automatic speech recognition (ASR) tasks. In a Transformer based framework, we investigate two methods for Tamil-English CS speech recognition, namely, (i) well-trained encoders of Monolingual Transformers as feature extractors to provide language discrimination, (ii) language information as tokens at the targets. Our results show that CS is efficiently handled by the second method, while the first method was efficient in discriminating languages.

The recent success of the Transformer based sequence-to-sequence framework for various Natural Language Processing tasks has motivated its application to Automatic Speech Recognition. In this work, we explore the application of Transformers on low resource Indian languages in a multilingual framework. We explore various methods to incorporate language information into a multilingual Transformer, i.e., (i) at the decoder, (ii) at the encoder. These methods include using language identity tokens or providing language information to the acoustic vectors. Language information to the acoustic vectors can be given in the form of one hot vector or by learning a language embedding. From our experiments, we observed that providing language identity always improved performance. The language embedding learned from our proposed approach, when added to the acoustic feature vector, gave the best result. The proposed approach with retraining gave 6% - 11% relative improvements in character error rates over the monolingual baseline.