Non-Matching Reference-based Speech Quality Assessment models typically require numerous references during inference to ensure stable and accurate predictions. However, this dependency introduces significant computational overhead, limiting their suitability for real-time applications. In this paper, we propose a novel training paradigm that directly addresses prediction instability at its source by integrating multiple references during training rather than during inference, as in existing approaches. This method allows the model to capture the inherent variability of reference signals, thereby enhancing prediction reliability. Additionally, we introduce an auxiliary variance loss function to minimize inconsistencies across predictions, ensuring stable assessments regardless of the number of references used. Experiments on the NISQA datasets demonstrate that, with the same training time, our method achieves consistent predictions with a single reference during inference, resulting in a 100-fold reduction in computational time while maintaining high accuracy.

The process of transcribing speech plays a critical role in developing automatic speech recognition (ASR) systems. During this process, human transcribers listen to spoken utterances and convert them into accurately written texts. However, our initial investigations using telephone conversation datasets have shown that approximately 40% of the recorded utterances are challenging to transcribe and often disregarded by transcribers due to various factors, such as unintelligibility. As a result, transcribers waste significant time and effort listening to such utterances, significantly slowing down the overall transcribing speed. In this paper, we explore the potential of pretrained speech recognition models to model transcribers’ behavior in the speech transcribing process. By leveraging the knowledge and insights encoded in these models, we aim to filter out problematic utterances that transcribers typically disregard before they reach human transcribers, thus optimizing their time and effort. Through experiments on practical telephone datasets, our approach utilizing pretrained speech recognition models successfully reduces unwanted utterances by 50% while preserving the high quality of the collected dataset.

It is challenging to obtain large amounts of native (matched) labels for speech audio in underresourced languages. This challenge is often due to a lack of literate speakers of the language, or in extreme cases, a lack of universally acknowledged orthography as well. One solution is to increase the amount of labeled data by using mismatched transcription, which employs transcribers who do not speak the underresourced language of interest called the target language (in place of native speakers), to transcribe what they hear as nonsense speech in their own annotation language (≠ target language). Previous uses of mismatched transcription converted it to a probabilistic transcription (PT), but PT is limited by the errors of nonnative perception. This paper proposes, instead, a multitask learning framework in which one deep neural network (DNN) is trained to optimize two separate tasks: acoustic modeling of a small number of matched transcription with matched target-language graphemes; and acoustic modeling of a large number of mismatched transcription with mismatched annotation-language graphemes. We find that: first, the multitask learning framework gives significant improvement over monolingual, semisupervised learning, multilingual DNN training, and transfer learning baselines; second, a Gaussian Mixture Model-Hidden-Markov Model (GMM-HMM) model adapted using PT improves alignments, thereby improving training; and third, bottleneck features trained on the mismatched transcriptions lead to even better alignments, resulting in further performance gains of the multitask DNN. Our experiments are conducted on the IARPA Georgian and Vietnamese BABEL corpora as well as on our newly collected speech corpus of Singapore Hokkien, an underresourced language with no standard written form.

We present exemplar-inspired low-resource spoken keyword search strategies for acoustic modeling, keyword verification, and system combination. This state-of-the-art system was developed by the SINGA team in the context of the 2015 NIST Open Keyword Search Evaluation (OpenKWS15) using conversational Swahili provided by the IARPA Babel program. In this work, we elaborate on the following: (1) exploiting exemplar training samples to construct a non-parametric acoustic model using kernel density estimation at test time; (2) rescoring hypothesized keyword detections through quantifying their acoustic similarity with exemplar training samples; (3 ) extending our previously proposed system combination approach to incorporate prosody features of exemplar keyword samples.

Dynamic Time Warping (DTW) is widely used in language independent query-by-example (QbE) spoken term detection (STD) tasks due to its high performance. However, there are two limitations of DTW based template matching, 1) it is not straightforward to perform approximate match of audio queries; 2) DTW is sensitive to the mismatch of signal conditions between the query and the speech search data. To allow approximate search, we propose a partial template matching strategy using phone time boundary information generated by a phone recognizer. To have more invariant representation of audio signals, we use bottleneck features (BNF) as the input of DTW. The BNF network is trained from augmented data, which is generated by adding reverberation and additive noises to the clean training data. Experimental results on QUESST 2015 task shows the effectiveness of the proposed methods for QbE-STD when the queries and search data are both distorted by reverberation and noises.