We present a novel set of keyword detection techniques to accelerate spoken term detection for known queries with minimal loss in accuracy. Using only ASR frame-level acoustic posteriors we can train multiple models to effectively detect non-target segments for which we need not perform full lattice decoding. We estimate phone n-gram soft counts for each segment in a single pass over the frame-level output. From this we can efficiently detect a fixed set of keywords with both linear and DNN-based classifiers. Furthermore we can train the linear classifiers on a small number of labeled examples. Experiments on the PSC and VAST English subset of NIST's 2019 OpenSAT evaluation demonstrate we can filter out half of the test audio segments while only increasing the keyword miss rate by under 3%.

We compare the multinomial i-vector framework from the speech community with LDA, SAGE, and LSA as feature learners for topic ID on multinomial speech and text data.We also compare the learned representations in their ability to discover topics, quantified by distributional similarity to gold-standard topics and by human interpretability.We find that topic ID and topic discovery are competing objectives.We argue that LSA and i-vectors should be more widely considered by the text processing community as pre-processing steps for downstream tasks, and also speculate about speech processing tasks that could benefit from more interpretable representations like SAGE.

Content-based recommender systems use preference ratings and features that characterize media to model users' interests or information needs for making future recommendations. While previously developed in the music and text domains, we present an initial exploration of content-based recommendation for spoken documents using a corpus of public domain internet audio. Unlike familiar speech technologies of topic identification and spoken document retrieval, our recommendation task requires a more comprehensive notion of document relevance than bags-of-words would supply. Inspired by music recommender systems, we automatically extract a wide variety of content-based features to characterize non-linguistic aspects of the audio such as speaker, language, gender, and environment. To combine these heterogeneous information sources into a single relevance judgement, we evaluate feature, score, and hybrid fusion techniques. Our study provides an essential first exploration of the task and clearly demonstrates the value of a multisource approach over a bag-of-words baseline.

We aim to improve speech retrieval performance by augmenting traditional N-gram language models with different types of topic context.We present a latent topic model framework that treats documents as arising from an underlying topic sequence combined with a cache-based repetition model.We analyze our proposed model both for its ability to capture word repetition via the cache and for its suitability as a language model for speech recognition and retrieval.We show this model, augmented with the cache, captures intuitive repetition behavior across languages and exhibits lower perplexity than regular LDA on held out data in multiple languages.Lastly, we show that our joint model improves speech retrieval performance beyond N-grams or latent topics alone, when applied to a term detection task in all languages considered.

We aim to improve spoken term detection performance by incorporating contextual information beyond traditional Ngram language models.Instead of taking a broad view of topic context in spoken documents, variability of word co-occurrence statistics across corpora leads us to focus instead the on phenomenon of word repetition within single documents.We show that given the detection of one instance of a term we are more likely to find additional instances of that term in the same document.We leverage this burstiness of keywords by taking the most confident keyword hypothesis in each document and interpolating with lower scoring hits.We then develop a principled approach to select interpolation weights using only the ASR training data.Using this re-weighting approach we demonstrate consistent improvement in the term detection performance across all five languages in the BABEL program.

We consider the task of identifying topics in recorded speech across many languages. We identify a statistically discriminative set of topic keywords, and examine the relationship between overall word error rate (WER), keyword-specific detection performance, and topic identification (Topic ID) performance on the Fisher Spanish corpus. Building increasingly constrained systems - from copious to limited training LVCSR to limited-vocabulary keyword spotting - we show that neither high WER (>60%) nor low-precision term detection (<;40%) are necessarily impediments to Topic ID. By using deep neural net acoustic models for keyword spotting, we can double recall and ranked retrieval performance over comparable PLP-based models and achieve Topic ID performance on par with well-trained LVCSR or human transcripts.

We aim to improve term detection performance by augmenting traditional N-gram language models with multiple levels of topic context. We demonstrate that incorporating complementary aspects of topicality leads to significant improvements in term detection accuracy. We represent broad topic context through document-specific latent topics inferred via a Bayesian topic model. We capture local topic context with a cache-based adaptive language model. Measured on four languages from from the IARPA Babel program, interpolating unigrams from the broad topic context improves term detection performance by up to 1% absolute via lattice re-scoring. Re-decoding with the same document-specific model improves accuracy by up to 2.1%. Adding local context via cached N-grams improves performance by up to 1.6%. A combined approach, re-decoding with latent topic information then re-scoring with the local cached N-grams gives an overall improvement of up to 2.4%. For all languages, combining broad and local topic information outperforms any individual method.

In many topic identification applications, supervised training labels are indirectly related to the semantic content of the documents being classified. For example, many topically distinct emails will all be assigned a single broad category label of "spam" or "not-spam", and a two-class classifier will lack direct knowledge of the underlying topic structure. This paper examines the degradation of topic identification performance on conversational speech when multiple semantic topics are combined into a single broad category. We then develop techniques using document clustering and Latent Dirchlet Allocation (LDA) to exploit the underlying semantic topics which improve performance over classifiers trained on the single category label by up to 20%.

We present a system to index and search conversational speech using a scoring heuristic on the expected posterior counts of phone n-grams in recognition lattices. We report significant improvements in retrieval effectiveness on five human languages over a strong 1-best baseline. The method is shown to improve the utility (mean average precision) of the retrieved lattices' rank order and to do so with a search cost negligible compared to the fastest yet known methods for the linear scanning of phonetic lattices.