We propose a neural network model to estimate word translation probabilities for Cross-Lingual Information Retrieval (CLIR). The model estimates better probabilities for word translations than automatic word alignments alone, and generalizes to unseen source-target word pairs. We further improve the lexical neural translation model (and subsequently CLIR), by incorporating source word context, and by encoding the character sequences of input source words to generate translations of out-of-vocabulary words. To be effective, neural network models typically need training on large amounts of data labeled directly on the final task, in this case relevance to queries. In contrast, our approach only requires parallel data to train the translation model, and uses an unsupervised model to compute CLIR relevance scores.

Individual ship detection from underwater audio is the task of deciding whether a specific ship is present, using sound captured by an underwater hydrophone. It is a task analogous to speaker identification (SID), in the sense that it is an open-class detection task; the ships present could be other irrelevant (“impostor”) ships, never encountered in the training data. We present two methodologies for tackling this problem, both motivated by our work in speech-related technologies: (i) one based on neural networks, which follows, to a large extent, the approach of [1], and (ii) one based on i-vectors and PLDA [2]. To the best of our knowledge, this is the first time that the topic of individual ship detection is approached as an open-class detection problem.

In this paper we describe the 2016 BBN conversational telephone speech keyword spotting system; the culmination of four years of research and development under the IARPA Babel program. The system was constructed in response to the NIST Open Keyword Search (OpenKWS) evaluation of 2016. We present our technological breakthroughs in building top-performing keyword spotting processing systems for new languages, in the face of limited transcribed speech, noisy conditions, and limited system build time of one week.

With the completion of the IARPA Babel program, it is possible to systematically analyze the performance of speech recognition systems across a wide variety of languages. We select 16 languages from the dataset and compare performance using a deep neural network-based acoustic model. The focus is on keyword spotting using the actual term-weighted value (ATWV) metric. We demonstrate that ATWV is keyword dependent, and that this must be accounted for in any cross-language analysis. Further, we show that while performance across languages does not track with any particular feature of the language, it is correlated with inter-annotator agreement.

Spoken term detection, especially of out-of-vocabulary (OOV) keywords, benefits from the use of sub-word systems. We experiment with different language-independent approaches to sub-word unit generation, generating both syllable-like and morpheme-like units, and demonstrate how the performance of syllable-like units can be improved by artificially increasing the number of unique units. The effect of unit choice is empirically evaluated using the eight languages from the 2016 IARPA BABEL evaluation.

We describe our submission to the NIST OpenSAD evaluation of speech activity detection of noisy audio generated by the DARPA RATS program. With frequent transmission degradation, channel interference and other noises added, simple energy thresholds do a poor job at SAD for this audio. The evaluation measured performance on both in-training and novel channels. Our approach used a system combination of feed-forward neural networks and bidirectional LSTM recurrent neural networks. System combination and unsupervised adaptation provided further gains on novel channels that lack training data. These improvements lead to a 26% relative improvement for novel channels over simple decoding. Our system resulted in the lowest error rate on the in-training channels and second on the out-of-training channels.

The most common pipelines in keyword spotting involve some kind of speech recognition, which leads to the generation of sets of plausible hypotheses (e.g., word lattices), which are subsequently explored. The case of out-of-vocabulary (OOV) keywords is of special interest, because it requires representing keywords and/or lattices in an alternative format, so that the two can match. A number of techniques for dealing with OOV keywords have appeared in the literature; here, we focus on (i) fuzzy-phonetic search using phonetic confusion networks [1], and (ii) proxy-keyword search [2]. As we demonstrate in this paper, the combination of these two diverse techniques improves the ATWV of OOV keywords by at least 3% on average over the five development languages used in the second year of the IARPA Babel program.

Pronunciation lexicons can range from fully graphemic (modeling \neach word using the orthography directly) to fully phonemic \n(first mapping each word to a phoneme string). Between these \ntwo options lies a continuum of modeling options. We analyze \ntechniques that can improve the accuracy of a graphemic system \nwithout requiring significant effort to design or implement. \nThe analysis is performed in the context of the IARPA Babel \nproject, which aims to develop spoken term detection systems \nfor previously unseen languages rapidly, and with minimal human \neffort. We consider techniques related to letter-to-sound \nmapping and language-independent syllabification of primarily \ngraphemic systems, and discuss results obtained for six languages: \nCebuano, Kazakh, Kurmanji Kurdish, Lithuanian, Telugu \nand Tok Pisin.

We explore the impact of morphological segmentation on keyword spotting (KWS).Despite potential benefits, stateof-the-art KWS systems do not use morphological information.In this paper, we augment a state-of-the-art KWS system with sub-word units derived from supervised and unsupervised morphological segmentations, and compare with phonetic and syllabic segmentations.Our experiments demonstrate that morphemes improve overall performance of KWS systems.Syllabic units, however, rival the performance of morphological units when used in KWS.By combining morphological, phonetic and syllabic segmentations, we demonstrate substantial performance gains.

As shown in [1, 2], score normalization is of crucial importance for improving the Average Term-Weighted Value (ATWV) measure that is commonly used for evaluating keyword spotting systems. In this paper, we compare three different methods for score normalization within a keyword spotting system that employs phonetic search. We show that a new unsupervised linear fit method results in better-estimated posterior scores, that, when fed into the keyword-specific normalization of [1], result in ATWV gains of 3% on average. Furthermore, when these scores are used as features within a supervised machine learning framework, they result in additional gains of 3.8% on average over the five languages used in the first year of the IARPA-funded project Babel.

In this paper we describe the Vietnamese conversational telephone speech keyword spotting system under the IARPA Babel program for the 2013 evaluation conducted by NIST. The system contains several, recently developed, novel methods that significantly improve speech-to-text and keyword spotting performance such as stacked bottleneck neural network features, white listing, score normalization, and improvements on semi-supervised training methods. These methods resulted in the highest performance for the official IARPA Babel surprise language evaluation of 2013.

This paper presents a set of techniques that we used to improve our keyword search system for the third phase of the DARPA RATS (Robust Automatic Transcription of Speech) program, which seeks to advance state of the art detection capabilities on audio from highly degraded radio communication channels. The results for both Levantine and Farsi, which are the two target languages for the keyword search (KWS) task, are reported. About 13% absolute reduction in word error rate (from 70.2% to 57.6%) is achieved by using acoustic features derived from stacked Multi-Layer Perceptrons (MLP) and Deep Neural Network (DNN) acoustic models. In addition to score normalization and score/system combination for keyword search, we showed that the false alarm rate at the target false reject rate (15%) was reduced by about 1% (from 5.39% to 4.45%) by reducing the deletion errors of the speech-to-text system. Index Terms: speech recognition, KWS, MLP, DNN

In this paper, we investigate semi-supervised training for low resource languages where the initial systems may have high error rate (≥ 70.0% word eror rate). To handle the lack of data, we study semi-supervised techniques including data selection, data weighting, discriminative training and multilayer perceptron learning to improve system performance. The entire suite of semi-supervised methods presented in this paper was evaluated under the IARPA Babel program for the keyword spotting tasks. Our semi-supervised system had the best performance in the OpenKWS13 surprise language evaluation for the limited condition. In this paper, we describe our work on the Turkish and Vietnamese systems.

We present two techniques that are shown to yield improved Keyword Spotting (KWS) performance when using the ATWV/MTWV performance measures: (i) score normalization, where the scores of different keywords become commensurate with each other and they more closely correspond to the probability of being correct than raw posteriors; and (ii) system combination, where the detections of multiple systems are merged together, and their scores are interpolated with weights which are optimized using MTWV as the maximization criterion. Both score normalization and system combination approaches show that significant gains in ATWV/MTWV can be obtained, sometimes on the order of 8-10 points (absolute), in five different languages. A variant of these methods resulted in the highest performance for the official surprise language evaluation for the IARPA-funded Babel project in April 2013.

We present our work on semi-supervised learning of discriminative language models where the negative examples for sentences in a text corpus are generated using confusion models for Turkish at various granularities, specifically, word, sub-word, syllable and phone levels. We experiment with different language models and various sampling strategies to select competing hypotheses for training with a variant of the perceptron algorithm. We find that morph-based confusion models with a sample selection strategy aiming to match the error distribution of the baseline ASR system gives the best performance. We also observe that substituting half of the supervised training examples with those obtained in a semi-supervised manner gives similar results.

This paper investigates semi-supervised methods for discriminative language modeling, whereby n-best lists are “hallucinated” for given reference text and are then used for training n-gram language models using the perceptron algorithm. We perform controlled experiments on a very strong baseline English CTS system, comparing three methods for simulating ASR output, and compare the results with training with “real” n-best list output from the baseline recognizer. We find that methods based on extracting phrasal cohorts - similar to methods from machine translation for extracting phrase tables - yielded the largest gains of our three methods, achieving over half of the WER reduction of the fully supervised methods.

Discriminative language modeling is a structured classification problem. Log-linear models have been previously used to address this problem. In this paper, the standard dot-product feature representation used in log-linear models is replaced by a non-linear function parameterized by a neural network. Embeddings are learned for each word and features are extracted automatically through the use of convolutional layers. Experimental results show that as a stand-alone model the continuous space model yields significantly lower word error rate (1% absolute), while having a much more compact parameterization (60%-90% smaller). If the baseline scores are combined, our approach performs equally well.

In many inference and learning tasks, collecting large amounts of labeled training data is time consuming and expensive, and oftentimes impractical. Thus, being able to efficiently use small amounts of labeled data with an abundance of unlabeled data-the topic of semi-supervised learning (SSL) [1]-has garnered much attention. In this paper, we look at the problem of choosing these small amounts of labeled data, the first step in a bootstrapping paradigm. Contrary to traditional active learning where an initial trained model is employed to select the unlabeled data points which would be most informative if labeled, our selection has to be done in an unsupervised way, as we do not even have labeled data to train an initial model. We propose using unsupervised clustering algorithms, in particular integrated sensing and processing decision trees (ISPDTs) [2], to select small amounts of data to label and subsequently use in SSL (e.g. transductive SVMs). In a language identification task on the CallFriend1 and 2003 NIST Language Recognition Evaluation corpora [3], we demonstrate that the proposed method results in significantly improved performance over random selection of equivalently sized training data.

Inverse Document Frequency (IDF) is an important quantity in many applications, including Information Retrieval. IDF is defined in terms of document frequency, df (w), the number of documents that mention w at least once. This quantity is relatively easy to compute over textual documents, but spoken documents are more challenging. This paper considers two baselines: (1) an estimate based on the 1-best ASR output and (2) an estimate based on expected term frequencies computed from the lattice. We improve over these baselines by taking advantage of repetition. Whatever the document is about is likely to be repeated, unlike ASR errors, which tend to be more random (Poisson). In addition, we find it helpful to consider an ensemble of language models. There is an opportunity for the ensemble to reduce noise, assuming that the errors across language models are relatively uncorrelated. The opportunity for improvement is larger when WER is high. This paper considers a pairing task application that could benefit from improved estimates of df. The pairing task inputs conversational sides from the English Fisher corpus and outputs estimates of which sides were from the same conversation. Better estimates of df lead to better performance on this task.

This paper summarizes the 2010 CLSP Summer Workshop on speech recognition at Johns Hopkins University. The key theme of the workshop was to improve on state-of-the-art speech recognition systems by using Segmental Conditional Random Fields (SCRFs) to integrate multiple types of information. This approach uses a state of-the-art baseline as a springboard from which to add a suite of novel features including ones derived from acoustic templates, deep neural net phoneme detections, duration models, modulation features, and whole word point-process models. The SCRF framework is able to appropriately weight these different information sources to produce significant gains on both die Broadcast News and Wall Street Journal tasks.

Given a number of machine translations of a source segment, the goal of system combination is to produce a new translation that has better quality than all of them. This paper describes a number of improvements that were recently added to the JHU system combination scheme: (i) A hypothesis ranking technique which orders the system outputs, on a per-segment basis, according to predicted translation quality, thus improving a subsequent incremental combination step. (ii) A two-pass combination procedure, which first produces several combination outputs with the given translations, and then performs one more combination step with these new outputs. Results from the NIST MT09 informal system combination evaluation on Arabic-to-English and Urdu-to-English1show that both approaches offer significant BLEU and TER gains over a baseline JHU combination scheme.

A novel self-supervised discriminative training method for estimating language models for automatic speech recognition (ASR) is proposed. Unlike traditional discriminative training methods that require transcribed speech, only untranscribed speech and a large text corpus is required. An exponential form is assumed for the language model, as done in maximum entropy estimation, but the model is trained from the text using a discriminative criterion that targets word confusions actually witnessed in first-pass ASR output lattices. Specifically, model parameters are estimated to maximize the likelihood ratio between words w in the text corpus and w's cohorts in the test speech, i.e. other words that w competes with in the test lattices. Empirical results are presented to demonstrate statistically significant improvements over a 4-gram language model on a large vocabulary ASR task.

Heteroscedastic Linear Discriminant Analysis (HLDA) was introduced in [1] as an extension of Linear Discriminant Analysis to the case where the class-conditional distributions have unequal covariances. The HLDA transform is computed such that the likelihood of the training (labeled) data is maximized, under the constraint that the projected distributions are orthogonal to a nuisance space that does not offer any discrimination. In this paper we consider the case of semi-supervised learning, where a large amount of unlabeled data is also available. We derive update equations for the parameters of the projected distributions, which are estimated jointly with the HLDA transform, and we empirically compare it with the case where no unlabeled data are available. Experimental results with synthetic data and real data from a vowel recognition task show that, in most cases, semi-supervised HLDA results in improved performance over HLDA. Index Terms: heteroscedastic linear discriminant analysis, nuisance space, semi-supervised learning. 1.

Csiszar introduced the mutual information of order alpha in [1] as a parameterized version of Shannon's mutual information. It involves a minimization over the probability space, and it cannot be computed in closed form. An alternating minimization algorithm for its computation is presented in this paper, along with a proof of its convergence. Furthermore, it is proved that the algorithm is an instance of the Csiszar-Tusnady iterative minimization procedure [2].

System combination is a technique which has been shown to yield significant gains in speech recognition and machine translation. Most combination schemes perform an alignment between different system outputs in order to produce lattices (or confusion networks), from which a composite hypothesis is chosen, possibly with the help of a large language model. The benefit of this approach is two-fold: (i) whenever many systems agree with each other on a set of words, the combination output contains these words with high confidence; and (ii) whenever the systems disagree, the language model resolves the ambiguity based on the (probably correct) agreed upon context. The case of machine translation system combination is more challenging because of the different word orders of the translations: the alignment has to incorporate computationally expensive movements of word blocks. In this paper, we show how one can combine translation outputs efficiently, extending the incremental alignment procedure of (A-V.I. Rosti et al., 2008). A comparison between different system combination design choices is performed on an Arabic speech translation task.

Iterative denoising trees were used by Karakos et al. (2005) for unsupervised hierarchical clustering. The tree construction involves projecting the data onto low-dimensional spaces, as a means of smoothing their empirical distributions, as well as splitting each node based on an information-theoretic maximization objective. In this paper, we improve upon the work of (Karakos et al., 2005) in two ways: (i) the amount of computation spent searching for a good projection at each node now adapts to the intrinsic dimensionality of the data observed at that node; (ii) the objective at each node is to find a split which maximizes a generalized form of mutual information, the Jensen-Renyi divergence; this is followed by an iterative Naive Bayes classification. The single parameter α of the Jensen-Renyi divergence is chosen based on the "strapping" methodology, which learns a meta-classifier on a related task. Compared with the sequential information bottleneck method, our procedure produces state-of-the-art results on an unsupervised categorization task of documents from the "20 Newsgroups" dataset.

The maximum likelihood set (MLS) is a novel candidate for nonparametric probability estimation from small samples that permits incorporating prior or structural knowledge into the estimator. It is a set of probability distributions which assign to the observed type (or empirical distribution) a likelihood that is no lower than the likelihood they assign to any other type. The MLS has been shown to have many highly desirable properties, including strong consistency of MLS-based estimates; yet the probability that the MLS contains the data-generating distribution may be arbitrarily small. In this paper, we propose to overcome this shortcoming via an epsiv-fattening of the MLS. The proposed set, called the High Likelihood Set (HLS), with epsiv rarr 0 slowly in sample size, ensures that the HLS contains the data- generating distribution with arbitrarily large probability, while retaining most desirable properties of the MLS. In particular, the HLS provides a "high-probability" bound on the estimation error, and experimental results in statistical language modeling show improved operational performance from HLS-based estimates.

The maximum likelihood set (MLS) was recently introduced in B. Jedynak and S. Khudanpur (2005) as an effective, parameter-free technique for estimating a probability mass function (pmf) from sparse data. The MLS contains all pmfs that assign merely a higher likelihood to the observed counts than to any other set of counts, for the same sample size. In this paper, the MLS is extended to the case of conditional pmf estimation. First, it is shown that, when the criterion for selecting a pmf from the MLS is the KL-divergence, the selected conditional pmf naturally has a back-off form, except for a ceiling on the probability of high frequency symbols that are not seen in particular contexts. Second, the pmf has a sparse parameterization, leading to efficient algorithms for KL-divergence minimization. Experimental results from bigram and trigram language modeling indicate that pmfs selected from the MLS are competitive with state-of-the-art estimates

Integrated sensing and processing decision trees (ISPDT) (Priebe et al. (2004)) were introduced as a tool for supervised classification of high-dimensional data. In this paper, we consider the problem of unsupervised classification, through a recursive construction of ISPDT, where at each internal node the data (i) are split into clusters, and (ii) are transformed independently of other clusters, guided by some optimization objective. We show that the maximization of information-theoretic quantities such as mutual information and /spl alpha/-divergences is theoretically justified for growing ISPDT, assuming that each data point is generated by a finite-memory random process given the class label. Furthermore, we present heuristics that perform the maximization in a greedy manner, and we demonstrate their effectiveness with empirical results from multispectral imaging.

A system which embeds watermarks in n-dimensional Gaussian data and distributes them in compressed form is studied. The watermarked/compressed data have to satisfy a distortion constraint, and the watermark has to be recoverable in a private scenario (in which the original data are available at the watermark detector). The performance of the system in the presence of additive Gaussian attacks is considered, and the region of achievable quantization and watermarking rate pairs (R/sub Q/,R/sub W/) is established. Moreover, two surprising facts are demonstrated: (1) at low R/sub Q/, the maximum achievable R/sub W/ is the same as when there are no attacks; and (2) at high (but finite) R/sub Q/, the maximum achievable R/sub W/ is the same as when there is no compression (R/sub Q/=/spl infin/). Finally, the performance of related schemes is also discussed.

Recent works in multispectral image processing advocate the employment of vector approaches for this class of signals. Vector processing operators that involve the minimization of a suitable error criterion have been proposed and shown appropriate for this task. In this framework, two main classes of vector processing filters have been reported in the literature. Astola et al. (1990) introduce the well-known class of vector median filters (VMF), which are derived as maximum likelihood (ML) estimates from exponential distributions. Trahanias et al. (see ibid., vol.2, no.4, p.528-34, 1993 and vol.5, no.6, p.868-80, 1996) study the processing of color image data using directional information, considering the class of vector directional filters (VDF). We introduce a new filter structure, the directional-distance filters (DDF), which combine both VDF and VMF in a novel way. We show that DDF are robust signal estimators under various noise distributions, they have the property of chromaticity preservation and, finally, compare favorably to other multichannel image processing filters.

The processing of color image data using directional information is studied. The class of vector directional filters (VDF), which was introduced by the authors in a previous work, is further considered. The analogy of VDF to the spherical median is shown, and their relation to the spatial median is examined. Moreover, their statistical and deterministic properties are studied, which demonstrate their appropriateness in image processing. VDF result in optimal estimates of the image vectors in the directional sense; this is very important in the case of color images, where the vectors' direction signifies the chromaticity of a given color. Issues regarding the practical implementation of VDF are also considered. In addition, efficient filtering schemes based on VDF are proposed, which include adaptive and/or double-window structures. Experimental and comparative results in image filtering show very good performance measures when the error is measured in the L*a*b* space. L*a*b* is known as a space where equal color differences result in equal distances, and therefore, it is very close to the human perception of colors. Moreover, an indication of the chromaticity error is obtained by measuring the error on the Maxwell triangle; the results demonstrate that VDF are very accurate chromaticity estimators.

It is generally accepted that multispectral image processing should be employed using a vector approach. Two are the main classes of vector processing filters that have been reported in the literature. Astola et al. (1990) introduce the well known class of vector median filters (VMF), which are derived as MLE estimates from exponential distributions. Trahanias et al. (see IEEE Trans. Image Process., vol.2, p.528-34, Oct. 1993) study the processing of color image data using directional information, considering the class of vector directional filters (VDF). We introduce a new class of filters, the directional-distance filters (DDF), which combine both VDF and VMF in a novel way. We show that DDF can eliminate the noise much more effectively than the VMF (even the impulsive noise), and that they have the property of chromaticity preservation.