In this article we present a three-step methodology for dynamically improving a statistical machine translation (SMT) system by incorporating human feedback in the form of free edits on the system translations. We target at feedback provided by casual users, which is typically error-prone. Thus, we first propose a filtering step to automatically identify the better user-edited translations and discard the useless ones. A second step produces a pivot-based alignment between source and user-edited sentences, focusing on the errors made by the system. Finally, a third step produces a new translation model and combines it linearly with the one from the original system. We perform a thorough evaluation on a real-world dataset collected from the Reverso.net translation service and show that every step in our methodology contributes significantly to improve a general purpose SMT system. Interestingly, the quality improvement is not only due to the increase of lexical coverage, but to a better lexical selection, reordering, and morphology. Finally, we show the robustness of the methodology by applying it to a different scenario, in which the new examples come from an automatically Web-crawled parallel corpus. Using exactly the same architecture and models provides again a significant improvement of the translation quality of a general purpose baseline SMT system.

We present a novel strategy to derive new translation units using an additional bilingual corpus and a previously trained SMT system. The units were used to adapt the SMT system. The derivation process can be applied when the additional corpus is very small compared with the original train corpus and it does not require to compute new word alignments using all corpora. The strategy is based in the Levenshtein Distance and its resulting path. We reported a statistically significant improvement, with a confidence level of 99%, when adapting an Ngram-based Catalan-Spanish system using an additional corpus that represents less than 0.5% of the original train corpus. The additional translation units were able to solve morphological and lexical errors and added previously unknown words to the vocabulary.

This paper describes an approach for computing a consensus translation from the outputs of multiple machine translation (MT) systems. The consensus translation is computed by weighted majority voting on a confusion network, similarly to the well-established ROVER approach of Fiscus for combining speech recognition hypotheses. To create the confusion network, pairwise word alignments of the original MT hypotheses are learned using an enhanced statistical alignment algorithm that explicitly models word reordering. The context of a whole corpus of automatic translations rather than a single sentence is taken into account in order to achieve high alignment quality. The confusion network is rescored with a special language model, and the consensus translation is extracted as the best path. The proposed system combination approach was evaluated in the framework of the TC-STAR speech translation project. Up to six state-of-the-art statistical phrase-based translation systems from different project partners were combined in the experiments. Significant improvements in translation quality from Spanish to English and from English to Spanish in comparison with the best of the individual MT systems were achieved under official evaluation conditions.

Ngram-based Statistical Machine Translation relies on a standard Ngram language model of tuples to estimate the translation pro-cess. In training, this translation model requires a segmentation of each parallel sentence, which involves taking a hard decision on tuple segmentation when a word is not linked during word align-ment. This is especially critical when this word appears in the target language, as this hard decision is compulsory. In this paper we present a thorough study of this situation, comparing for the rst time each of the proposed techniques in two independent tasks, namely EnglishSpanish European Parlia-ment Proceedings large-vocabulary task and ArabicEnglish Basic Travel Expressions small-data task. In the face of this comparison, we present a novel segmentation technique which incorporates lin-guistic information. Results obtained in both tasks outperform all previous techniques. Index Terms: statistical machine translation, tuple segmentation, n-gram-based SMT, linguistic information

This paper addresses the problem of reordering in statistical machine translation (SMT). We describe an elegant and efficient approach to couple reordering (word order monotonization) and decoding, which does not need for any additional model. We use linguistically motivated reordering rules to extend a monotonic search graph (with reordering hypotheses). The extended graph is traversed in decoding when a fully- informed decision can be taken (no preprocessing decision about reordering is taken). We also show how the N-gram translation model can be successfully used as reordering model when estimated with reordered source words (to harmonize the source and target word order). Experiments are reported on the Euparl task (Spanish- to-English and English-to-Spanish). Results are presented regarding translation accuracy and computational efficiency, showing significant improvements in translation quality for both translation directions at a very low computational cost.

This article describes in detail an n-gram approach to statistical machine translation. This approach consists of a log-linear combination of a translation model based on n-grams of bilingual units, which are referred to as tuples, along with four specific feature functions. Translation performance, which happens to be in the state of the art, is demonstrated with Spanish-to-English and English-to-Spanish translations of the European Parliament Plenary Sessions (EPPS).

In this paper, a novel phrase alignment strategy combining linguistic knowledge and cooccurrence measures extracted from bilingual corpora is presented. The algorithm is mainly divided into four steps, namely phrase selection and classification, phrase alignment, one-to-one word alignment and postprocessing. The first stage selects a linguistically-derived set of phrases that convey a unified meaning during translation and are therefore aligned together in parallel texts. These phrases include verb phrases, idiomatic expressions and date expressions. During the second stage, very high precision links between these selected phrases for both languages are produced. The third step performs a statistical word alignment using association measures and link probabilities with the remaining unaligned tokens, and finally the fourth stage takes final decisions on unaligned tokens based on linguistic knowledge. Experiments are reported for an English-Spanish parallel corpus, with a detailed description of the evaluation measure and manual reference used. Results show that phrase cooccurrence measures convey a complementary information to word cooccurrences and a stronger evidence of a correct alignment, successfully introducing linguistic knowledge in a statistical word alignment scheme. Precision, Recall and Alignment Error Rate (AER) results are presented, outperforming state-of-the-art alignment algorithms.

In this paper we describe MARIE, an Ngram-based statistical machine translation decoder. It is implemented using a beam search strategy, with distortion (or reordering) capabilities. The underlying translation model is based on an Ngram approach, extended to introduce reordering at the phrase level. The search graph structure is designed to perform very accurate comparisons, what allows for a high level of pruning, improving the decoder efficiency. We report several techniques for efficiently prune out the search space. The combinatory explosion of the search space derived from the search graph structure is reduced by limiting the number of reorderings a given translation is allowed to perform, and also the maximum distance a word (or a phrase) is allowed to be reordered. We finally report translation accuracy results on three different translation tasks.

This paper introduces a rule-based classification of single-word and compound verbs into a statistical machine translation approach. By substituting verb forms by the lemma of their head verb, the data sparseness problem caused by highly-inflected languages can be successfully addressed. On the other hand, the information of seen verb forms can be used to generate new translations for unseen verb forms. Translation results for an English to Spanish task are reported, producing a significant performance improvement. 1.

In Statistical Machine Translation, the use of reordering for certain language pairs can produce a significant improvement on translation accuracy. However, the search problem is shown to be NP-hard when arbitrary reorderings are allowed. This paper addresses the question of reordering for an Ngram-based SMT approach following two complementary strategies, namely reordered search and tuple unfolding. These strategies interact to improve translation quality in a Chinese to English task. On the one hand, we allow for an Ngram-based decoder (MARIE) to perform a reordered search over the source sentence, while combining a translation tuples Ngram model, a target language model, a word penalty and a word distance model. Interestingly, even though the translation units are learnt sequentially, its reordered search produces an improved translation. On the other hand, we allow for a modification of the translation units that unfolds the tuples, so that shorter units are learnt from a new parallel corpus, where the source sentences are reordered according to the target language. This tuple unfolding technique reduces data sparseness and, when combined with the reordered search, further boosts translation performance. Translation accuracy and efficency results are reported for the IWSLT 2004 Chinese to English task.

This paper describes a statistical machine translation system that uses a translation model which is based on bilingual n-grams. When this translation model is log-linearly combined with four specific feature functions, state of the art translations are achieved for Spanish-to-English and English-to-Spanish translation tasks. Some specific results obtained for the EPPS (European Parliament Plenary Sessions) data are presented and discussed. Finally, future research issues are depicted.

This paper shows the common framework that underlies the translation systems based on phrases or driven by finite state transducers, and summarizes a first comparison between them. In both approaches the translation process is based on pairs of source and target strings of words (segments) related by word alignment. Their main difference comes from the statistical modeling of the translation context. The experimental study has been carried out on an English/Spanish version of the VERB-MOBIL corpus. Under the constrain of a monotone composition of translated segments to generate the target sentence, the finite state based translation outperforms the phrase based counterpart. 1.

The goal of this paper is to explore and describe the potential of multilingual acoustic models for automatic speech recognition of the languages spoken in Morocco. The basic experimental framework comes from the OrienTel project, mainly the sound inventory of the Arabic languages and the speech databases. Monolingual and multilingual automatic speech recognition systems for Modern Colloquial and Standard Arabic (MCA and MSA, respectively) and French languages are developed and evaluated, in order to envisage the phonetic exchange and similarity among the three languages. As a main result, it can be stated that a combined modeling of MSA and MCA or, even a trilingual design, does not harm the performance of the recognition system.

In this paper, a statistical speech-to-speech translation system, developed at TALP during the last months, is presented. By adapting well-known speech recognition techniques to the specific translation setting, th e system is able to integrate speech sign al into a fin ite state tran sdu cer th at tran slates statistically domain-constrained Span ish sen tences into En glish on es.

We present Basurde[lite], a complete human-machine dialogue system for accessing railway information, based on machine-driven dialogues, a small vocabulary and grammar, and word and phrase spotting based on unrestricted grammars. We give preliminary evaluation results. Although machine-driven, the system accepts however some flexibility: in some cases the user may give a reply not corresponding to the system’s question, and may use out-ofvocabulary words and expressions. The system mimics the user in his choice of time and date formats and language used for place names. 1.

This paper introduces a first approach to emotion recognition using RAMSES, the UPC’s speech recognition system. The approach is based on standard speech recognition technology using hidden semi-continuous Markov models. Both the selection of low level features and the design of the recognition system are addressed. Results are given on speaker dependent emotion recognition using the Spanish corpus of INTERFACE Emotional Speech Synthesis Database. The accuracy recognising seven different emotions—the six ones defined in MPEG-4 plus neutral style—exceeds 80% using the best combination of low level features and HMM structure. This result is very similar to that obtained with the same database in subjective evaluation by human judges. Dealing with the speaker’s emotion is one of the latest challenges in speech technologies. Three different aspects can be easily identified: speech recognition in the presence of emotional speech, synthesis of emotional speech, and emotion recognition. In this last case, the objective is to determine the emotional state of the speaker out of the speech samples. Possible applications include from help to psychiatric diagnosis to intelligent toys, and is a subject of recent but rapidly growing interest [1]. This paper describes the TALP researchers first approach to emotion recognition. The work is inserted in the scope of the INTERFACE project [2]. The objective of this European Commission sponsored project is “to define new models and implement advanced tools for audio-video analysis, synthesis and representation in order to provide essential technologies for the implementation of large-scale virtual and augmented environments. The work is oriented to make man-machine interaction as natural as possible, based on everyday human communication by speech, facial expressions and body gestures.” In the field of emotion recognition out of speech, the main goal of the INTERFACE project will be the construction of a real-time multi-lingual speaker independent emotion recogniser. For this purpose, large speech databases with recordings from many speakers and languages are needed. As these resources are not available yet, a reduced problem will be addressed first: emotion recognition in multi-speaker language dependent conditions. Namely, this paper deals with the recognition of emotion for two Spanish speakers using standard hidden Markov models technology.

This paper explores the repercussion of contextual information into confidence measuring for continuous speech recognition results. Our approach comprises three steps: to extract confidence predictors out of recognition results, to compile those predictors into confidence measures by means of a fuzzy inference system whose parameters have been estimated, directly from examples, with an evolutionary strategy and, finally, to upgrade the confidence measures by the inclusion of contextual information. Through experimentation with two different continuous speech application tasks, results show that the context re-scoring procedure improves the capabilities of confidence measures to discriminate between correct and incorrect recognition results for every level of thresholding, even when a rather simple method to add contextual information is considered.

During the last years two different approaches have been widely used in order to improve the acoustic modeling in continuous speech recognition systems: discriminative training algorithms and context dependent subword units. However, while the use of each of these techniques leads to much better results than standard maximum likelihood trained phone models, their combination, i.e. discriminative training of context dependent units, has revealed to be a much more dificult task. In this paper we deal with minimum confusibility training of demiphones using TIMIT database. By applying this approach recently introduced by the authors, the string error rate in the recognition of TIDIGITS using demiphones is reduced some 24% with respect to maximum likelihood training. This improvement is added to the 8% reduction already provided by demiphones with respect to minimum confusibility trained phones.

In this paper, a task independent discriminative training framework for subword units based continuous speech recognition is presented. Instead of aiming at the optimisation of any task independent figure, say the phone classification or recognition rates, we focus our attention to the reduction of the number of errors committed by the system when a task is defined. This consideration leads to the use of a segmental approach based on the minimisation of the confusability over short chains of subword units. Using this framework, a reduction of 32% in the string error rate may be achieved in the recognition of unknown length digit strings using task independent phone like units.

X-grams are a generalization of the n-grams, where the number of previous conditioning words is different for each case and decided from the training data. X-grams reduce perplexity with respect to trigrams and need less number of parameters. In this paper, the representation of the x-grams using finite state automata is considered. This representation leads to a new model, the non-deterministic x-grams, an approximation that is much more efficient, suffering small degradation on the modeling capability. Empirical experiments for a continuous speech recognition task show how, for each ending word, the number of transitions is reduced from 1222 (the size of the lexicon) to around 66. 1. REVIEW OF X-GRAMS One of the main components in continuous speech recognition is the language model. The model has to estimate the a priori probability of each possible sentence. The most extended models are based on n-grams, basically bigrams or trigrams, which estimate the probability of each word t...

The performances of the demiphone (a context dependent subword unit that models independently the left and the right parts of a phoneme) and the triphone are compared. Continuous density hidden Markov modeling for both types of units is tested with the HTK software using decision-tree state clustering. The speech material is taken from the SpeechDat Spanish database, composed by continuous speech utterances recorded through the public telephone network. The training corpus is speaker and task independent. Two testing sets are tried: isolated words corresponding to speaker names, city names and phonetically rich words; and numbers of Spanish identification cards and dates. The main conclusion is that the demiphone simplifies the recognition system and yields a better performance than the triphone. This result may be explained by the ability of the demiphone to provide an excellent tradeoff between a detailed coarticulation modeling and a proper parameter estimation. 1. INTRODUCTION Ac...

It is well known that canonical Spanish, the dialectal variant `central' of Spain, so called Castilian, can be transcribed by rules. This paper deals with the automatic grapheme to phoneme transcription rules in several Spanish dialects from Latin America. Spanish is a language spoken by more than 300 million people, has an important geographical dispersion compared among other languages and has been historically influenced by many native languages. In this paper authors expand the Castilian transcription rules to a set of different dialectal variants of Latin America. Transcriptions are based on SAMPA symbols. The paper includes an identification of sounds that doesn't appear in Castilian, extend accepted SAMPA symbols for Spanish (Castilian) to different dialectal variants, describes the necessary rules to implement an automatic Orthographic to Phonetic transcription in several dialectal Spanish variants and show some quantitative results of dialectal differences.

Discriminative training is a powerful tool in acoustic modeling for automatic speech recognition. Its strength is based on the direct minimisation of the number of errors committed by the system at recognition time. This is usually accomplished by defining an auxiliary function that characterises the behaviour of the system, and adjusting the parameters of the system in a way that this function is minimised. The main drawback of this approach is that a task specific training database is needed. In this paper an alternative procedure is proposed: task adaptation using task independent databases. It consists in the combination of acoustic information|estimated using a general purpose training database|, and linguistic information|taken from the definition of the task|. In the experiments carried out, this technique has led to great improvement in the recognition of two different tasks: clean speech digit strings in English and dates in Spanish over the telephone wire.

Although having revealed to be a very powerful tool in acoustic modelling, discriminative training presents a major drawback: the lack of a formulation guaranteeing convergence in no matter which initial conditions, such as the Baum-Welch algorithm in maximum likelihood training. For this reason, a gradient descent search is usually used in this kind of problem. Unfortunately, standard gradient descent algorithms rely heavily on the election of the learning rates. This dependence is specially cumbersome because it represents that, at each run of the discriminative training procedure, a search should be carried out over the parameters ruling the algorithm. In this paper we describe an adaptive procedure for determining the optimal value of the step size at each iteration. While the calculus and memory overhead of the algorithm is negligible, results show less dependence on the initial learning rate than standard gradient descent and, using the same idea in order to apply self-scaling, it clearly outperforms it.

In this paper we introduce the demiphone as a contextual phonetic unit for continuous speech recognition. A phone is divided into two parts: a left demiphone that accounts for the left side coarticulation and a right demiphone that copes with the right side context. This new unit discards the dependence between the effects of both side contexts, but provides a better training of the transition between phones. The demiphone can be seen as a heuristic clustering of states that allows a more smoothed training of hidden Markov models and additionally supplies a simple way to create unseen triphones. We report experimental evidence that demiphones outperform the usual combination of triphones, right-side and left-side biphones and monophones. 1. INTRODUCTION Acoustic modeling for continuous speech recognition is a topic under permanent research, because the performance of a speech recognition system greatly depends on the acoustic modeling quality. Hidden Markov models (HMM) of phones are ...

In this paper, an extension of n-grams is proposed.In this extension, the memory of the model (n) is not fixed U priori.Instead, first, large memories are accepted and afterwards, merging criteria are applied to reduce complexity and to ensure reliable estimations.The results show how the perplexity obtained with x-grums is smaller than that of n-grums.Furthermore, the complexity is smaller than nigrum and can become close to bigrums.

In EuroSpeech'95, we presented the first version of Sethos, the speech understanding system which has been developed at the UPC.In this paper some improvements are incorporated at different levels of Sethos: language model, models of the semantic units and acoustic models.These improvements increase the percentage of correctly decoded sentences from 60% to 80%.Some experiments are presented to evaluate the influence of each information source on the final performance.Furthermore, the computational cost is analyzed arriving to an important conclusion: the configuration which gives the best performance is also the less expensive.The reason is that as better is the modeling, narrower is the beam of the search.

In this paper, a framework for discriminative training of acoustic models based on Generalised Probabilistic Descent (GPD) method is presented. The key feature of our proposal, Maximum Likelihood based Discriminative Training of Acoustic Models (MLDT), is the use of maximum likelihood trained HMM's instead of the original speech signal. We focus our attention in performing discriminative training applied to a discrete hidden Markov models continuos speech recogniser, achieving a 4.6% error rate reduction on a Spanish speaker-independent phoneme recognition task.

The performance of existing speech recognition systems degrades rapidly in the presence of background noise when training and testing cannot be done under the same ambient conditions. The aim of this paper is to propose the application of a simple multilabeling method, instead of the standard vector quantization -so called labeling-, as the front end for a speech recognizer based on the Vector Quantization (VQ) and Hidden Markov Models (HMM) approaches in order to increase its robustness to noise. Furthermore, not only cepstrum but also other features such as energy and dynamic parameters are evaluated and quantized independently in the multilabeling stage to represent more accurately characteristics of speech. The result of this process is a multiple multilabeling. Experimental results in the presence of additive white noise and car environment clearly demonstrate its good performance in isolated word recognition in noisy environments.

The problem of detecting a given set of words in fluent speech is one of the most interesting topics in speech recognition for practical real time applications. This paper present the TELEMACO system for automatic voice dialling which is based on the use of the keyword spotting technology to detect the dialling commands in fluent speech used by the IBERCOM Spanish telephone system. The user interface is based on a PC computer with a DSP board. The DSP board runs the speech recognition task and the interaction with the telephone line. The keyword vocabulary is composed by commands to dial, answer, hang-up, cancel, recall, store, etc. Each keyword is modeled by means of a discrete Hidden Markov Model. To model the non-keyword speech, syllabic fillers models and background models are used. The keyword spotting algorithm is a null grammar time-synchronous Viterbi search with two search spaces. The first search is over all the models (keywords and fillers) and the second search is only over the filler model. Thus, we can compare the behaviour of the filler model with the candidate keyword for each detection and decide if the keyword has been uttered or not. This process is done frame by frame. When a keyword is detected, the DSP board send the recognition word to the PC to take the corresponding action. The system has been implemented in a Windows environment.

This paper presents a combination of out-of-vocabulary (OOV) word modeling and rejection techniques in an attempt to accept utterances embedding a keyword and reject utterances with nonkeywords. The goal of this research is to develop a robust, task-independent Spanish keyword spotter and to develop a method for optimizing confidence thresholds for a particular context. To model OOV words, we employed both word and sub-word units as fillers, combined with n-gram language models. We also introduce a methodology for optimizing confidence thresholds to control the tradeoffs between acceptance, confirmation, and rejection of utterances. Our experiments are based on a Mexican Spanish auto-attendant system using the SpeechWorks recognizer release 6.5 Second Edition, in which we achieved a reduction in error of 8.9% as compared to the baseline system. Most of the error reduction is attributed to better keyword detection in utterances that contain both keywords and OOV words.

This paper describes the phonetic content of Albayzin, a spoken database for Spanish designed for speech recognition purposes. A statistical study of a large sample of spontaneous speech is presented, and the phonetic and statistical criteria for the final constitution of the database are discussed. Finally, the contents of the phonetic database are analyzed

The authors describe the acoustic processor of a Spanish continuous speech recognition system based on demisyllable units. The acoustic processor is based on a spotting algorithm which takes as input the unknown utterance, the HMM (hidden Markov model) of the reference demisyllables, and the lexical knowledge in terms of a finite state network. The spotting algorithm is a modified version of the one-step Viterbi algorithm with multiple hypotheses. The output of the system is a lattice of word hypotheses suitable to be parsed by a linguistic analyzer. The proposed acoustic processor was tested using the integers from 0 to 1000 and telephonic numbers in a speaker-independent approach. The results show the good performance of the demisyllable as a recognition unit for the Spanish language and the efficiency of the spotting algorithm.>

Peer Reviewed

A procedure for feature selection in isolated word recognition is discussed. The feature selection is performed in two steps. The first step takes into account the temporal correlation among feature vectors in order to obtain a transformation matrix which projects the initial template of N feature vectors to a new space where they are uncorrelated. This step gives a new template of M feature vectors, where M>

One of the most popular algorithms for connected word (or subword phonetic unit) recognition is the one-stage dynamic programming algorithm. In its available formulation, this algorithm is not designed to provide multiple hypothesis; such a limitation is currently becoming a drawback, since the need of alternative recognitions in the systems presently under research is being acknowledged. This paper introduces a modified version of the one-stage dynamic programming algorithm tailored to afford multiple hypothesis.

This communication reports the use of demisyllables for continuous speech recognition in a specific application: the recognition of Spanish numbers. After a brief outline of the recognition system, a description of demisyllable syntactic constraints and one-speaker reference generation is provided. Finally, the recognition performance is assessed by means of two experiments: the recognition of integer numbers from zero to one thousand and telephone numbers uttered in a Spanish way (strings of integers from zero to ninety nine), in both applications the results that the system yielded were excellent.

This paper presents a new algorithm to train multilayered perceptrons, using quadratical potential functions. This new algorithm is compared in an isolated word recognition task, with the back propagation algorithm that uses linear combinations of the inputs. Some pattern recognition techniques are also used to reduce the dimensinality of the input pattern in order to reduce the computational burden of the training and the recognition. The algorithm that uses quadratic potential functions yields better results in the recognition task.

In this work we present a waveform speech coding system including vector quantization. This system can be seen as a vector version of the scalar ADPCM speech coder. In such system the speech samples are grouped in vectors that are coded by a vector quantizer when its prediction subtraction has been made. This prediction is obtained in the coder by a vector predictor. Due to the non-stationarity of the speech signal, the code must be continuously adapted to the local characteristics of the current input signal of the speech. We propose the use of an adaptive vector predictor that follows the speech statistics variations so the prediction error to be coded presents the minimum dynamic range. On the other side it is wellknown that the prediction error is proportional to the signal energy. To compensate this effect a vector quantizer "gain-shape" model has been proposed, so vectors gain and its shape are separately coded. The obtained empirical results are very promising and exhibit good competitivity with other solutions existing in the literature.

The channel noise effects on linear delta modulation (LDM) systems have not yet been adequately analyzed. This paper presents a new and general formulation of these effects, based on the theoretical work by Wolf [1]. A comparative discussion of our formulas with previous results is also included. Finally, the application of our methods and the validity of our comments are illustrated by some numerical examples.