We describe here a collection of speech data of bilingual and trilingual speakers of English, French, German and Italian. In the context of speech to speech translation (S2ST), this database is designed for several purposes and studies: training CLSA systems (cross-language speaker adaptation), conveying emphasis through S2ST systems, and evaluating TTS systems. More precisely, 36 speakers judged as accentless (22 bilingual and 14 trilingual speakers) were recorded for a set of 171 prompts in two or three languages, amounting to a total of 24 hours of speech. These sets of prompts include 100 sentences from news, 25 sentences from Europarl, the same 25 sentences with one acted emphasised word, 20 semantically unpredictable sentences, and finally a 240-word long text. All in all, it yielded 64 bilingual session pairs of the six possible combinations of the four languages. The database is freely available for non-commercial use and scientific research purposes.

Phonemic or phonetic sub-word units are the most commonly used atomic elements to represent speech signals in modern ASRs.However they are not the optimal choice due to several reasons such as: large amount of effort required to handcraft a pronunciation dictionary, pronunciation variations, human mistakes and under-resourced dialects and languages.Here, we propose a data-driven pronunciation estimation and acoustic modeling method which only takes the orthographic transcription to jointly estimate a set of sub-word units and a reliable dictionary.Experimental results show that the proposed method which is based on semi-supervised training of a deep neural network largely outperforms phoneme based continuous speech recognition on the TIMIT dataset.

Feature subset selection, as a special case of the general subset selection problem, has been the topic of a considerable number of studies due to the growing importance of data-mining applications. In the feature subset selection problem there are two main issues that need to be addressed: (i) Finding an appropriate measure function than can be fairly fast and robustly computed for high-dimensional data. (ii) A search strategy to optimize the measure over the subset space in a reasonable amount of time. In this article mutual information between features and class labels is considered to be the measure function. Two series expansions for mutual information are proposed, and it is shown that most heuristic criteria suggested in the literature are truncated approximations of these expansions. It is well-known that searching the whole subset space is an NP-hard problem. Here, instead of the conventional sequential search algorithms, we suggest a parallel search strategy based on semidefinite programming (SDP) that can search through the subset space in polynomial time. By exploiting the similarities between the proposed algorithm and an instance of the maximum-cut problem in graph theory, the approximation ratio of this algorithm is derived and is compared with the approximation ratio of the backward elimination method. The experiments show that it can be misleading to judge the quality of a measure solely based on the classification accuracy, without taking the effect of the non-optimum search strategy into account.

Feature subset selection, as a special case of the general subset selection problem, attracted a lot of research attention due to the growing importance of data-mining applications. However, since finding the optimal subset is an NP-hard problem, very different heuristic search methods have been suggested to approximate it. Here we propose a new second-order cone programming based search strategy to efficiently solve the feature subset selection for large-scale problems. Experimentally, it is shown that its performance is almost always better than the greedy search methods especially when the features are strongly dependent.

Large recordings like radio broadcasts or audio books are an interesting additional resource for speech research but often have nothing but an orthographic transcription available in terms of annotation. The alignment between text and speech is an important first step for further processing. Conventionally, this is done by using automatic speech recognition (ASR) on the speech corpus and then aligning recognition result and transcription. This has the drawback that an ASR system needs to be available for the target language. In this paper, we introduce an approach based on forced alignment with hidden Markov models (HMM) normally applied only to shorter utterances. We show that by using a set of generalized phone models computed over phonetic groups, forced alignment is able to reliably align text and speech while being robust against transcription errors. In contrast to ASR methods, the alignment models can be used in a language-independent way. Index Terms: speech corpora annotation, sentence segmentation, large corpora processing

Given K utterances of a word and a set of sub-word units one may need a generalization of the conventional one-dimensional Viterbi algorithm to jointly decode them in order to derive their underlying word model (pronunciation). This extension is called k-dimensional Viterbi. However, as the number of utterances increases, the complexity of the k-dimensional Viterbi algorithm exponentially increases causing prohibitive computational burden. Here, we propose an approximation algorithm for the k-dimensional Viterbi which efficiently uses the available utterances to estimate the pronunciation. In addition to automatic dictionary generation, it can be used in computationally expensive applications such as lexicon-free training and joint pattern alignment. Index Terms: pronunciation, joint decoding, k-dimensional, viterbi

Under real conditions, severe signal cancellation often occurs in adaptive beamformers because of reverberation, microphone transfer function mismatch or steering vector error. Therefore, usually voice activity information is necessary to pause the beamformer update during the speech activity. However, this information is not available or not sufficiently accurate in most applications. Here we propose a new algorithm in order to mitigate the signal cancellation effect. The algorithm extracts the desired source-to-microphones transfer functions from the data covariance matrix by using rough estimates of some source locations. We show that it is robust against reverberation, microphone mismatch and imprecisely estimated target direction.

In this paper, we describe a prosody generation system for speech synthesis that makes direct use of syntax trees to obtain duration and pitch. Instead of transforming the tree through special rules or extracting isolated features from the tree, we make use of the tree structure itself to construct a superposi-tional model that is able to learn the relation between syntax and prosody. We implemented the system in our SVOX text-to-speech system and evaluated it against the existing rule-based system. Informal listening tests showed that structural informa-tion from the tree is carried over to the prosody. Index Terms: speech synthesis, prosody, syntax anaysis 1.

This paper deals with the problem of finding the optimal sequence of sub-word unit HMMs for a number of given utterances of a word. For this problem we present a new solution based on an extension of the Viterbi algorithm which maximizes the joint probability of the utterances and all possible sequences of sub-word units and hence guarantees to find the optimal solution. The new algorithm was applied in an isolated word recognition experiment and compared to simpler approaches to determining the sequence of sub-word units. We report a significant reduction of the recognition error rate with the new algorithm.

This paper presents a measure that scores various aspects of phone quality. The measure is designed to penalize phone instances with one or several characteristics that are not desirable in concatenation-based speech synthesis. Depending on the phone type, these aspects amongst others include spectrum, phase, fundamental frequency, duration, voicing and plosive quality. We applied this quality measure to select diphone sets from four different speech databases and demonstrate the quality of these diphone sets by means of synthesis examples. The quality of these examples showed that the proposed measure can be applied to select a high-quality diphone set from a speech database. Index Terms: speech synthesis, phone quality measure 1.

This paper describes a new approach to pitch marking. Unlike other approaches that use the same combination of features for the whole signal, we take into account the signal properties and apply different features according to some heuristic. Basically we use a special type of energy contour for pitch marking. Where the energy information turns out to be not suitable as an indicator we resort to the fundamental wave computed from a contiguous F0 contour in combination with detailed voicing information. Our experiments demonstrate that the proposed pitch marking algorithm clearly improves the quality of synthesised speech generated by a concatenative text-to-speech system that uses TD-PSOLA for prosodic modifications. Index Terms: speech synthesis, glottal closure instants, pitch marks, automatic pitch marking, signal analysis.

While automatic methods for phonetic segmentation of speech can help with rapid annotation of corpora, most methods rely either on manually segmented data to initially train the process or manual post-processing. This is very time-consuming and slows down porting of speech systems to new languages. In the context of prosody corpora for text-to-speech (TTS) systems, we investigated methods for fully automatic phoneme segmentation using only the corpora to be segmented and an automatically generated transcription. We present a new method that improves the performance of HMM-based segmentation by correcting the boundaries between the training stages of the phoneme models with high precision. We show that, while initially aimed at single speaker corpora, it performs equally well for multi-speaker corpora. Index Terms: speech synthesis, segmentation, boundary correction

We present a new method for the estimation of a continuous fundamental frequency (F0) contour. The algorithm implements a global optimization and yields virtually error-free F0 contours for high quality speech signals. Such F0 contours are subsequently used to extract a continuous fundamental wave. Some local properties of this wave, together with a number of other speech features allow to classify the frames of a speech signal into five classes: voiced, unvoiced, mixed, irregularly glottalized and silence. The presented F0 detection and frame classification can be applied to F0 modeling and prosodic modification of speech segments in high-quality concatenative speech synthesis. Index Terms: continuous F0 contour estimation, globally optimized F0 contour, voiced-unvoiced classification, irregular

We propose a new approach of integrating a precision grammar into speech recognition. The approach is based on a novel robust parsing technique and discriminative reranking. By reranking 100-best output of the LIMSI German broadcast news transcription system we achieved a significant reduction of the word error rate by 9.6 % relative. To our knowledge, this is the first significant improvement for a real-world broad-domain speech recognition task due to a precision grammar. Index Terms: speech recognition, precision grammar 1.

We pursue an approach to text-independent speaker verification which takes the decision whether two speech-signals are from the same speaker or not by matching segments that are common to both signals. A previous system using this pattern matching approach has shown good results but the used algorithm to seek for common segments was not satisfactory in terms of speed. In this paper we present an alternative algorithm which we developed to make this search faster. The approach which is based on hidden Markov models is unconstrained with respect to the size of the common segments which can be detected and it is basically language-independent. With the algorithm presented in this paper we could speed up the segment search by a factor of 5 while even improving the segments regarding their matching quality.

We present a new approach to quasi text-independent speaker verification based on pattern matching. Our method first seeks phonetically matched segments in two speech signals. For all aligned frame pairs of these segments we compute the probability that they were uttered by the same speaker. Based on these frame-level probabilities we take the decision whether the two signals were spoken by the same speaker or not. Our method to find phonetically matched segments does not depend on a speech recognizer. We show that our system performs better than a baseline speaker verification system based on Gaussian mixture models when the signals are long enough. Especially interesting is the fact that a combination of the devised system with the baseline system performs much better than either of the systems alone.

We show that an elaborate linguistic model of a natural language can be a valuable knowledge source to improve large vocabulary continuous speech recognition (LVCSR). Our approach is to complement a statistical language model with rule-based linguistic knowledge. A hidden Markov model based recognizer and an N-gram language model are used to compute a word lattice which is subsequently processed by a parser. We succeeded in enhancing recognition performance by favouring word sequences which the parser identified as being grammatically correct.

An automatic system for segmenting speech signals used for the training of statistical prosody models is presented. Starting from a canonical transcription, the system simultaneously delivers an accurate phonetic segmentation and the matched phonetic transcription indicating pronunciation variants. Although the system is HMM-based, it uses only the speech signals of the prosody database which typically consists of a few hundred sentences with some 30 minutes total duration. Initial phone HMMs are generated with flat-start training using the canonical transcriptions of the sentences. Then iterative Viterbi search for best-matching pronunciation variants and HMM retraining is applied until convergence is attained.

Polyglot text-to-speech synthesis, i.e. the synthesis of sentences containing one or more inclusions from other languages, primarily depends on an accurate morphosyntactic analyzer for such mixed-lingual texts. From the output of this analyzer, the pronunciation can be derived by means of phonological transformations which are language-specific and depend on various contexts. In this paper a new rule formalism for such phonological transformations is presented, which complies also with the requirements of the mixed-lingual situation.

A new approach to continuous speech recognition (CSR) for German is presented, which integrates both statistical knowledge (at the acoustic-phonetic level) and rule-based knowledge (at the word and sentence levels). We introduce a flexible framework allowing bidirectional processing and virtually any search strategy given an acoustic model and a context-free grammar. An implementation of this class of recognizers by means of a word spotter and an island chart parser is presented. A word recognition accuracy of 93.5% is reported on a speaker dependent recognition task with a 4k words dictionary.

In forensic casework, the application of automatic speaker verification (SV) aims to determine the likelihood ratio of a suspect being vs. being not the speaker of an incriminating speech recording. For that purpose, the likelihood of the anti-speaker has to be estimated from the speech of an adequate number of other speakers. In many cases, speech signals of such an anti-speaker population are not available and it is generally too expensive to make an appropriate collection.

Text-to-speech (TTS) synthesis is more and more confronted with the language mixing phenomenon. An important step towards the solution of this problem and thus towards a socalled polyglot TTS system is an analysis component for mixedlingual texts. In this paper it is shown how such an analyzer can be realized for a set of languages, starting from a corresponding set of monolingual analyzers which are based on DCGs and chart parsing.

The paper addresses the problem of designing a language independent phonetic inventory for the speech recognisers with multilingual vocabulary. A new clustering algorithm for the definition of multilingual set of triphones is proposed. The clustering algorithm bases on a definition of a distance measure for triphones defined as a weighted sum of explicit estimates of the context similarity on a monophone level. The monophone similarity estimation method based on the algorithm of Houtgast. The clustering algorithm is integrated in a multilingual speech recognition system based on HTK V2.1.1. The ongoing experiments are based on the SpeechDat II databases. So far, experiments included the Slovenian, Spanish and German 1000 FDB SpeechDat (II) database. Current results are very promising. The use of clustering algorithm resulted in a significant reduction of the number of triphones at acceptable level of word and language identification accuracy degradation.

The aim of this work was to develop a procedure that allows prosodic modications of speech signals without impairing the quality.The developed procedure is based on the Fourier analysis/synthesis technique with several improvements on the analysis side, such as the analysis of signals with rapidly changing F0 and the analysis of weak spectral components.Also for the modication of the short-time spectrum and for the reconstruction of the speech signal some new methods have been introduced.The most important one, in terms of speech quality, is the way of phase compensation that limits the absolute time shift to half the pitch period.The developed procedure is used in our high-quality text-tospeech synthesis system that is based on concatenation of prosodically modied diphones.

A programmable real-time speech processor is presented. The design goal was a high-speed, special-purpose machine, on which different, even very complex speech processing algorithms can be investigated in real-time. The hardware, consisting of a 24-bit word-length signal processor based on the MC 10800 family of ECL LSI micro-processor circuits, a host processor (INTEL 8080) and a double-sided diskette storage drive, is described from the architectural viewpoint, and its microprogramming features are presented. Finally, it is reported on the implementation of a linear predictive coding vocoder illustrating the speech processor's performance.