In this paper, we introduce two new confidence measures for large vocabulary speech recognition systems. The major feature of these measures is that they can be computed without waiting for the end of the audio stream. We proposed two kinds of confidence measures: frame-synchronous and local. The frame-synchronous ones can be computed as soon as a frame is processed by the recognition engine and are based on a likelihood ratio. The local measures estimate a local posterior probability in the vicinity of the word to analyze. We evaluated our confidence measures within the framework of the automatic transcription of French broadcast news with the EER criterion. Our local measures achieved results very close to the best state-of-the-art measure (EER of 23% compared to 22.0%). We then conducted a preliminary experiment to assess the contribution of our confidence measure in improving the comprehension of an automatic transcription for the hearing impaired. We introduced several modalities to highlight words of low confidence in this transcription. We showed that these modalities used with our local confidence measure improved the comprehension of automatic transcription.

International audience

International audience

International audience

In this paper, we present the idea of cross-lingual triggers. We exploit this formalism in order to build up a bilingual dictionary for machine translation. We describe the idea of crosslingual triggers, the way to exploit and to make good use of them in order to produce a bilingual dictionary. We then compare it to ELRA and a free downloaded dictionaries. Finally, our dictionary is evaluated by comparing it to the one achieved by GIZA++ [1] (which is an extension of the program GIZA [2]) into an entire translation decoding process supplied by Pharaoh [3]. The experiments showed that the obtained dictionary is well constructed and is suitable for machine translation. The experiments have been conducted on a parallel corpus of 19 million French words and of 17 million English words. Finally, the encouraging results allow us to put forward the concept of cross-lingual triggers which could have so many applications in machine translation. 1.

This paper presents a fully automated approach for the recognition of non-native speech based on acoustic model modification. For a native language (L1) and a spoken language (L2), pronunciation variants of the phones of L2 are automatically extracted from an existing non-native database as a confusion matrix with sequences of phones of L1. This is done using LTs and L2's ASR systems. This confusion concept deals with the problem of non existence of match between some L2 and L1 phones. The confusion matrix is then used to modify the acoustic models (HMMs) of L2 phones by integrating corresponding L1 phone models as alternative HMM paths. We introduce graphemic constraints in the confusion extraction process: the phonetic confusion is established for each couple of 'L2-phone' and the grapheme(s) corresponding to that phone. We claim that pronunciation errors may depend on the graphemes related to each phone. The modified ASR system achieved an improvement between 32% and 40% (relative, L1=French and L2=English) in WER on the French non-native database used for testing. The introduction of graphemic constraints in the phonetic confusion allowed further improvements

This paper addresses the problem of spectrographic mask estimation in the context of missing data recognition. At the difference of other denoising methods, missing data recognition does not match the whole spectrum with the acoustic models, but rather considers that some time-frequency pixels are missing, i.e. corrupted by noise. Correctly estimating these "masks" is very important for missing data recognizers. We propose a new approach that exploits some a priori knowledge about these masks in typical noisy environments to address this difficult challenge. The proposed mask is then obtained by combining these noise dependent masks. The combination is led by an environmental "sniffing" module that estimates the probability of being in each typical noisy condition. This missing data mask estimation procedure has been integrated in a complete missing data recognizer using bounded marginalization. Our approach is evaluated on the Auroral database

In this paper we present an automated approach for non-native speech recognition. We introduce a new phonetic confusion concept that associates sequences of native language (NL) phones to spoken language (SL) phones. Phonetic confusion rules are automatically extracted from a non-native speech database for a given NL and SL using both NL's and SL's ASR systems. These rules are used to modify the acoustic models (HMMs) of SL's ASR by adding acoustic models of NL's phones according to these rules. As pronunciation errors that non-native speakers produce depend on the writing of the words, we have also used graphemic constraints in the phonetic confusion extraction process. In the lexicon, the phones in words' pronunciations are linked to the corresponding graphemes (characters) of the word. In this way, the phonetic confusion is established between couples of (SL phones, graphemes) and sequences of NL phones. We evaluated our approach on French, Italian, Spanish and Greek non-native speech databases. The spoken language is English. The modified ASR system achieved significant improvements ranging from 20.3% to 43.2% (relative) in sentence error rate and from 26.6% to 50.0% in WER.

Missing data recognition has been developped in order to increase noise robustness in automatic speech recognition. Many different factors, including the speech decoding process itself, shall be considered to locate the masks. In this work, we are considering Bayesian models of the masks, where every spectral feature is classified as reliable or masked, and is independent from the rest of the signal. This classification strategy can produce unrelated small ``spots'', while experiments suggest that oracle reliable and unreliable features tend to be clustered into time-frequency blocks. We call this undesired effect: the ``checkerboard'' effect. In this paper, we propose a new Bayesian missing data classifier that integrates frequency and temporal constraints in order to reduce, or avoid, this ``checkerboard'' effect. The proposed classifier is evaluated on the Aurora2 connected digit corpora. Integrating such constraints in the missing data classification leads to significant improvements in recognition accuracy.

The problem of speech/music discrimination is a challenging research problem which significantly impacts Automatic Speech Recognition (ASR) performance. This paper proposes new features for the Speech/Music discrimination task. We propose to use a decomposition of the audio signal based on wavelets, which allows a good analysis of non stationary signal like speech or music. We compute different energy types in each frequency band obtained from wavelet decomposition. Two class/non-class classifiers are used : one for speech/non-speech, one for music/non-music. On the different test corpora, the proposed wavelet approach gives better results than the MFCC one. For instance, we have a significant relative improvements of the error rate of 71.6% on the ``Scheirer'' corpus for the speech/music discrimination task.

The agreement in gender and number is a critical problem in sta-tistical language modeling. One of the main difficulties in speech recognition of French language is the presence of misrecognized words due to the bad agreement (in gender and number) between words. Statistical language models do not treat this phenomena di-rectly. This paper focuses on how to handle the issue of this agree-ment. We introduce an original model called Features-Cache (FC) to estimate the gender and the number of the word to predict. It is a dynamic variable-length Features-Cache. The size of the cache is automatically determined in accordance to syntagm delimitors. The main advantage of this model is that there is no need to any syntactic parsing: it is used as any other statistical language model. Several models have been carried out and the best one achieves an improvement of approximatively 9 points in terms of perplexity. This model has been integrated in a speech recognition system ba-sed on JULIUS engine. Tests have been carried out on 280 sen-tences provided by AUPELF for the French automatic speech re-cognition evaluation campaign. This new model outperforms the baseline one, in terms of word error, by 3%.

This paper presents a fully automated approach for the recognition of non-native speech based on acoustic model modification. For a native language (L1) and a spoken language (L2), pronunciation variants of the phones of L2 are automatically extracted from an existing non-native database as a confusion matrix with sequences of phones of L1. This is done using L1's and L2's ASR systems. This confusion concept deals with the problem of non existence of match between some L2 and L1 phones. The confusion matrix is then used to modify the acoustic models (HMMs) of L2 phones by integrating corresponding L1 phone models as alternative HMM paths. In this way, no lexicon modification is carried. The modified ASR system achieved an improvement between 32% and 40% (relative, L1=French and L2=English) in WER on the French non-native database used for testing.

This paper presents some confidence measures for large vocabulary speech recognition which are based on word graph or N-Best List structures. More and more applications need fast estimation of any measures in order to stay real-time. We propose some simple and fast measures, locally computed, that can be directly used within the first decoding recognition process. We also define some other measures combining word graph and N-Best List. Experimental results on a French broadcast news corpus are also presented.

Colloque avec actes et comité de lecture. internationale.

Support vector machines (SVM) is a new and very promising classification technique developed from the theory of structural risk minimisation. We propose an alternative out-of-vocabulary word detection method relying on confidence measures and support vector machines. Confidence measures are computed from phone level information provided by a hidden Markov model (HMM) based speech recognizer. We use three kinds of average techniques as arithmetic, geometric and harmonic averages to compute a confidence measure for each word. The acceptance/rejection decision of a word is based on the confidence feature vector which is processed by a SVM classifier. The performance of the proposed SVM classifier is compared with methods based on the averaging of confidence measures.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

This work presents several statistical methods for topic identification on two kinds of textual data: newspaper articles and e-mails. Five methods are tested on these two corpora: topic unigrams, cache model, TFIDF classijier, topic peqdexity, and weighted model. Our work aims to study these methods by confronting them to very diferent data. This study is very fruitful for our research. Statistical topic identiJication methods depend not only on a corpus, but also on its type. One of the methods achieves a topic identiJcation of 80% on a general newspaper corpus but does not exceed 30% on e-mail corpus. Another method gives the best result on e-mails, but has not the same behavior on a newspaper corpus. We also show in this paper that almost all our methods achieve good results in retrieving the first two manually annotated labels.

In this paper, an algorithm for continuous speech recognition systems based on the multi-band principle is proposed. This algorithm allows the bands to be asynchronous and has a practical complexity that is very close to the complexity of the classical Viterbi algorithm. The question of whether the bands should be constrained to be synchronous or not is discussed. We show that it is advantageous to let the bands be asynchronous, as the increase of complexity, compared to the Viterbi algorithm, is low with our algorithm. Moreover, the accuracy must be at least as good as when the bands are synchronous, and, more importantly, different models than phones, can be used in the bands.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

Colloque avec actes et comité de lecture. internationale.

We present several methods for topic detection on newspaper articles, using either a general vocabulary or topic-specific vocabularies. Specific vocabularies are determined manually or statistically. In both cases, we aim at finding the most representative words of a topic. Several methods have been experimented, the first one is based on perplexity, this method achieves a 100% topic identification rate, on large test corpora, when the two first propositions are taken into account. Other methods are based on statistical counts and achieve 94% of identification on smaller test corpora. The major challenge of this work is to identify topics with only few words in order to be able, during speech recognition, to determine the best adequate language model.

Colloque avec actes et comité de lecture.

Colloque avec actes et comité de lecture.

An ubiquitous phenomenon in psychology is the `repetition effect': a repeated stimulus is processed better on the second occurrence than on the first. Yet, what counts as a repetition? When a spoken word is repeated, is it the acoustic shape or the linguistic type that matters? In the present study, we contrasted the contribution of acoustic and phonological features by using participants with different linguistic backgrounds: they came from two populations sharing a common vocabulary (Catalan) yet possessing different phonemic systems. They performed a lexical decision task with lists containing words that were repeated verbatim, as well as words that were repeated with one phonetic feature changed. The feature changes were phonemic, i.e. linguistically relevant, for one population, but not for the other. The results revealed that the repetition effect was modulated by linguistic, not acoustic, similarity: it depended on the subjects' phonemic system. \n

Colloque avec actes et comité de lecture.

An adaptation algorithm using the theoretically optimal maximum a posteriori (MAP) formulation, and at the same time accounting for parameter correlation between different classes is desirable, especially when using sparse adaptation data. However, a direct implementation of such an approach may be prohibitive in many practical situations. We present an algorithm that approximates the above mentioned correlated MAP algorithm by iteratively maximizing the set of posterior marginals. With some simplifying assumptions, expressions for these marginals are then derived, using the principle of minimum cross-entropy. The resulting algorithm is simple, and includes conventional MAP estimation as a special case. The utility of the proposed method is tested in adaptation experiments for an alphabet recognition task.

Adaptation techniques that benefit from distribution correlation are important in practical situations having sparse adaptation data. The so called extended MAP (EMAP) algorithm provides an optimal, though expensive, solution. In this article we start from EMAP, and propose an approximate optimisation criterion, based on maximising a set of local densities. We then obtain expressions for these local densities based on the principle of minimum cross-entropy (MCE). The solution to the MCE problem is obtained using an analogy with MAP estimation, and avoids the use of complex numerical procedures, thus resulting in a simple adaptation algorithm. The implementation of the proposed method for the adaptation of HMMs with mixture Gaussian densities is discussed, and its efficiency is evaluated on an alphabet recognition task.

We describe a continuous speech recognition system that uses the multi-band paradigm. This principle is based on the recombination of several independent sub-recognizers, each one assigned to a specific frequency band. The major issue of such systems consists of deciding at which time the recombination must be done. Our algorithm lets each band be totally independent from the others, and uses the different solutions to resegment the initial sentence. Finally, the bands are synchronously merged together, according to this new segmentation. The whole system is too complex to be entirely described here, and, in this paper, we concentrate on the synchronous recombination part, which is achieved by a classifier. The system has been tested in clean and noisy environments, and proved to be especially robust to noise.

International audience

A unified approach to the acoustic mismatch problem is proposed. A maximum likelihood state-based additive bias compensation algorithm is developed for the continuous density hidden Markov model (CDHMM). Based on this technique, specific bias models in the mel cepstral and the linear spectral domains are presented. Among these models, a new polynomial trend bias model in the mel cepstral domain is derived, which proved effective for Lombard speech compensation. In addition, a joint estimation algorithm for additive and convolutive bias compensation is proposed. This algorithm is based on applying the expectation maximization (EM) technique in both above-mentioned domains, in conjunction with a parallel model combination (PMC) based transformation. The compensation of the dynamic (difference) coefficients in the proposed framework is also studied. The evaluation data base consists of a 21 confusable word vocabulary uttered by 24 speakers. Three mismatched versions of the data base are considered, i.e., Lombard speech, 15 dB noisy Lombard speech, and 5 dB noisy Lombard speech. The proposed techniques result in 50.9%, 74.6%, and 67.3% reduction in the performance difference between matched and uncompensated word error rates for the three mismatch conditions, respectively. When dynamic coefficients are considered the corresponding reductions are 46.8%, 72.4%, and 70.9%.

In the context of continuous density hidden Markov model (CDHMM) we present a unified maximum likelihood (ML) approach to acoustic mismatch compensation. This is achieved by introducing additive Gaussian biases at the state level in both the mel cepstral and linear spectral domains. Flexible modelling of different mismatch effects can be obtained through appropriate bias tying. A maximum likelihood approach for joint estimation of both mel cepstral and linear spectral biases from the observed mismatched speech given only one set of clean speech models is presented, where the obtained bias estimates are used for the compensation of clean speech models during decoding. The proposed approach is applied to the recognition of noisy Lombard speech, and significant improvement in the word recognition rate is achieved.

In segment based recognizers, variable length speech segments are mapped to the basic speech units (phones, diphones, ...). We address the acoustical modeling of these basic units in the framework of segmental posterior distribution models (SPDM). The joint posterior probability of a unit sequence u_ and a segmentation s_, Pr(u_,s_|X_) can be written as the product of the segmentation probability Pr(s_|X_) and the unit classification probability Pr(u_|s_,X_), where X_ is the sequence of acoustic observation parameter vectors. In particular, we point out the role of the segmentation probability and demonstrate that it does improve the recognition accuracy. We present evidence for this in two different tasks (speaker dependent continuous word recognition in French and speaker independent phone recognition in American English) in combination with two different unit classification models.

The problem addressed by this paper is to enhance the continuous speech recognizers robustness to noise. For this purpose, the acoustic signal is filtered into several spectral bands, and independent recognition is achieved in each band. Then, the system recombines the results given by each recognizer and delivers a unique solution. The main advantage of this method is to consider the signal only in the bands which are relevant, and to ignore spectral bands which are corrupted by noise. We are developping a speaker-independent continuous speech recognizer based on this principle.

International audience

We propose an extension of the Viterbi algorithm that makes second-order hidden Markov models computationally efficient. A comparative study between first-order (HMM1s) and second-order Markov models (HMM2s) is carried out. Experimental results show that HMM2s provide a better state occupancy modeling and, alone, have performances comparable with HMM1s plus postprocessing.

The expectation-maximization (EM) algorithm is a general technique for maximum likelihood estimation (MLE). In this paper, we present two important theoretical issues concerning Gaussian mixture modeling (GMM) within the EM framework. First, we propose an EM algorithm for estimating the parameters of a GMM structure dedicated to speaker recognition, the probabilistic mapping network (PMN), where the Gaussian probability density function is realized as an internal node. Hence, the EM algorithm is extended to deal with the supervised learning of a multicategory classification problem and serves as a parameter estimator of the neural network classifier. Then, a generalized EM (GEM) algorithm is developed as an alternative to the MLE problem of PMN. The effectiveness of the proposed PMN architecture and developed EM algorithms are assessed by conducting a set of speaker recognition experiments. It is shown that GEM converges faster than EM to the same solution space.

In this paper, a trajectory mixture hidden Markov model (TMHMM) is proposed to cope with the problems of trajectory variability and speaker time-drifting. A theoretical formulation of TMHMM learning is presented. By introducing two pragmatic adaptation schemes, the practical issues which demonstrate the use of the model in capturing the time-drifting of speaker model for speaker recognition are addressed. To evaluate with the YOHO corpus, a set of phonetic units is defined. The effectiveness of the modeling approach is confirmed by a set of experiments. It is shown that an error rate of 0.07% is obtained for closed-set speaker recognition with a total population of 138 talkers. TMHMM can be considered as a special HMM topology dedicated to the time-drifting adaptation problem.

Source localisation is among the most important steps in array processing. The authors present a neuromimetic approach in signal processing. A set of neural networks is used to find the azimuth of one or several sources impinging on a linear array of equally spaced sensors. Each network in this set is specialised to determine if there is an emitter in a given angular sector. Thus a neural network has a specific architecture suited to detect and enhance the signal coming from the angular sector it is associated with. The performances of this method on real underwater signals confirm the encouraging results obtained on simulation tests.>

We present a new linked predictive neural network structure-hierarchical LPNN-and the speech recognition system based on it. This neural network is specially developed for the noise degraded speech recognition problem and integrates the speech recognition and noise reduction in a unified structure. The network is trained according to joint optimization criterion both for the signal prediction and for the noise reduction. We present results which indicate that joint optimization results in a better speech recognition performance of the all over system for noise degraded speech signals.>

Models observations of phoneme-based speech units as clusters of trajectories in their parameter space. The trajectories are modeled by a mixture of state sequences of multi-variate Gaussian density functions, optimized at the state sequence level. The duration of trajectories are integrated in the modeling. The authors also provide an algorithm for sentence recognition based on the modeling. In an alphabet recognition task the resulting system trained in context-independent mode demonstrated substantially better recognition accuracy, compared to a conventional context-dependent, whole word HMM.>

Reasoning about time in presence of incomplete information is central in a wide range of artificial intelligence applications. For this purpose, time must be accounted for while reasoning about assumptions. To deal with the hypothetical dimension of inference, De Kleer's assumption-based truth maintenance approach proved valuable in many situations. However, it cannot handle temporally-qualified assertions. This paper contributes to fill the gap. It addresses the problem of performing assumption-based truth maintenance in a window-based temporal logic.>

The problem of producing timely responses when faced with deadlines is an important issue for real-time systems. To deal with this problem different kinds of scheduling algorithms have been developed within the real-time community. Unfortunately, these approaches fail when applied to knowledge based systems, mainly because AI techniques rely on time-consuming algorithms with unpredictable, or highly variable performances. To face this problem researchers in AI have introduced deliberative techniques that enable to adapt the way a task is working in function of the available time. For this purpose approximation algorithms of two kinds have been developed: iterative refinement and multiple methods. The model we propose in this paper, belongs to iterative refinement methods. The main advantage of our approach is the capability of designing iterative refinement methods using a general rule language.>

A major step in the process of speech understanding is the acoustic-phonetic decoding which can be defined as the automatic mapping of the continuous speech wave into a set of predetermined linguistic units such as phones, diphones, syllables, etc. This paper relates to the approach of this problem which consists in exploiting an explicit description of all kinds of available knowledge about the speech communication phenomena, in the general framework of an artificial intelligence knowledge-based system. We will first recall the main difficulties of acoustic-phonetic decoding with a practical example. We will then present the APHODEX system that we have been designing for the past eight years, in terms of software architecture and of knowledge representation and reasoning. The practical evaluation of this system will then be carried out at the different levels of feature extraction, segmentation and labelling. Finally, we will discuss the limitations of our approach and present the ongoing effort to overcome these limitations, especially through the use of abductive reasoning.

International audience

The authors address the problem of speaker recognition using very short utterances, both for training and for recognition. The authors propose to exploit speaker-specific correlations between two suitably defined parameter vector sequences. A nonlinear vectorial interpolation technique is used to capture speaker-specific information, through least-square-error minimization. The experiments show the feasibility of recognizing a speaker among a population of about 100 persons using only an utterance of one word both for training and for recognition.>

To deal with complex problems, many kinds of reasoning and representation techniques are needed. The cooperation between different reasoning models constitutes one of the objectives of second-generation systems. The need to mix temporal and hypothetical reasoning for qualitative simulation is highlighted. A system built around an ATMS, a TCP, and a model that reasons on orders of magnitude is proposed. This system relies on an extension of HEART (see C. Joubel and O. Raiman, 1990). A new method for making histories concise has been developed.>

communication to : Proc. Intern. Conf. on Spoken Language Processing, Banff (Alberta, Canada), October 1992

Extrait de : Proc. International Conference on Spoken Language Processing, Banff (Alberta, Canada), October 1992

The authors present an NN-IIR (neural network/infinite impulse response filter) system for phoneme recognition in continuous speech based on the idea of modeling the recognition process by state evolution and interpretation equations. This work gives a solution to temporal information representation in phoneme recognition using neural networks and recursive filters, yielding better recognition results for continuous speech. This recognition system has two promising properties, i.e., capabilities for dealing with sequential properties and for interpreting speech signals by means of a training process. It was shown experimentally that the NN-IIR network obtained good performance for continuous speech recognition. Preliminary experiments with limited training data indicate that the NN-IIR provides good discrimination power for plosives, which are highly context-dependent.>

The correlations between vectors in a sequence of analysis frames are supposed to be specific to phonetic units in acoustic-phonetic decoding of speech. The authors propose nonlinear vector interpolation techniques to represent this correlation and to recognize phonemes. The interpolation is based on the decomposition of a frame sequence into two parts and on the construction of a function that interpolates one part using information from the second part. According to quantities to be interpolated, three families of interpolator models are developed. In a recognition system, each phonemic symbol is associated with a nonlinear vector interpolator which is trained to give minimum interpolation error for that specific phoneme. Multilayer feedforward neural networks are used to implement the nonlinear vector interpolators. For continuous speech under the phoneme spotting test using 16 PLCC-derived cepstrum coefficients as parametric vectors, the three categories of models gave compatible results.>

The authors propose an approach to phoneme-based continuous speech recognition when a time function of the plausibility of observing each phoneme (spotting result) is given. They introduce a criterion for the best sentence, based on the sum of plausibilities of individual symbols composing the sentence. Based on the idea of making use of high plausibility regions to reduce the computational load while maintaining optimality, the method finds the most plausible sentences relating to the input speech. Two optimization procedures are defined to deal with the following embedded search processes: (1) finding the best path connecting peaks of the plausibility functions of two successive symbols, and (2) finding the best time transition slot index for two given peaks. Experimental results show that the method gives better recognition precision while requiring about 1/20 of the computing time of the traditional DP-based methods. The experimental system obtained a 95% sentence recognition rate on a multispeaker test.>

A new temporal representation combining the notions of intervals, dates, and durations is presented. The manipulation of this representation is based on the notion of time map managers (TMMs) allowing both kinds of constraints, symbolic or numeric. These algorithms are a generalization of AC4, an optimal algorithm for arc-consistency, and can handle n-ary constraints.>

International audience

A method of signal-to-string conversion based on embedded dynamic programming (DP) which can adapt its search to the variation of the input signal is proposed. The optimizing process is guided by high-valued portions of the likelihood function of symbols composing the string and is solved by two embedded dynamic programming processes. Algorithms in a Pascal-like language relating to the solution are given. When applied to continuous speech recognition on a 100-word vocabulary using the phoneme as the basic recognition unit, the method is shown to achieve a 4% improvement in the recognition rate compared to a classical DP-based method.>

The principle of trajectory space comparison for text-independent speaker recognition and some solutions to the space comparison problem based on vector quantization are presented. The comparison of the recognition rates of different solutions is reported. The experimental system achieves a 99.5% text-independent speaker recognition rate for 23 speakers, using five phrases for training and five for test. A speaker-independent continuous speech recognition system is built in which this principle is used for speaker adaptation.>

A recognizer for isolated words based on second order hidden Markov models (HMMs) is presented. This system is compared with the usual first order one. To evaluate the performances of both systems, a database of French digits is used. Results are more accurate with the recognizer using second-order hidden Markov process. A version of the Viterbi algorithm which is suited to continuous-speech recognition is described. This version, called Viterbi-block allows segmentation and labeling of an unknown utterance during its processing, whereas the classical Viterbi algorithm waits until the end of the sentence before backtracking the best path. This algorithm is based on a local optimum comparison of the different probabilities computed by the Viterbi algorithm during the time warping of a shift window of fixed length in the signal and the different HMMs. Different criteria of comparison between the probabilities of different paths are tried and assessed.>

The signal-to-symbol conversion level of a generic signal interpretation system is presented. Using an application-independent structure, the system can compile user-supplied application-specific task descriptions to generate programs capable of performing statistic and structural pattern recognition, and reasoning in a given application domain. The computation of three types of symbols, quantitative symbols, qualitative symbols, and compound symbols, is outlined. The notion of context of symbols and its use in the system are discussed. In this system, symbolic processing, signal processing routine, and similarity measure can be specified in terms of situations. An evaluation of the system performance in an application of nondestructive testing of a steam generator using eddy current signals is given.>

A spoken Chinese understanding system which accepts continuous speech and produces Lisp-executable functions is presented. A multiple-knowledge-source organization model, syntactic structure construction by forward-backward inference, a phoneme model of speech profiles, and knowledge-based tone classification are proposed and implemented in order to remove the uncertainty of speech signals. Experimental results are given and discussed. In the single-speaker mode, 90% of the test sentences were recognized as first proposition by the system. 3% were rejected, and the rest were given within the five propositions of best quality.>

Presents the basic principles and some preliminary applications of a novel connectionist model for building up a neural network system. The model uses the cortical column instead of the classical neuron as the basic processing unit; it is closely related to the neurobiological modeling of the human cortex. The main features of this model which make it different from classical approaches concern the local connectivity and the integration of time by causality learning. The model provides a basic unit well -adapted to solve humanlike problems by integrating particular difficulties in its own structure (e.g. the coarticulation effect in speech recognition). The model has been applied to two difficult problems in the artificial intelligence field: speech recognition (more precisely, the acoustic-phonetic decoding of continuous speech) and biomedical X-ray image interpretation. The two systems that were designed for these applications demonstrate the ability of the cortical column to solve perceptive and cognitive tasks

Proceedings 1990 international conference on spoken language processing, Kobe (Japon), novembre 1990

Publie dans : Proceedings ICSLP90 (International conference on spoken language processing), Kobe (Japan), November 1990

The authors propose a novel approach to neuronlike models that consists in simulating cortical columns, i.e. associations of neurons having a specific, functional activity. This model is built according to a theory which is consistent with neurobiological data. The model is implemented through a network of columns that simulate the inherently parallel functioning of the nervous system. A description is given of this model, and the experimental conditions in which it is now being tested are described.>

Proceedings European conference of speech processing and technology, pages 268-271, Paris, September 1989

Publie dans : Proceedings EUROSPEECH 89 (European conference on speech communication and technology), Paris, September 1989

The authors propose a homogeneous architecture for organizing and controlling multiple knowledge sources in signal interpretation systems based on the decomposition of the interpretation problem into subproblems at successive conceptual levels. Partial interpretations and knowledge about the problem are partitioned into associations each of which consisting of several specialists. Information exchange is assured by a common data structure within knowledge sources at each given level and by a message passing mechanism between two different levels. Different strategies adapted to the nature of the problem may be implemented at each level. The interpretation process consists in executing the groups of knowledge sources in multiple phases controlled by data and models in each concept domain and allowing incremental construction of solutions. The authors illustrate the architecture by an application to continuous spoken Chinese understanding.>

The various approaches proposed to solve the difficult problem of acoustic phonetic decoding (PD) are briefly recalled, including the hidden Markov model (HMM), which appears to be particularly efficient. It is then proposed to consider PD as a knowledge-intensive process, and the issues involved in the knowledge-based approach to this problem are identified. The APHODEX (acoustic expert) phonetic decoding system is used to illustrate this approach. Results obtained show that the knowledge acquired from expert phoneticians can substantially improve the performances of PD systems. The incorporation of this knowledge into efficient operational models such as HMMs or neural networks represents a good basis for further developments in the field.>

In the framework of man-machine communication, oral dialogue has a particular place since human speech presents several advantages when used either alone or in multimedia interfaces. The last decade has witnessed a proliferation of research into speech recognition and understanding, but few systems have been defined with a view to managing and understanding an actual man-machine dialogue. The PARTNER system that we describe in this paper proposes a solution in the case of task oriented dialogue with the use of artificial languages. A description of the essential characteristics of dialogue systems is followed by a presentation of the architecture and the principles of the PARTNER system. Finally, we present the most recent results obtained in the oral management of electronic mail in French and German.

We present in this paper an application of a continuous speech understanding system to the control of a sonar console by a human operator. This application is really useful since it does correspond to a practical need of the operator who has his eyes busy looking at the sonar screen. It presents three original features: - the acoustic-phonetic decoding part of the system is not based on a classical phoneme labeling process but it yields for each segment of speech a set of acoustic phonetic labels that describe this segment very precisely. This phonetic labeling is associated with a special procedure for lexical access; - the system is actually a man-machine dialog system and not only a system capable of understanding a single sentence as it is often the case. The history of the dialog is used as a special knowledge source during the understanding process. This point highly increases the overall performance of the system; - the understanding process, and particularly the emission of hypotheses, is based on the knowledge about the language (lexicon and syntax) but also on the context of the dialog ; pragmatic knowledge is thus intimately associated with the analysis of a sentence. The paper presents the architecture of the system and its various components. It also discusses experimental results obtained in a multispeaker mode.

We present in this paper a new approach to acoustic-phonetic decoding of continuous speech that consists of integrating the two most promising present techniques in the field, i.e. stochastic modeling and knowledge-based expert systems. Our group has been developing several systems based on these two techniques during the past 15 years or so. Our present goal is to mix both in order to improve the overall quality of automatic phonetic decoding. This paper is concerned with the first developments of the project. We first present the main characteristics of the components of the system and its general architecture. We also present and discuss preliminary results concerning the segmentation of the speech wave into phonetic units and the gross labeling of these segments.

The acoustic-phonetic decoding of sentences is a major bottleneck in continuous speech recognition. Our group has been working in this area for the past twelve years. In order to improve the accuracy of present phonetic decoders, we are now developing an expert system called APHODEX that should emulate the competence of an experienced spectrogram reader. After a brief description of the methodology used to elucidate the expert's knowledge and strategies, we focus on the expert's approach and results. Finally we explain how this expertise has been implemented; some results from the present experimental version of our system are given.

The acoustic-phonetic decoding of complete sentences is a major bottleneck in continuous speech recognition. Our group, who has been working in this area for the past twelve years, first developed systems based on synchronous centi-second pattern-matching. In order to improve the accuracy of present phonetic decoders, we are now developing an expert production-rule system that should emulate the competence of an experienced spectrogram reader. After a brief description of the methodology used to elucidate the expert's knowledge and strategies, the paper explains how this expertise has been implemented ; results from the present experimental version of our system (APHODEX) are discussed. In order to increase efficiency while maintaining the same degree of accuracy, we are concurrently designing a system in which the expert's knowledge is represented by frames. An overall description of the system, which describes the phonetic decoding process in terms of a frame language is given in the second part of this paper.

The acoustic-phonetic decoding of a sentence is a major bottleneck in continuous speech recognition. Our group has been working in this area for the past twelve years and has developed several systems relying on synchronous centi-second pattern matching techniques and on rule-based phoneme segmentation and identification. We propose in this paper a new method for knowledge representation and use in phonetic decoding of continuous speech. Our system is based on the notion of frame. This formalism makes it possible to use different types of acoustic analyses and of phonetic features and also to control the recognition process by a planning system. This latter point corresponds to a more sophisticated control than in classical systems. This planning system is described by a frame grammar which is described in details in the paper. This system has been designed in close relation with a phonetic decoding expert system called SYSTEXP described in the 1984 ICASSP Proceedings. SYSTEXP formalizes the expertise of a human phonetician reading speech spectrograms in terms of rules and meta-rules. The results obtained in this manner are used in our frame system in order to obtain a reliable, multi-speaker segmentation of the speech wave into phonemic units, and to apply the phonetic knowledge for phoneme identification. This paper gives an insight description of the principles of the system and of the system itself, including the frame language. It also presents and discusses the recognition results which have been already obtained and which show the interest of the method.

We report on a long term project on the phonetic decoding of continuous speech, within the framework of an expert system. Spectrogram reading is the field of expertise under study, although it is clear that an automatic phonetic decoder should not be in the form of an expert system. We expect major improvements in the recognition rate when expert's rules will be incorporated into existing systems. We describe here both the method used to retrieve the human expertise and results by an experienced spectrogram reader on 50 phonetically-balanced sentences. We also sketch various ways in which the expert rules can be validated through an "inference engine" and actually used in a system. Finally, we discuss the impact this approach has on the design.

Our group has been designing for the past twelve years several speech recognition systems, from isolated vocabulary pattern matching systems to continuous speech understanding systems. The experiments we carried out showed us that the systems designed for restricted vocabularies task were not readily extensible to large vocabularies. We therefore started some years ago implementing a 200 word recognition system using a phonetic approach. This system was tested successfully in 1980. In continuation of this research we decided to extend our approach to a 1000 word vocabulary. This paper describes the principles involved in this system together with the preliminary results already obtained. The basic idea is to reduce the number of word candidates for the recognition by looking for robust phonetic features computed from the input signal. These features are used as a key for accessing the lexicon. Since the determination of the features is carried out in parallel with the phonetic decoding of the input word, it is possible to design a multiprocessor structure in order to reduce the overall recognition time. The determination of crude phonetic features is described together with the organization of the lexicon. Some preliminary results are finally presented and discussed.

Mathematical formulas are good examples of two-dimensional patterns as well as pictures or graphics. The use of syntactic methods is useful for interpreting such complex patterns. In this paper we propose a system for the interpretation of 2-D mathematic formulas based on a syntactic parser. This system is able to recognize a large class of 2-D mathematic formulas written on a graphic tablet. It starts the parsing by localization of the ``principal'' operator in the formula and attempts to partition it into subexpressions which are similarly analyzed by looking for a starting character. The generalized parser used in the system has been developed in our group for continuous speech recognition and picture interpretation.

MYRTILLE II is a connected speech understanding system which is presently designed by our research group in order to recognize a wide subset of spoken French. The syntax accepted by this system is approximately the one of basic French without pronominal references, the vocabulary is presently about 400 words. The application is concerned with a meteorological inquiry system. A first version of MYRTILLE II using a best-first left-to-right strategy is presently operational and has already been tested on a number of sentences. In this paper we present several practical examples illustrating the various choices made in the system and the interactions between processing levels : acoustic-phonetic-lexical decoding, syntactic-semantic analysis, interaction between the recognition and dialog modules, together with several programs which are intended to help in the development and use of this system.

MYRTILLE II is a system which is now being implemented at CRIN in order to understand a wide subset of natural spoken French. In this paper we will concentrate on the syntactic-semantic component of the system. The general principle and overall architecture of MYRTILLE II are first briefly described. The way in which syntax and semantics are integrated in a unified representation is then presented. This representation is equivalent to a network with procedural nodes. Several examples of processed sentences are finally discussed. The system yields a tree structure of an input sentence. The examples presented show how syntax and semantics are used for solving typical problems which occur in speech understanding.

This paper describes the SIRENE system which is being developed in our laboratory. SIRENE is an interactive computer-based system of speech-training aids for the deaf. It also includes a variety of procedures for analysis and classification of pathological voices. The basic idea of speech-training aids consists of compensating for the lack of auditory feedback in deaf children by use of visual displays. The system is intended to be used by speech teachers ; several acoustic and phonetic parameters of speech can be displayed and trained : pitch, voicing, intensity, etc... SIRENE also features the use of automatic speech recognition algorithms in the training of sounds and words.

This paper describes an experiment of recognition of sequences of digits pronounced continuously. This experiment takes place in the design of a practical system for postal codes recognition. The speech signal is analyzed by a 15-channel spectral analyzer. The digit sequence is then recognized using a semi-global procedure which consists of two successive steps: - segmentation of the sequence into the digits which comprise the string, - recognition of the individual digits by a dynamic programming technique which was previously designed in the laboratory. Typical Results of segmentation and recognition will be given and discussed.

In the framework of MYRTILLE II Speech Understanding System under development in our Laboratory we have realized a prototype for the acoustic-phonetic processing level. This prototype makes it possible to test various parameters and strategies for phonemic transcription of continuous speech. It can be considered as a metasystem in the sense that, given a hierarchy of recognition algorithms and a strategy, it can generate the optimal system for phoneme recognition. The system directly works on the digitized speech wave, which makes it possible to get the best accuracy on the parameters. The speech signal is segmented into phoneme-like units by a decision function which incorporates voicing, energy, zero-crossing rate and curve length. The segments thus obtained are then processed by the recognition system which can be viewed as a tree structure the nodes of which are algorithms. These algorithms take into account one or several features and their answers can be phoneme classes and/or other algorithms. Problems involved in the design of such a system are also presented in this paper together with a particular implementation.

This paper describes a connected speech understanding system being implemented in Nancy, thanks to the work done in automatic speech recognition since 1968. This system is made up of four parts : an acoustic recognizer which gives a string of phoneme-like segments from a spoken sentence, a syntactic parser which controls the recognition process, a word recognizer working on words predicted by the parser and a dialog procedure which takes in account semantic constraints in order to avoid some of the errors and ambiguities. Some original features of the system are pointed out : modularily (e.g. the language used is considered as a parameter), possibility of processing slightly syntactically incorrect sentences, ... The application both in data management and in oral control of a telephone center has given very promising results. Work is in progress for generalizing our model : extension of the vocabulary and of the grammar, multi-speaker operation, etc.