Dates are often treated as exact measures, yet their precision is limited. This applies not only to physical clocks but also to human memory. In conversations, we typically share timeframes relevant to the topic, not pinpoint accuracy. Precise birth times, for example, hold little conversational value compared to their biographical significance. Similarly, couples cherish their wedding day but might recall only the year they bought their house. This paper explores the advantages of incorporating fuzzy dates into personal knowledge graphs constructed through dialogues within the LifeLine feature of the “my eViTA” smartphone app. Fuzzy dates acknowledge the inherent imprecision of human memory and prioritize contextually relevant timeframes for a more natural and insightful knowledge representation.

Abstract With life expectancy growing rapidly over the past century, societies are being increasingly faced with a need to find smart living solutions for elderly care and active ageing. The e-VITA project, which is a joint European (H2020) and Japanese (MIC) funded project, is based on an innovative approach to virtual coaching that addresses the crucial domains of active and healthy ageing. In this paper we describe the role of spoken dialogue technology in the project. Requirements for the virtual coach were elicited through a process of participatory design in workshops, focus groups, and living labs, and a number of use cases were identified for development using the open-source RASA framework. Knowledge Graphs are used as a shared representation within the system, enabling an integration of multimodal data, context, and domain knowledge.

The main objective of the EMPATHIC project has been the design and development of a virtual coach to engage the healthy-senior user and to enhance well-being through awareness of personal status. The EMPATHIC approach addresses this objective through multimodal interactions supported by the GROW coaching model. The paper summarizes the main components of the EMPATHIC Virtual Coach (EMPATHIC-VC) and introduces a demonstration of the coaching sessions in selected scenarios.

T e-Coaches are promising intelligent systems that aims at supporting human everyday life, dispatching advices through different interfaces, such as apps, conversational interfaces and augmented reality interfaces. This workshop aims at exploring how e-coaches might benefit from spatially and time-multiplexed interfaces and from different communication modalities (e.g., text, visual, audio, etc.) according to the context of the interaction.

International audience

Speech recordings are a rich source of personal, sensitive data that can be used to support a plethora of diverse applications, from health profiling to biometric recognition. It is therefore essential that speech recordings are adequately protected so that they cannot be misused. Such protection, in the form of privacy-preserving technologies, is required to ensure that: (i) the biometric profiles of a given individual (e.g., across different biometric service operators) are unlinkable; (ii) leaked, encrypted biometric information is irreversible, and that (iii) biometric references are renewable. Whereas many privacy-preserving technologies have been developed for other biometric characteristics, very few solutions have been proposed to protect privacy in the case of speech signals. Despite privacy preservation this is now being mandated by recent European and international data protection regulations. With the aim of fostering progress and collaboration between researchers in the speech, biometrics and applied cryptography communities, this survey article provides an introduction to the field, starting with a legal perspective on privacy preservation in the case of speech data. It then establishes the requirements for effective privacy preservation, reviews generic cryptography-based solutions, followed by specific techniques that are applicable to speaker characterisation (biometric applications) and speech characterisation (non-biometric applications). Glancing at non-biometrics, methods are presented to avoid function creep, preventing the exploitation of biometric information, e.g., to single out an identity in speech-assisted health care via speaker characterisation. In promoting harmonised research, the article also outlines common, empirical evaluation metrics for the assessment of privacy-preserving technologies for speech data.

This paper presents a novel application of convolutional neural networks to phoneme recognition. Thephonetic transcription of the TIMIT speech corpus is used to label spectrogram segments for training theconvolutional neural network. A window of a fixed size slides over the spectrogram of the TIMIT utterancesand the resulting spectrogram patches are assigned to the appropriate phone class by parsing TIMIT’s phonetranscription. The convolutional neural network is the standard GoogLeNet implementation trained withstochastic gradient descent with mini batches. After training, phonetic rescoring is performed in the usual wayto map the TIMIT phone set to the smaller standard set. Benchmark results are presented for comparison toother state-of-the-art approaches. Finally, conclusions and future directions with regard to extending theapproach are discussed.

This paper presents a strategy for enabling speech recognition to be performed in the cloud whilst preserving the privacy of users. The approach advocates a demarcation of responsibilities between the client and server-side components for performing the speech recognition task. On the client-side resides the acoustic model, which symbolically encodes the audio and encrypts the data before uploading to the server. The server-side then employs searchable encryption to enable the phonetic search of the speech content. Some preliminary results for speech encoding and searchable encryption are presented.

International audience

Automatic speech recognition accuracy is affected adversely by the presence of noise. In this paper we present a novel noise removal and speech enhancement technique based on spiking neural network processing of speech data. The spiking network has a recurrent lateral topology that is biologically inspired, specifically by the inhibitory cells of the cochlear nucleus. The network can be configured for different acoustic environments and it will be demonstrated how the connectivity results in enhancement of temporal correlation between similar frequency bands and removal of uncorrelated noise sources. Demonstration of the speech enhancement capability will be provided with data taken from the TIMIT database with different levels of additive Gaussian white noise. Future directions for further development of this novel approach to noise removal and signal processing will also be discussed.

In this paper we propose a generic framework to index and retrieve audio. In this framework, audio data is transformed into a sequence of symbols using the ALISP tools. In such a way the audio data is represented in a compact way. Then an approximate matching algorithm inspired from the BLAST technique is exploited to retrieve the majority of audio items that could be present in radio stream. The evaluations of the proposed systems are done on a private radio broadcast database provided by YACAST and other publicly available corpora. The experimental results show an excellent performance in audio identification (for advertisement and songs), audio motif discovery (for advertisement and songs), speaker di-arization and laughter detection. Moreover, the ALISP-based system has obtained the bestresults in ETAPE 2011 (Evaluations en Treatment Automatique de la Parole) evaluation campaign for the speaker diarization task.

Stephan Schlögl, Pierrick Milhorat, Gérard Chollet, Jérôme Boudy. Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics. 2014.

International audience

In this paper, a speaker diarization system based on data-driven segmentation is proposed. In addition to the usual segmentation and clustering steps, a new module which detects repeated segments between the same shows broadcasted on different dates is added. This process is achieved by using the ALISP-based audio identification system which segments audio data into pseudo-phonetic units. The ALISP segmentation is then used to identify the similar audio segments in TV and radio shows. The system was evaluated during the ETAPE 2011 evaluation campaign and obtained a Diarization Error Rate - DER of 16.23% which was the best result among seven participants.

In this paper, we present a generic methodology to detect nonlinguistic vocalizations using ALISP (Automatic Language Independent Speech Processing), which is a data-driven audio segmentation approach. Using Maximum Likelihood Linear Regression (MLLR) and Maximum A Posterior (MAP) techniques, the proposed method adapts ALISP models, which then facilitate detection of local regions of nonlinguistic vocalizations with the standard Viterbi decoding algorithm. We also illustrate how a simple majority voting scheme, using a sliding window on ALISP sequences, can be helpful in eliminating outliers from the Viterbi-predicted sequence automatically. We evaluate the performance of our method on detection of laughter, a nonlinguistic vocalization, in comparison with global acoustic models such as GMMs, left-to-right HMMs and ergodic HMMs. The results indicate that adapted ALISP acoustic models perform better than global acoustic models in terms of F-measure. Moreover, our majority voting scheme on ALISP-sequences further improves the performance yielding, in total, an increase of 19.6%, 8.1% and 5.6% on the F-measure against global acoustic models GMMs, left-to-right HMMs, and ergodic HMMs respectively.

This paper presents an audio indexing system to search for known advertisements in radio broadcast streams, using automatically acquired segmental units. These units, called Automatic Language Independent Speech Processing (ALISP) units, are acquired using temporal decomposition and vector quantization and modeled by Hidden Markov Models (HMMs). To detect commercials, ALISP transcriptions of reference advertisements are compared to the transcriptions of the test radio stream using the Levenshtein distance. The system is described and evaluated on one day broadcast audio streams from 11 French radio stations containing 2070 advertisements. With a set of 2,172 reference advertisements we achieve a mean precision rate of 99% with the corresponding recall value of 96%. Moreover, this system allowed us to detect some annotation errors.

In this paper, a generic audio identification system is introduced to identify advertisements and songs in radio broadcast streams using automatically acquired segmental units. A new fingerprinting method based on ALISP data-driven segmentation is presented. A modified BLAST algorithm is also proposed for fast and approximate matching of ALISP sequences. To detect commercials and songs, ALISP transcriptions of references composed of large library of commercials and songs, are compared to the transcriptions of the test radio stream using Levenshtein distance. The system is described and evaluated on broadcast audio streams from 12 French radio stations. For advertisement identification, a mean precision rate of 100% with the corresponding recall value of 98% were achieved. For music identification, a mean precision rate of 100% with the corresponding recall value of 95% were achieved.

This paper presents recent developments on our “silent speech interface ” that converts tongue and lip motions, captured by ultrasound and video imaging, into audible speech. In our previous studies, the mapping between the observed articulatory movements and the resulting speech sound was achieved using a unit selection approach. We investigate here the use of statistical mapping techniques, based on the joint modeling of visual and spectral features, using respectively Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM). The prediction of the voiced/unvoiced parameter from visual articulatory data is also investigated using an artificial neural network (ANN). A continuous speech database consisting of one-hour of high-speed ultrasound and video sequences was specifically recorded to evaluate the proposed mapping techniques. Index Terms: silent speech interface, GMM, HMM, ultrasound, video, multimodal, statistical mapping

Audiovisual identity verification exploits both image and audio information to improve the performance of the identification system. Unfortunately, both image and audio systems are sensitive to signal quality. In this paper, we propose a method to combine output classifiers based on both image and audio quality measures. We define classes of signal degradation within which we estimate the fusion weights and normalization parameters. Results of experiments on the BANCA database show that fusion using quality measures improves verification performance by 25% compared to the baseline fusion method.

Recent improvements are presented for phonetic decoding of continuous-speech from ultrasound and optical observations of the tongue and lips in a silent speech interface application. In a new approach to this critical step, the visual streams are modeled by context-dependent multi-stream Hidden Markov Models (CD-MSHMM). Results are compared to a baseline system using context-independent modeling and a visual feature fusion strategy, with both systems evaluated on a one-hour, phonetically balanced English speech database. Tongue and lip images are coded using PCA-based feature extraction techniques. The uttered speech signal, also recorded, is used to initialize the training of the visual HMMs. Visual phonetic decoding performance is evaluated successively with and without the help of linguistic constraints introduced via a 2.5k-word decoding dictionary. Index Terms: silent speech interface, visual speech recognition, multi-stream modeling 1.

Event detection is essential for sports video summarization, indexing and retrieval [1]. In this paper, based on three generally defined shot types, goal events are detected by combining heuristic rules with unsupervised fuzzy c-means algorithm. First heuristic rule based primary selection/filtering for potential goals is carried out in the shot layer which composed of three generally defined shot types, together with the number of frames within each shot is recorded as representative feature. Then to further classify goal events from other potential goals unsupervised fuzzy c-means (FCM) algorithm is adopted. The main contribution of this work is the combination of heuristic rule which is based on three generally defined shot types with unsupervised fuzzy c-means algorithm. And when defuzzified with prior knowledge of the number of goals within each match, accurate and robust results can be achieved over five half matches from different series produced by different broadcast stations.

We expose the limitations of existing frameworks designed for the evaluation of audiovisual biometric authentication algorithms. The weakness of a classical audiovisual authentication system is uncovered when confronted to realistic deliberate impostors. A client-dependent audiovisual synchrony measure is used in order to deal with deliberate impostors and three new fusion strategies and their performance against random and deliberate impostors are studied.

The BioSecure Network of Excellence has collected a large multi- biometric publicly available database and organized the BioSecure Multimodal Evaluation Campaigns (BMEC) in 20072. This paper reports on the Talking Faces campaign. Open source reference systems were made available to participants and four laboratories submitted executable code to the organizer who performed tests on sequestered data. Several deliberate impostures were tested. It is demonstrated that forgeries are a real threat for such systems. A technological race is ongoing between deliberate impostors and system developers.

This article presents a framework for a phonetic vocoder driven by ultrasound and optical images of the tongue and lips for a “silent speech interface” application. The system is built around an HMM-based visual phone recognition step which provides target phonetic sequences from a continuous visual observation stream. The phonetic target constrains the search for the optimal sequence of diphones that maximizes similarity to the input test data in visual space subject to a unit concatenation cost in the acoustic domain. The final speech waveform is generated using “Harmonic plus Noise Model” synthesis techniques. Experimental results are based on a onehour continuous speech audiovisual database comprising ultrasound images of the tongue and both frontal and lateral view of the speaker’s lips.

Latest results on continuous speech phone recognition from video observations of the tongue and lips are described in the context of an ultrasound-based silent speech interface. The study is based on a new 61-minute audiovisual database containing ultrasound sequences of the tongue as well as both frontal and lateral view of the speaker’s lips. Phonetically balanced and exhibiting good diphone coverage, this database is designed both for recognition and corpus-based synthesis purposes. Acoustic waveforms are phonetically labeled, and visual sequences coded using PCA-based robust feature extraction techniques. Visual and acoustic observations of each phonetic class are modeled by continuous HMMs, allowing the performance of the visual phone recognizer to be compared to a traditional acoustic-based phone recognition experiment. The phone recognition confusion matrix is also discussed in detail.

We investigate the use of audio-visual speech synchrony measure in the framework of identity verification based on talking faces. Two synchrony measures based on canonical correlation analysis and co-inertia analysis respectively are introduced and their performances are evaluated on the specific task of detecting synchronized and not-synchronized audio-visual speech sequences. The notion of high-effort impostor attacks is also introduced as a dangerous threat for current biometric system based on speaker verification and face recognition. A novel biometric modality based on synchrony measures is introduced in order to improve the overall performance of identity verification, and more specifically its robustness to replay attacks.

The article compares two approaches to the description of ultrasound vocal tract images for application in a "silent speech interface," one based on tongue contour modeling, and a second, global coding approach in which images are projected onto a feature space of Eigentongues. A curvature-based lip profile feature extraction method is also presented. Extracted visual features are input to a neural network which learns the relation between the vocal tract configuration and line spectrum frequencies (LSF) contained in a one-hour speech corpus. An examination of the quality of LSFs derived from the two approaches demonstrates that the Eigemongues approach has a more efficient implementation and provides superior results based on a normalized mean squared error criterion.

We present a new method that permits arbitrary region description in video sequence using a reduced number of bits allowing promising results in video compression. A robust unsupervised 3D (2D + t) segmentation is used to detect arbitrary regions for motion compensated video coding. The video sequence is first modeled using a Gaussian mixture model where each pixel, defined by its spatiotemporal position and its color vector is supposed to be generated by one of the mixture components. This permits to segment the video sequence into objects each one modeled by one Gaussian distribution. Grouping the mixtures in a binary tree defines a hierarchical representation of the video objects and a gradual segmentation. This segmentation is then used for region description in a motion compensated video coder. This provides a large improvement in motion bits budget. When compared to a H.264 video coder, promising results were obtained.

The article describes a video-only speech recognition system for a “silent speech interface” application, using ultrasound and optical images of the voice organ. A one-hour audiovisual speech corpus was phonetically labeled using an automatic speech alignment procedure and robust visual feature extraction techniques. HMM-based stochastic models were estimated separately on the visual and acoustic corpus. The performance of the visual speech recognition system is compared to a traditional acoustic-based recognizer.

We describe an algorithm that detects a lack of correspondence between speech and lip motion by detecting and monitoring the degree of synchrony between live audio and visual signals. It is simple, effective, and computationally inexpensive; providing a useful degree of robustness against basic replay attacks and against speech or image forgeries. The method is based on a cross-correlation analysis between two streams of features, one from the audio signal and the other from the image sequence. We argue that such an algorithm forms an effective first barrier against several kinds of replay attack that would defeat existing verification systems based on standard multimodal fusion techniques. In order to provide an evaluation mechanism for the new technique we have augmented the protocols that accompany the BANCA multimedia corpus by defining new scenarios. We obtain 0% equal-error rate (EER) on the simplest scenario and 35% on a more challenging one

The article deals with a technique of voice forgery using the ALISP (automatic language independent speech processing) approach. Such a technique allows the voice of an arbitrary person (the impostor) to be transformed, forging the identity of another person (the client). Our goal is to demonstrate that an automatic speaker recognition system could be seriously threatened by a transformation of this kind. For this purpose, we use a speaker verification system to calculate the likelihood that the forged voice belongs to the genuine client. Experiments on NIST 2004 evaluation data show that the equal error rate for the verification task is significantly increased by our voice transformation.

State of the art Speech Recognition systems use statistical language modeling and in particular N-gram models to represent the language structure. The Arabic language has a rich morphology, which motivates the introduction of morphological constraints in the language model. Class-based N-gram models have shown satisfactory results, especially for language model adaptation and training from reduced datasets. They were also proven quite effective in their use of memory space. In this paper, we investigate a new morphological classbased language model. Morphological rules are used to derive the different words in a class from their stem. As morphological analyzer, a rule-based stemming method is proposed for the Arabic language. The language model has been evaluated on a database composed of articles from Lebanese newspaper Al-Nahar for the years 1998 and 1999. In addition, a linear interpolation between the N-gram model and the morphological model is also evaluated. Preliminary experiments detailed in this paper show satisfactory results.

Multilingual speech recognition pushes to us study the acoustic modeling of target language units using one or more source languages' units. This paper presents a study of manual and data driven association of two possible target units with source language's phonemes. The target units studied are words and phonemes. Algorithms for data-driven association are described. While phoneme-to-phoneme association is more practical, words' transcription provides better results. It has been shown that more precise and rich source models are more suitable to determine those association. Experiments are conducted with French as source language and Arabic as target language.

Support vector machines (SVM) is a new and very promising classification technique developed from the theory of structural risk minimisation. We propose an alternative out-of-vocabulary word detection method relying on confidence measures and support vector machines. Confidence measures are computed from phone level information provided by a hidden Markov model (HMM) based speech recognizer. We use three kinds of average techniques as arithmetic, geometric and harmonic averages to compute a confidence measure for each word. The acceptance/rejection decision of a word is based on the confidence feature vector which is processed by a SVM classifier. The performance of the proposed SVM classifier is compared with methods based on the averaging of confidence measures.

Current best performing speaker recognition algorithms are based on Gaussian Mixture Models (GMM). Their results are not satisfactory for all experimental conditions, especially for the mismatched (train/test) conditions. Support Vector Machine is a new and very promissing technique in statistical learning theory. Recently, this technique produced very interesting results in image processing [2], [3], [4] and for the fusion of experts in biometric authentification [5]. In this paper we address the issue of using the Support Vector Learning technique in combination with the currently well performing GMM models, in order to improve speaker verification results.

An extension of the multi-band model including inter-band control of time asynchrony is described. It is based on the framework of Markov random fields. The law of the speech process is given by a parametric Gibbs distribution and a maximum likelihood parameter estimation algorithm is developed. This random field model is applied to isolated word recognition. It is shown that similar performances are obtained with the new model and with standard HMM techniques in the mono-band case. In the multi-band case, it is shown that the recognition rate decreases when the number of bands is increased but that modeling inter-band synchrony limits the performance decrease.

Speech can be represented as a time/frequency distribution of energy using a multiband filter bank. A Markov random field model, which takes into account the possible time asynchrony across the bands, is estimated for each segmental units to be recognized. The law of the speech process is given by a parametric Gibbs distribution and a maximum likelihood parameter estimation algorithm is developed. Experiments are conducted on an isolated word recognition problem. It is shown that similar performances are obtained with the new model and with standard HMM techniques in the mono-band case. In the multiband case, it is shown that modeling interband synchrony is an interesting approach to increase the performance when the number of bands increases.

The problem of labelling speaker turns by automatically segmenting and clustering a continuous audio stream is addressed. A new clustering scheme is presented and evaluated using a clustering efficiency score which treats both agglomerative and divisive clustering strategies equally. Results show an efficiency of 70% can be obtained on both manually and automatically derived segments on the 1996 Hub4 development data. For the task of identifying potentially unknown anchor speakers within broadcast news shows, the frame classification error rate is very important. To reflect this, a frame-based cluster efficiency is defined and the results show a 90% frame-based efficiency can be achieved. Finally a frame-based comparison between the manually and automatically derived segment/cluster sets shows that approximately one third of the errors are introduced during segmentation and two-thirds during clustering.

In this paper a new automatic speech recognition (ASR) CPU-based software, called AlfaNum, with the chosen few heuristics optimized for applications in heterogeneous conditions is described. AlfaNum is a discrete speaker-independent ASR product intended for application in the largest bank-by-phone interactive voice response (IVR) system in Yugoslavia, with a lot of customers all over Serbia. That means a large variety of dialects, telephone line quality, and microphones used. This system has been tested on 500 speakers and it achieved an average accuracy of 98,2% in real life conditions. The whole software is developed in C++ programming language. Object oriented programming gave the software an elegant look, and minimized all possible errors. On the other hand, the power of C++ language and its tight interaction with machine made the software fast and efficient.

We describe a speaker tracking and detection system, for Switchboard conversations, that uses a two-speaker and silence hidden Markov model (HMM) with a minimum state duration constraint and Gaussian mixture model (GMM) state distributions adapted from a single gender- and handset-independent imposter model distribution. Speaker tracking is used to segment speakers for detection, which is carried out by averaging frame scores of the Viterbi path and HNORM’ing via a novel parameter interpolation extension of HNORM for use with files of arbitrary lengths. Use of duration statistics augmenting the acoustic scores is also introduced via a nonlinear combination function. Results are reported on the NIST 1998 Multispeaker development evaluation dataset.

The problem of very low bit rate segmental speech coding is addressed. The basic units are found automatically in the training database using temporal decomposition, vector quantization and multigrams. They are modelled by HMMs. The coding is based on recognition and synthesis. In single speaker tests, we obtained intelligible and naturally sounding speech at a mean rate of 211.2 b/s. In the end, future extensions of our scheme (diphone-like synthesis and speaker adaptation) as well as possible use of automatically derived units in recognition are discussed.

LIDIAP

In this paper, we present a new technique for statistical modeling of speech segments based on Markov random fields. Classical and multi-stream HMMs are particular cases of this more general family of models. However, the Random Field Model (RFM) proposed here can be seen as an extension of the multiband HMM in which interactions between the frequency bands have been added. In a first experiment, samples are drawn from different models and compared to real observations. This experiment shows that the RFM is able to produce realistic samples but a single HMM still performs better. Isolated word recognition experiments stress the fact that more work must be done on the RFM in order to reach the performances of classical hidden Markov modeling techniques. For the moment, the RFM parameters are estimated using a heuristic. We believe that a real maximum likelihood parameter estimation algorithm should improve the results. The main advantage of this new model is that it can easily be extended since a model is defined by some local interactions and the Gibbs potential functions associated to those interactions. 1

Keywords: Non-Linear Signal Processing ; Speech Processing Reference LANOS-CONF-1998-024 Record created on 2004-12-03, modified on 2017-05-12

The contribution of this paper is twofold: (1) to formulate a fusion problem encountered in the design of a multi-modal identity verification system as a particular classification problem, (2) to propose a simple classifier to solve this problem. The multi-modal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called the score, stating how well the claimed identity is verified. A fusion module receiving as input the d scores has to take a binary decision: accept or reject identity. We have solved this fusion problem using a classic k-nearest-neighbor (k-NN) classifier. The most important problem encountered with this simple classifier is the unbalance between the number of reference points in either class. Adapting the classic k-NN classifier using distance weighting and vector quantization principles enables to reduce the influence and the number of impostor reference points respectively. This constitutes the originality of this paper. The performances of these different fusion modules have been evaluated on a multi-modal database, containing both vocal and visual modalities.

The multigrams allow us to split a string of symbols into a stream of variable length sequences. The direct application of this method to vector-quantized speech spectra fails, we develop an extension of the method called modified multigrams or multigrams with distance. The algorithm for modified multigram dictionary training as well as experimental results are presented. We found a significant improvement of rate/distortion ratio in comparison to vector quantization with small codebooks. For precise spectrum representation, this method is less suitable and we see its application rather in speech segmentation or in very low bit rate coding.

In this paper, we attempt to validate the exible vocabulary approach for speaker independent isolated word and connected words recognition.We compare the performance of classical whole word HMMs against dierent sets of subword units.For this purpose, we model phonemes, diphones and words of the (Swiss) French language.The recognition rates obtained with phoneme models are monitored as we increase the amount of training data.The results of the described experiments validate the exible vocabulary approach and show advantages and disadvantages of both propposed subword units against common whole word HMMs.

The aim of this paper is to describe how the combination of speaker verification algorithms with a priori decision thresholds can improve the overall robustness of a real application. The evaluation is performed in the context of a field application where each client is verified from a 7 digit pin code. This paper demonstrate that it is possible to increase the global performances of the system on combining the result of several algorithms.

The goal is to improve recognition rate by optimisation of Mel Frequency Cepstral Coecients (MFCCs): modications concern the time-frequency representations used to estimate these coecients.There are many w a ys to obtain a spectrum out of a signal which dier in the method itself (Fourier, Wavelets,...), and in the normalisation.We show here that we can obtain noise resistant cepstral coecients, for speaker independent connected word recognition.The recognition system is based on a continuous whole word hidden Markov model.An error reduction rate of approximately 50% is achieved with word models.

The non-supervised self organizing map of Kohonen (SOM), the supervised learning vector quantization algorithm (LVQ3), and a method based on second-order statistical measures (SOSM) were adapted, evaluated and compared for speaker verification on 57 speakers of a POLYPHONE-like data base. The SOM and LVQ3 were trained by codebooks with 32 and 256 codes and two statistical measures; one without weighting (SOSM1) and another with weighting (SOSM2) were implemented. As the decision criterion, the equal error rate (EER) and best match decision rule (BMDR) were employed and evaluated. The weighted linear predictive cepstrum coefficients (LPCC) and the /spl Delta/LPCC were used jointly as two kinds of spectral speech representations in a single vector as distinctive features. The LVQ3 demonstrates a performance advantage over SOM. This is due to the fact that the LVQ3 allows the long-term fine-tuning of an interested target codebook using speech data from a client and other speakers, whereas the SOM only uses data from the client. The SOSM performs better than the SOM and the LVQ3 for long test utterances, while for short test utterances the LVQ is the best method among the methods studied.

Keywords: speech Reference EPFL-CONF-82322 Record created on 2006-03-10, modified on 2017-05-10

The paper compares, on a database recorded in a car, a number of signal analysis and speech enhancement techniques as well as some approaches to adapt speech recognition systems. It is shown that a new nonlinear spectral subtraction associated with Mel frequency cepstral coefficients (MFCC) is an adequate compromise for low-cost integration. The Lombard effect is analyzed and simulated. Such a simulation is used to derive realistic training utterances from noise-free utterances. Adapting a continuous-density hidden Markov model (CDHMM) to these artificially generated training samples yields a very high performance with respect to that achieved within the ESPRIT adverse environment recognition of speech (ARS) project, i.e., an average of 1% error for all driving conditions. Finally, the paper shows, both theoretically and experimentally, that whatever the noise estimation technique is, it is better to add this noise estimate to the reference clean models than to subtract it from the noisy data.>

Theoretical aspects and practical applications are reported of two variants of the AR-vector modeling technique: the standard AR-vector model and the target driven (or multistep excited) AR-vector model. The standard version supposes a white excitation, while the target driven model assumes a piecewise constant input. The standard AR-vector model turns out to be extremely efficient for speaker recognition, since, for a set of 420 different speakers, the recognition score ranges from 93% to 100%, depending on the duration of the test speech sample. The target driven AR-vector model shows very interesting properties for speech analysis and segmentation. There exists a strong correspondence between the steps in the input function and the underlying phonetic content of speech. Moreover, under some normalization, the values of the steps can be interpreted as acoustic targets.>

An attempt was made to enhance the performance of a DTW (dynamic time warping) speech recognizer by preprocessing speech parameters using a neural network transformation. A multilayer perceptron trained with speech utterances of a single speaker has been used in front of a DTW recognizer. Results show an improvement of about 15% in the recognition rate in all cases, even with a speaker that was not used for training. If the network is not completely speaker independent, a dynamic adaptation to the speaker could be performed.>

The authors report results obtained in the framework of phonetic feature extraction using TDNNs (time-delay neural networks). Experiments were conducted on a set of 5270 phonemes manually segmented from natural continuous speech uttered by one male speaker. Nearly all scores on binary phonetic features range between 90% and 99%. Comments on the visualization of the weights in several networks are also presented. In particular, it is noted that there exist quantitative similarities between patterns of weights within the TDNN and acoustical characteristics of the phonetic feature.>

Speech enhancement through Kalman filtering is compared to environment adaptation using spectral transformations. Important results on general characteristics of speech in noise and a possible correction to the Lombard effect are reported. It is shown that traditional speech enhancement offers little improvement to recognition. Best results are obtained with transformations applied to the reference models.>

A report is presented of intraspeaker and interspeaker variability as a major source of error in automatic speech recognition. The authors report on two series of experiments using multilayer feed-forward automata (MLFFA) to control some aspects of this variability. The first series concerns the classification of spectral targets obtained from a robust implementation of temporal decomposition. An MLFFA accepts three successive targets to output an allophonic label. No improvement has been found so far from traditional classification techniques (i.e. k-nearest neighbors). In a second series of experiments spectral transformations using MLFFA are introduced for the adaptation to new speakers. Compared to linear techniques (multivariate regression and canonical correlation analysis), the MLFFA approach offers some improvement.>

The automatic recognition of continuous speech may use a symbolic representation of the acoustic signal in order to facilitate lexical access. The allophones of the language form a practical set of symbols. A major issue is a reliable localisation of these units in the speech stream and their identification. Localisation is obtained using a robust implementation of temporal decomposition, a technique originally proposed by Atal (1983), for speech coding. Speech is decomposed in terms of overlapping events characterized by both a spectral target and a time-limited interpolation function. An undershot target may be reestimated using neighbours and the associated functions. The possibility of undoing the effects of coarticulation is the major contribution of this work. The identification of these corrected targets is therefore possible with no further contextual rules. The recognition of spelled surnames (letters of the alphabet) is used for evaluation. 76% of correct phones allow 70% of correct letters.>

Localization is realized using a robust implementation of temporal decomposition, a technique originally proposed for speech coding. Speech is decomposed in terms of overlapping events characterized by spectral targets and time-limited interpolation functions. The spectral targets are evaluated iteratively. An undershot target may be reestimated using neighbors and the associated functions. The possibility of undoing the effects of coarticulation is also suggested. The identification of these corrected targets is therefore possible with no further contextual rules. Applications to speech recognition strategies are discussed. The recognition of spelled surnames (letters of the alphabet) is used for evaluation. The correct recognition of 76% of correct phones is the basis of 70% correct recognition letters.>

ATAL [1] introduced a technique for decomposing speech into phone-length temporal events in terms of overlapping and interacting articulatory gestures. This paper reports on simplifications of this technique with applications to acoustic-phonetic synthesis. Spectral evolution is represented by time-indexed trajectories in the p-dimensional space of Log-Area Ratios{y_{i}= \Ln ((1+k_{i})/(1-k_{i}))}where kiare the reflection coefficients obtained from short-time stationary LPC analysis. The vocal tract configuration (spectral vector) associated with each interpolation function belongs to a finite set of articulatory targets (vector quantization code book). A set of speech segments ("polysons") has been encoded using this technique. It includes diphones, demi-syllables, and other units that are difficult to segment. Temporal decomposition using target spectra can break the complex encoding of these segments. In particular, coarticulation effects are analyticaiy explained and modeled. It is demonstrated that these new tools provide an adequate environment in our search for better rules in acoustic speech synthesis.

This paper describes a technique of spectral transformation for improved adaptation of a knowledge data base or reference templates to new speakers in automatic speech recognition (ASR). Based on a statistical analysis tool (Canonical correlation analysis) the proposed method permits to improve speaker independance in Large vocabulary ASR. Application to an isolated word recognizer improved a 70% correct score to 87%.

Dynamic Time Warping is implemented using an array of identical processing elements. Each processing element is designed to compute a local distance and update a global measure of dissimilarity. It is made up of 1900 transistors using a 2.5 micron NMOS technology. 25 processing elements and their local interconnections fit within 35mm2 of silicon that can be packaged in a standard 40 pin packaging. A single chip can handle 300 words in real time. An array of 22 chips will recognize within 200msec a syllable size pattern from a set of 6000. Various applications are taken up.

A methodology is described to obtain a set of segments and rules that represents adequately the speech performance of a given speaker. This methodology proceeds from an initial set of diphones extracted from a neutral context and modify this set with larger and/or smaller segments depending on the match with natural utterances. Each segment is stored as a sequence of frames coded using LPC coefficients. An estimate of the likelihood of timescale distortion is associated with each frame. It represents knowledge on temporal variability that can be used by synthesis rules and/or pattern matching algorithms. It is then shown how such a segment data base can be used for 1) speech coding at very low bit rate ( ∼ 400 bit/sec), 2) synthesis from unrestricted text, 3) continuous speech recognition.

The most straightforward way to compare the performance of speech recognizers is to test them with an identical data base. An agreement on such data base to be used as a standard could be difficult to reach. An alternative is to agree on a set of algorithms (a reference system) and compare each system to this reference. Furthermore, the reference is used to quantify the difficulty of the test sets. Differences in performance between the system under test and the reference will be meaningful to the speech community if the reference system is made widely available. The complete specifications (in FORTRAN) of a set of speech analysis and pattern discrimination algorithms are proposed here for this purpose. This recognizer uses dynamic time warping to optimize the match between unknown and reference utterances. Every utterance is coded as a sequence of vectors of cepstral coefficients. These coefficients are obtained from a short time power spectrum expressed on a mel frequency scale.

At some stage of the recognition process, a choice has to be made between lexical items. This choice is the most difficult if the items form a minimal series (they differ only by one phoneme). A selection of such minimal series has been used to test a number of commercially available word recognition systems (CNET 'Dynamo', INTERSTATE 'VRM', LIMSI-VECSYS 'Primo', THRESHOLD 'T 600') and several speech recognizers developped in France. Recognition scores beeing highly dependant on the quality of the acoustic samples used for training and testing, performance of a system is expressed as the noise level necessary to obtain identical human recognition scores on the same test material. Confusion matrices can be used by the experimenter to correct defficiencies of the algorithms and techniques used, or by the user to select an application vocabulary.

The rules for vocalic sound transitions usually consider a limited frequency range for each formant. Current acoustic knowledge of speech production allows us to give a different interpretation of vocalic sound transitions. This interpretation would explain the existence of the so-called "spurious" formants, the observed "asynchrony" between transitions, and the effects of coarticulation across consonants. We show as an example that the second formant of /w/ is transformed through the transition into the fourth formant of /i/ in the sequence /wi/; a result implied in K.N. STEVENS theoretical model (1). Implications for automatic speech pattern discrimination and synthesis are discussed.