The rapid increase in web services and mobile apps, which collect personal data from users, has also increased the risk that their privacy may be severely compromised. In particular, the increasing variety of spoken language interfaces and voice assistants empowered by the vertiginous breakthroughs in deep learning have prompted important concerns in the European Union in terms of preserving the privacy of speech data. For instance, an attacker can record speech from users and impersonate them to obtain access to systems that require voice identification. By extracting speaker, linguistic (e.g., dialect), and paralinguistic features (e.g., age) from a speech signal, the speaker profiles can also be hacked from users through existing technology. To mitigate these weaknesses, in this study, we present a speech anonymization method based on autoencoders and adversarial training. Given an utterance, we first extract an x-vector, which is a powerful utterance-level embedding used in state-of-the-art speaker recognition. This original x-vector is transformed by an autoencoder producing a new x-vector, where speaker, gender, and accent information are suppressed through adversarial training. The anonymized speech is finally generated through a neural speech synthesizer driven by the anonymized x-vector, fundamental frequency, and phoneme information extracted from the input speech. For the evaluation, we followed the VoicePrivacy Challenge framework, where anonymization or privacy is measured using automatic speaker verification and the preservation of the intelligibility is evaluated through automatic speech recognition. Our experimental results show that the proposed method achieves higher privacy than the VoicePrivacy baseline (i.e., a higher speaker verification error) while preserving a similar intelligibility for the spoken content (i.e., a similar word error rate).

This paper addresses driver identification and verification using Deep Learning (DL) on tri-axial accelerometer signals from drivers’ smartphones. The proposed driver identification architecture includes ResNet-50 followed by two Stacked Gated Recurrent Units (SGRUs). ResNet provides a deep layer model, thanks to shortcut connections, is able to extract rich features from accelerometers, and GRU layers model the dynamics of drivers’ behavior. ResNet-50 pre-trained on image classification has been evaluated testing two approaches to map 1D accelerometer signals into 2D images. Siamese Neural Networks and Triplet Loss Training have been proposed for driver verification. The Siamese architecture is built on the same ResNet-50 + GRU model of driver identification, while the Triplet loss has required obtaining embeddings at journey level. Experimental results have been obtained for a dataset of 25 drivers, performing 20,025 daily life journeys with more than 800 per driver. Driver identification achieved top-1 and top-5 accuracies of 71.89% and 92.02%, respectively, and driver verification a F1 score of 74.09%. These results are competitive with state-of-the-art research that have generally tested smaller databases (in many cases based only on predefined routes), and have relied on information sources other than accelerometers, such as gyroscopes, magnetometers and GPS. Therefore, we believe that the proposed DL architectures are suitable for developing efficient driver monitoring applications based on only energy-efficient smartphone accelerometer signals.

We have developed an educational software tool (Aprendiendo) to reinforce the learning process of children with special educational needs. This tool makes use of a variety of interactive technologies such as Embodied Conversational Agent and multimedia elements (video images from a WebCam, pictures and photos). In this paper we propose a multidisciplinary approach to system design and the evaluation of the users' experience, involving the engineers, teachers therapists and the children themselves, and combining Wizard-of-Oz with other techniques commonly employed in this educational context. We illustrate our approach with a real user evaluation of Aprendiendo's emotions module.

Proceedings of Interspeech 2008, Brisbane (Australia)

As a result of the evolution in the field of biometrics, a new breed of techniques and methods for user identity recognition and verification has appeared based on the recognition and verification of several biometric features considered unique to each individual. Signature and voice characteristics, facial features, and iris and fingerprint patterns have all been used to identify a person or just to verify that the person is who he/she claims to be. Although still relatively new, these new technologies have already reached a level of development that allows its commercialization. However, there is a lack of studies devoted to the evaluation of these technologies from a user-centered perspective. This paper is intended to promote user-centered design and evaluation of biometric technologies. Towards this end, we have developed a platform to perform empirical evaluations of commercial biometric identity verification systems, including fingerprint, voice and signature verification. In this article, we present an initial empirical study in which we evaluate, compare and try to get insights into the factors that are crucial for the usability of these systems.

This paper presents an algorithm for formant tracking using HMMs and analyzes the influence of HMM initialization, training and context-dependency on the accuracy of the formant tracks obtained with the HMMs. Formant trackers usually include two different phases: one in which the speech is analyzed and formant candidates are obtained, and another in which, by imposing different constraints, the most likely formants are chosen. While the first stage usually relies on standard spectrum estimation techniques, the second stage has evolved notably in the recent years. Traditionally the second phase tries to impose continuity constraints on the formant selection process. Lately there has been ongoing research to include phonemic knowledge in the second stage to make formant tracking more reliable. In order to incorporate phonemic knowledge newer approaches make use of the orthographic transcription of the speech utterance. From the orthographic transcription, the phonemic transcription is obtained, and from this and the speech itself a phonemic segmentation can be obtained. This phonemic segmentation, along with the phonemic transcription and some knowledge of the nominal formant positions for the different phonemes provides extra information that can be used to obtain more accurate formant tracks. This paper presents a complete HMM-based data-driven algorithm for formant tracking suitable to combine different levels of acoustic and phonemic information. A detailed analysis on the performance of this algorithm is discussed for: different initialization strategies using different levels of knowledge, different degrees of training, and context-independent and dependent HMMs. Experimental speaker-dependent results show that the efficient use of phonemic information in HMM training and context-dependent modeling significantly reduces the formant tracking error rate especially for formants$ F_2$and$ F_3$.

Proceedings of Interspeech-Eurospeech 2005, Lisbon (Portugal)

This paper presents the results and conclusions of a thorough study on automatic phonetic segmentation. It starts with a review of the state of the art in this field. Then, it analyzes the most frequently used approach-based on a modified Hidden Markov Model (HMM) phonetic recognizer. For this approach, a statistical correction procedure is proposed to compensate for the systematic errors produced by context-dependent HMMs, and the use of speaker adaptation techniques is considered to increase segmentation precision. Finally, this paper explores the possibility of locally refining the boundaries obtained with the former techniques. A general framework is proposed for the local refinement of boundaries, and the performance of several pattern classification approaches (fuzzy logic, neural networks and Gaussian mixture models) is compared within this framework. The resulting phonetic segmentation scheme was able to increase the performance of a baseline HMM segmentation tool from 27.12%, 79.27%, and 97.75% of automatic boundary marks with errors smaller than 5, 20, and 50 ms, respectively, to 65.86%, 96.01%, and 99.31% in speaker-dependent mode, which is a reasonably good approximation to manual segmentation.

This paper describes an approach to estimating the parameters of continuous density HMMs for visual speech recognition. One of the key issues of image-based visual speech recognition is normalization of lip location and lighting condition prior to estimating the parameters of HMMs. We present an average-intensity and location normalized training method, in which the normalization process is integrated in the model training. The proposed method provides a theoretically-well-defined algorithm based on a maximum likelihood formulation, hence the likelihood for the training data is guaranteed to increase at each iteration of the normalized training. Experimental results show that the recognition performance can be significantly improved by the normalized training.

This study compares the VOT measurements between Persian and English stutterers. The speech samples of five adult stutterers (four males and one female) of each language were recorded during a reading task. 98 stuttering events in words beginning with plosives and 34 fluent plosives (i.e., final fluent iterations following the stuttered portions of speech) were identified in the data of both groups. The VOT measurements were obtained from both stuttered as well as fluent productions of speech for voiceless plosives /p, t, k/ and voiced plosives /b, d, J/ respectively.

In this work automatic methods for determining the number of gaussians per state in a set of Hidden Markov Models are studied. Four different mix-up criteria are proposed to decide how to increase the size of the states. These criteria, derived from Maximum Likelihood scores, are focused to increase the discrimination between states obtaining different number of gaussians per state. We compare these proposed methods with the common approach where the number of density functions used in every state is equal and pre-fixed by the designer. Experimental results demonstrate that performance can be maintained while reducing the total number of density functions by 17% (from 2046 down to 1705). These results are obtained in a flexible large vocabulary isolated word recognizer using context dependent models.

There are many Spontaneous Dialogue Recognition based applications like home-banking ones where long numbers recognition facilities are crucial to complete a request from the user. Rejection and Utterance Verification (UV) are difficult problems in these applications. In this contribution we improve our previously proposed UV procedure [3] in order to increase the correction of recognition errors, to solve grammatical ambiguities from the user, and to make more efficient the rejection of misrecognized or out-of-vocabulary (OOV) utterances. In spite of the verification performance, the proposed algorithm complies with the real-time constrains which are mandatory in real applications.

Utterance verification (UV) is a critical function of an automatic speech recognition (ASR) system working on real applications where spontaneous speech, out-of-vocabulary (OOV) words and acoustic noise are present. We present a new UV procedure with two major features. (a) Confidence tests are applied to decoded string hypotheses obtained from using word and garbage models that represent OOV words and noise. Thus the ASR system is designed to deal with what we refer to as word spotting and noise spotting capabilities. (b) The UV procedure is based on three different confidence tests, two based on acoustic measures and one founded on linguistic information, applied in a hierarchical structure. Experimental results from a real telephone application on a natural number recognition task show a 50% reduction in recognition errors with a moderate 12% rejection rate of correct utterances and a low 1.5% rate of false acceptance.

One of the most important demands for future text-to-speech (TTS) systems is their ability to improve naturalness when embedded in a particular task or application that requires a particular speaking style for a particular speaker. We present a new prosodic modeling procedure for improving naturalness by adapting a TTS system to a new speaker and a new speaking style. The proposed procedure is an extension of our automatic data-driven methodology, to model both fundamental frequency and segmental duration. Automatic linguistic and acoustic analysis are performed on both a task dependent text corpus and the recorded material from the selected speaker.

In accordance with the new emerging voice response systems that use flexible vocabulary recognizers (FVRs), the prediction of word confusabilities have received increasing interest during the last few years. We present a new method for transcription confusabilities estimation based on a new statistical modelling criterion. We propose the use of the new transcription confusability measure in two different word error rate (WER) reduction procedures for FVRs: an automatic vocabulary selection procedure suitable for those applications where the set of vocabulary words is not totally defined by the application, and an automatic procedure for generation of alternative transcriptions. Experimental results using a telephonic database show a 20% WER relative reduction using the automatic alternative transcription generation procedure for a 37 word vocabulary, and over a 50% (20%) WER relative reduction using our unrestricted (restricted by groups of synonyms) vocabulary selection procedure instead of random word selection.

We describe a novel discriminative training procedure for a Gaussian mixture model (GMM) speaker identification system. The proposal is based on the segmental generalized probabilistic descent (GPD) algorithm formulated to estimate the GMM parameters. Two major innovations over similar formulations of segmental GPD training are proposed. (1) A misclassification measure based on an individual representation of competing speakers, that explicitly allows to take into account different learning strategies for correctly or incorrectly classified speakers. (2) An empirical loss function to control the training procedure convergence, with a likelihood-based selection of correctly or incorrectly classified competing speakers. A comparison between the proposed method and the traditional GPD algorithm is also presented.

On-line garbage modeling techniques have been shown to be a good alternative to traditional explicitly-trained garbage HMMs due to their low computational cost, their independence respect to any garbage training set and to the good results provided in several tasks related with rejection and keyword spotting. We present an extension of this techniques to word, pulse and utterance verification for real continuous speech recognition based applications. The combination of this method with application-dependent knowledge has provided a drastic control on the false alarm rate (down to 3.2%), while maintaining low and balanced performance in terms of string error rate and string rejection rate.

One of the most important issues in large-vocabulary speech recognition systems is the vocabulary-independence. In order to achieve this goal, modeling of vocabulary words by subwords units is mandatory. This paper studies the use of different subword units, comparing context-independent and context-dependent units (biphones, triphones and artificially generated triphones). From the experimental results we conclude the enormous importance of context dependence units and the good behaviour of biphone units compared to triphones for Spanish speech recognition considering the database limitations and the hardware restrictions in real systems. In order to consider the most important contextual effects in Castilian speech recognition we propose a new and simple approach based on the use of left and right-side biphones. Recognition results of 91.03% correct for vocabulary dependent data (448 surnames), and 89.6% correct for vocabulary independent data (955 surnames), using only 467 senones and 4 mixtures per state, on real telephone speech and speaker independent data are reported.

In this paper we present the results obtained when evaluating the Natural Numbers Recognizer of Telefónica I+D over some particular dialects of Spanish from Spain and America.The evaluation was made over two different data sets corresponding to two different situations.A first set includes dialects of Spanish from Spain, that were considered in the training and design of our baseline system, and a second set corresponds to Argentinian Spanish, that was not considered to train the original system.Just because we are interested in a system able to be used by a wide range of users, we tested the possibilities of MAP (Maximum-A-Priori techniques) to adapt the original HMMs in order to represent all the dialects.The experimental results show the capabilities of our recognizer to be used in applications spread over a great number of Spanishspeaking countries.

The aim of the proposed paper is to discuss how to model representations of both fundamental frequency and suprasegmental duration in TTS converters for Spanish. For this purpose we use a data-driven methodology that is able to represent both fundamental frequency and suprasegmental duration in order to model the prosody of a text-to-speech system for Spanish.>

The authors describe the main topics related to a real-time implementation of the Proposed Federal Standard 1016 (PFS-1016) 4800 bps CELP (code excited linear prediction) voice coder together with some alternatives to extend real-time CELP schemes over a wide range of bit rates (8000, 6500, and 2400 bps are considered). Several procedures are evaluated in order to select the options in PFS-1016 that provide the highest quality for an implementation based on a single AT&T DSP32C chip. From the basic CELP scheme, higher and lower bit rates are derived after evaluation of the efficiency obtained for the different elements in this speech coding algorithm.>

The main results from work on analysis techniques and quantization procedures for real-time implementation of the Proposed Federal Standard 1016 4800-b/s voice coder developed by the US DoD and AT&T Bell Laboratories are presented. Reduced search procedures are described for the adaptive and stochastic codebooks, two basic functions in code-excited linear predictive coders (CELPs), where great computational effort is required. An efficient procedure for the joint computation and quantization of the line spectrum pair (LSP) is also proposed. Details about a preliminary real-time implementation on a commercial board with a single DSP32C processor are described.>

A speech coder is presented which combines vector excitation coding (VXC) with frequency-domain representations so as to obtain a high-quality efficient scheme at 4.8 kb/s based on the use of different coding strategies and bit allocations for different frames of speech and different frequency bands. Attention is focused on the representation of voiced sounds and on multiband procedures for representing the short-time Fourier transform. The proposed scheme starts with a phonetically based frame segmentation. Multiband long-term prediction is included only when necessary, and random excitation in voiced sounds is spectrally shaped by using a linear predictive coding envelope from the long-term prediction error.>

The authors discuss various possible synthesis procedures for frequency-domain vector-excited coding (VXC). The algorithm, called vector adaptive transform coding (VATC), is a two-stage vector quantizer for the short-time Fourier transform (STFT) of speech. One stage represented by an adaptive codebook based on long-term prediction and the other based by a random codebook. The authors focus on the differences between time and frequency VXC, stressing the importance of the long-term prediction. A fast algorithm for long-term prediction is introduced, and different strategies for noise reduction are proposed. VATC is presented as a particular case of a general vector linear prediction (VLP) scheme. Following this approach, a coder based on a self-spectral shaping procedure is proposed. This coder has the potential of producing telephonic-quality speech at rates lower than 4.8 kb/s.>

A frequency-domain search procedure is presented that requires a computational load of approximately 1.5 MFLOPS, reduces the storage cost, and maintains the speech quality. A vector-excitation-coding (VXC) method in the transform domain called vector adaptive transform coding (VATC) is proposed. VATC provides a general framework for VXC in the transform domain, and the quality for the synthetic speech is close to that obtained in code-excited line prediction (CELP). The major advantages of VATC are the different ways of including information from previous frames, different perceptual distortion measurements, different transformations, efficient search algorithms, and the use of vector predictive quantization techniques so as to have a fully quantized coder at 4.8 kb/s.>

Both multipulse and stochastic coders render telephonic, or close to telephonic quality, for speech signals. Stochastic coders are more efficient from the point of view of bit rate and multipulse coders are the most efficient ones if we are looking for low computational cost. In this paper we study some possibilities of coding speech using the multipulse model but as efficiently as the stochastic coders does. Synthetic speech is generated from a multipulse sequence, obtained from a codebook, through a LPC syntesis filter. Then the model for speech signal is the multipulse model; it resembles the stochastic one in that the input sequence is obtained from a codebook but differs since it does not use pitch filter.

In this paper, we propose a simplification of the model introduced by B.S. Atal and M.R. Schroeder for a code-excited linear predictor. Our scheme is based on a two-step search of the optimum code-word. In the first step, a sub-set in the code-book is selected by using a simple distance measure and taking into account the selections already made for the previous frames of speech. In the second step, the original model of Atal and Schroeder, including perceptual weighting, is applied to the sub-set in order to make the final selection of the optimum code-word. The proposed scheme reduces the computational load involved in the original one, and our preliminar results indicate that the quality of the synthetic speech is fairly good.

We present: i) an (inner-product based) algebraic framework for using and reunderstanding Prony Spectral Line Estimation (PSLE) methods for sinusoidal fitting to a set of data. ii) an efficient way of applying PSLE to 2-D problems. The main result of i) is that Prony frequencies, rather than be obtained through a polynomial rooting, are obtained by computing the eigenvalues of a tridiagonal matrix. Item ii) will reduce the 2-D PSLE problem to a pair of 1-D PSLE solutions.