While recent efforts in countering spoofing attacks on voice biometric systems have primarily focused on detecting synthetic speech, Physical Access (PA) attacks, such as audio replay, still pose a serious and unresolved challenge. This research gap has been mainly due to the lack of new, realistic speech corpora for training and testing effective and generalizable countermeasure systems. Given the difficulty in collecting actual audio samples from this kind of attack, simulation has been proposed as an alternative to provide audio replay training data. The objective of this work is the generation of a novel simulated database, called RIRplay, that is both realistic, in the sense of reproducing the actual spoofing process, and representative of a wide variety of possible acoustic contexts. Our results show that training with the RIRplay corpus reduces the Equal Error Rate (EER) by nearly 10 percentage points on the challenging ASVspoof 2021 evaluation set, from 36.89% to 28.04%, compared to models trained on the ASVspoof 2019 corpus, demonstrating significant improvements in out-of-domain generalization.

This work explores the performance of large speech self- supervised models as robust audio deepfake detectors. Despite the current trend of fine-tuning the upstream network, in this paper, we revisit the use of pre-trained models as feature extractors to adapt specialized downstream audio deepfake classifiers. The goal is to keep the general knowledge of the audio foundation model to extract discriminative features to feed up a simplified deepfake classifier. In addition, the generalization capabilities of the system are improved by augmenting the training corpora using additional synthetic data from different vocoder algorithms. This strategy is also complemented by various data augmentations covering challenging acoustic conditions. Our proposal is evaluated under different benchmark datasets for audio deepfake and anti-spoofing tasks, showing state-of-the-art performance. Furthermore, we analyze the relevant parts of the downstream classifier to achieve a robust system.

This paper proposes an ensembling model as spoofed speech countermeasure, with a particular focus on synthetic voice. Despite the recent advances in speaker verification based on deep neural networks, this technology is still susceptible to various malicious attacks, so that some kind of countermeasures are needed. While an increasing number of anti-spoofing techniques can be found in the literature, the combination of multiple models, or ensemble models, still proves to be one of the best approaches. However, current iterations often rely on fixed weight assignments, potentially neglecting the unique strengths of each individual model. In response, we propose a novel ensembling model, an adaptive neural network-based approach that dynamically adjusts weights based on input utterances. Our experimental findings show that this approach outperforms traditional weighted score averaging techniques, showcasing its ability to adapt to diverse audio characteristics effectively.

The success achieved by conformers in Automatic Speech Recognition (ASR) leads us to their application in other domains, such as spoofing detection for automatic speaker verification (ASV), where the conformer self-attention mechanism might effectively model and detect the artifacts introduced in spoofed speech signals. Also, conformers can naturally handle the variable duration of speech utterances. However, as with transformers, the conformer performance may degrade when trained with limited data. To address this issue, we propose utilizing conformers in conjunction with self-supervised learning, specifically leveraging a pre-trained model called wav2vec 2.0, which is pre-trained using a substantial amount of bonafide data. Our experimental results demonstrate that our proposed method achieves one of the best results in the recent ASVspoof 2021 logical access (LA) and deep fake (DF) databases.

The COVID-19 pandemic has led to the saturation of public health services worldwide.In this scenario, the early diagnosis of SARS-Cov-2 infections can help to stop or slow the spread of the virus and to manage the demand upon health services.This is especially important when resources are also being stretched by heightened demand linked to other seasonal diseases, such as the flu.In this context, the organisers of the DiCOVA 2021 challenge have collected a database with the aim of diagnosing COVID-19 through the use of coughing audio samples.This work presents the details of the automatic system for COVID-19 detection from cough recordings presented by team PANACEA.This team consists of researchers from two European academic institutions and one company: EURECOM (France), University of Granada (Spain), and Biometric Vox S.L. (Spain).We developed several systems based on established signal processing and machine learning methods.Our best system employs a Teager energy operator cepstral coefficients (TECCs) based frontend and Light gradient boosting machine (LightGBM) backend.The AUC obtained by this system on the test set is 76.31% which corresponds to a 10% improvement over the official baseline.

Biometric systems are exposed to spoofing attacks which may compromise their security, and voice biometrics based on automatic speaker verification (ASV), is no exception. To increase the robustness against such attacks, anti-spoofing systems have been proposed for the detection of replay, synthesis and voice conversion-based attacks. However, most proposed anti-spoofing techniques are loosely integrated with the ASV system. In this work, we develop a new integration neural network which jointly processes the embeddings extracted from ASV and anti-spoofing systems in order to detect both zero-effort impostors and spoofing attacks. Moreover, we propose a new loss function based on the minimization of the area under the expected (AUE) performance and spoofability curve (EPSC), which allows us to optimize the integration neural network on the desired operating range in which the biometric system is expected to work. To evaluate our proposals, experiments were carried out on the recent ASVspoof 2019 corpus, including both logical access (LA) and physical access (PA) scenarios. The experimental results show that our proposal clearly outperforms some well-known techniques based on the integration at the score- and embedding-level. Specifically, our proposal achieves up to 23.62% and 22.03% relative equal error rate (EER) improvement over the best performing baseline in the LA and PA scenarios, respectively, as well as relative gains of 27.62% and 29.15% on the AUE metric.

This article presents a recursive expectation-maximization algorithm for online multichannel speech enhancement. A deep neural network mask estimator is used to compute the speech presence probability, which is then improved by means of statistical spatial models of the noisy speech and noise signals. The clean speech signal is estimated using beamforming, single-channel linear postfiltering and speech presence masking. The clean speech statistics and speech presence probabilities are finally used to compute the acoustic parameters for beamforming and postfiltering by means of maximum likelihood estimation. This iterative procedure is carried out on a frame-by-frame basis. The algorithm integrates the different estimates in a common statistical framework suitable for online scenarios. Moreover, our method can successfully exploit spectral, spatial and temporal speech properties. Our proposed algorithm is tested in different noisy environments using the multichannel recordings of the CHiME-4 database. The experimental results show that our method outperforms other related state-of-the-art approaches in noise reduction performance, while allowing low-latency processing for real-time applications.

This paper deals with multi-channel speech recognition in scenarios with multiple speakers. Recently, the spectral characteristics of a target speaker, extracted from an adaptation utterance, have been used to guide a neural network mask estimator to focus on that speaker. In this work we present two variants of speakeraware neural networks, which exploit both spectral and spatial information to allow better discrimination between target and interfering speakers. Thus, we introduce either a spatial preprocessing prior to the mask estimation or a spatial plus spectral speaker characterization block whose output is directly fed into the neural mask estimator. The target speaker’s spectral and spatial signature is extracted from an adaptation utterance recorded at the beginning of a session. We further adapt the architecture for low-latency processing by means of block-online beamforming that recursively updates the signal statistics. Experimental results show that the additional spatial information clearly improves source extraction, in particular in the same-gender case, and that our proposal achieves state-of-the-art performance in terms of distortion reduction and recognition accuracy.

Automatic speaker verification (ASV) systems are exposed to spoofing attacks which may compromise their security. While anti-spoofing techniques have been mainly studied for clean scenarios, it has also been shown that they perform poorly in noisy environments. In this work, we aim at improving the performance of spoofing detection for ASV in clean and noisy scenarios. To achieve this, we first propose the use of Gated Recurrent Convolutional Neural Networks (GRCNNs) as a deep feature extractor to robustly represent speech signals as utterance-level embeddings, which are later used by a back-end recognizer for the final genuine/spoofed classification. Then, to enhance the robustness of the system in noisy conditions, we propose the use of signal-to-noise masks (SNMs) as new input features to inform the anti-spoofing system about the time-frequency regions of the input spectral features that are mostly affected by noise and, hence, should be neglected when computing the embeddings. To evaluate our proposals, experiments were carried out on the clean and noisy versions of the ASVspoof 2015 corpus for detecting logical access attacks, as well as on the ASVspoof 2017 database to detect replay attacks. Additional results are provided for the ASVspoof 2019 corpus, including both logical and physical scenarios. The experimental results show that our proposal clearly outperforms some well-known methods based on classical features and other similar deep feature based systems for both clean and noisy conditions.

This letter proposes a perceptual metric for speech quality evaluation, which is suitable, as a loss function, for training deep learning methods. This metric, derived from the perceptual evaluation of the speech quality algorithm, is computed in a per-frame basis and from the power spectra of the reference and processed speech signal. Thus, two disturbance terms, which account for distortion once auditory masking and threshold effects are factored in, amend the mean square error (MSE) loss function by introducing perceptual criteria based on human psychoacoustics. The proposed loss function is evaluated for noisy speech enhancement with deep neural networks. Experimental results show that our metric achieves significant gains in speech quality (evaluated using an objective metric and a listening test) when compared to using MSE or other perceptual-based loss functions from the literature.

Speech communications in real-world scenarios need high performance enhancement algorithms to address the distortions that can degrade the intelligibility and quality of the speech signal. Current portable devices usually integrate multiple microphones that can conveniently be exploited to improve the signal quality. In this paper we present a dual-microphone speech enhancement approach suitable for smartphones with primary (front) and reference (back) microphones. Our proposal is based on the use of deep neural networks which are able to obtain a non-linear mapping function between noisy and clean speech signals. We explore two different architectures: a feedforward deep neural network (DNN) with temporal context and a gated recurrent unit (GRU) recurrent neural network (RNN). The proposed system is evaluated under different acoustic conditions in close- and far-talk device positions. A comparison with other single- and dual-channel approaches shows that our proposal obtains the best performance in terms of perceptual quality.

One way to improve automatic speech recognition (ASR) performance on the latest mobile devices, which can be employed on a variety of noisy environments, consists of taking advantage of the small microphone arrays embedded in them. Since the performance of the classic beamforming techniques with small microphone arrays is rather limited, specific techniques are being developed to efficiently exploit this novel feature for noise‐robust ASR purposes. In this study, a novel dual‐channel minimum mean square error‐based feature compensation method relying on a vector Taylor series (VTS) expansion of a dual‐channel speech distortion model is proposed. In contrast to the single‐channel VTS approach (which can be considered as the state‐of‐the‐art for feature compensation), the authors’ technique particularly benefits from the spatial properties of speech and noise. Their proposal is assessed on a dual‐microphone smartphone (a particular case of interest) by means of the AURORA2‐2C synthetic corpus. Word recognition results, also validated with real noisy speech data, demonstrate the higher accuracy of their method by clearly outperforming minimum variance distortionless response beamforming and a single‐channel VTS feature compensation approach, especially at low signal‐to‐noise ratios.

Summary form only given. In this paper, we propose a novel multi-mode error concealment algorithm that aims at obtaining high quality reconstructions with reduced computational burden. Block-based coding schemes in packet loss-environment are considered. The proposed technique exploits the excellent reconstructing abilities of the kernel-based minimum mean square error (K-MMSE) estimator [1]. The complexity of our technique is dynamically adapted to the visual complexity of the area being reconstructed. The technique outperforms other state of the art algorithms and produces high quality reconstructions, equivalent to K-MMSE, while requiring less than one fourth of its computational time.

The purpose of signal extrapolation is to estimate unknown signal parts from known samples. This task is especially important for error concealment in image and video communication. For obtaining a high quality reconstruction, assumptions have to be made about the underlying signal in order to solve this underdetermined problem. Among existent reconstruction algorithms, frequency selective extrapolation (FSE) achieves high performance by assuming that image signals can be sparsely represented in the frequency domain. However, FSE does not take into account the low-pass behaviour of natural images. In this paper, we propose a modified FSE that takes this prior knowledge into account for the modelling, yielding significant PSNR gains.

Many image error concealment (EC) algorithms can be employed to reconstruct a lost region by dividing it into a set of (sub)blocks which are estimated from a set of known context pixels. In turn, these former pixels may have been previously obtained by estimation. In this situation, the order in which the lost region is recursively filled will clearly condition the resulting reconstruction. This paper proposes a novel filling order aimed to improve the performance of EC algorithms that are applied recursively over the lost area. It takes into account the reconstruction quality of the already concealed blocks in order to determine the filling order. Blocks surrounded by known pixels or by high quality reconstructions are prioritized. The proposed technique is applicable to a wide range of EC algorithms and achieves an improvement of up to 1dB (in PSNR) with negligible additional computational load.

This paper proposes a novel approach for multimedia signal reconstruction based on kernel density estimation (KDE). We make use of a vector formalism in which vectors consist of a first subvector containing a set of missing samples and a second one containing a set of available context samples. The missing subvector is reconstructed by a minimum mean square error estimator which employs a probability density function (pdf) obtained by KDE. As in any kernel-based method, the main issue to deal with is the estimation of an appropriate kernel bandwidth. We propose an adaptive procedure for bandwidth estimation (BE) especially conceived for signal reconstruction. Thus, unlike general KDE or kernel-based regression, which try to obtain a general fit, the focus of this BE procedure is on the specific missing subvector. Also, in order to exploit local signal correlations, our BE proposal adopts a scaling approach in which the bandwidth is computed as the local covariance matrix scaled by two factors. These two scale factors are obtained by minimization of two different approximations to the reconstruction error. The resulting reconstruction methodology is tested on a spatial error concealment (EC) application in which intracoded images have been transmitted through an error prone channel. The experimental results show the superiority of the proposed approach over a wide range of existing EC techniques.

Signal processing has been proven to be an useful tool to characterize damaged materials under ultrasonic nondestructive evaluation. In this work, we hypothesize that the transfer function of multilayered materials for a through-transmission configuration can be represented as a classical all-pole model with sparse coefficients. To test this hypothesis, we propose an analysis-by-synthesis scheme which, by assuming an underlying sparse digital signal model of the specimen, infers the order and extent of the model parameters corresponding to a certain impact damage level. Then, we exploit the sparse structure of the obtained digital filter for practical NDE applications, with emphasis on impact damage identification of carbon-fiber reinforced polymer plates.

This paper proposes a methodology for the application of multivariate kernel density estimation (KDE) to MMSE-based image/video error concealment (EC). We show that the estimation of the kernel bandwidth matrix for EC must follow a criterion different from that of typical KDE problems. In particular, we propose a bandwidth built as the product of a structure matrix and a scale factor obtained with a minimum square error criterion. We show that our proposal can achieve average PSNR improvements larger than 1 dB with respect to other state-of-the-art techniques.

This paper addresses the problem of feature compensation in the log-spectral domain by using the missing-data (MD) approach to noise robust speech recognition, that is, the log-spectral features can be either almost unaffected by noise or completely masked by it. First, a general MD framework based on minimum mean square error (MMSE) estimation is introduced which exploits the correlation across frequency bands to reconstruct the missing features. This framework allows the derivation of different MD imputation approaches and, in particular, a novel technique taking advantage of truncated Gaussian distributions is presented. While the proposed technique provides excellent results at high and medium signal-to-noise ratios (SNRs), its performance diminishes at low SNRs where very few reliable features are available. The reconstruction technique is therefore extended to exploit temporal constraints using two different approaches. In the first approach, time-frequency patches of speech containing a number of consecutive frames are modeled using a Gaussian mixture model (GMM). In the second one, the sequential structure of speech is alternatively modeled by a hidden Markov model (HMM). The proposed techniques are evaluated on Aurora-2 and Aurora-4 databases using both oracle and estimated masks. In both cases, the proposed techniques outperform the recognition performance obtained by the baseline system and other related techniques. Also, the introduction of a temporal modeling turns out to be very effective in reconstructing spectra at low SNRs. In particular, HMMs show the highest capability of accounting for time correlations and, therefore, achieve the best results.

In this paper, we propose a novel sequential error concealment algorithm for video and images based on sparse linear prediction. Block-based coding schemes in packet loss environments are considered. Images are modelled by means of linear prediction, and missing macroblocks are sequentially reconstructed using the available groups of pixels. The optimal predictor coefficients are computed by applying a missing data regression imputation procedure with a sparsity constraint. Moreover, an efficient procedure for the computation of these coefficients based on an exponential approximation is also proposed. Both techniques provide high-quality reconstructions and outperform the state-of-the-art algorithms both in terms of PSNR and MS-SSIM.

In this paper we propose a novel spatial error concealment algorithm for video and images based on convex optimization. Block-based coding schemes in packet loss environment are considered. Missing macro blocks are sequentially reconstructed by filling them with a weighted set of templates extracted from the available neighbourhood. Moreover, a fast approximation of the optimization method is proposed. The technique produces high quality reconstructions that outperforms the state-of-the-art algorithms both in terms of PSNR and MS-SSIM.

The use of model-based cepstral features has been shown as an effective characterization of damaged materials tested with ultrasonic non-destructive evaluation (NDE) techniques. In this work, we focus our study on carbon-fiber reinforced polymer plates and show that the use of signal models with physical meaning can provide a cepstral representation with a high discriminative power. First, we introduce a complete digital signal model based on a physical analysis of wave propagation inside the plate. The resulting model has several drawbacks: a high number of parameters to estimate and the difficulty of expressing it as a classical rational transfer function, which does not allow a model parameter estimation through classical least-squares signal modeling techniques. In order to overcome these problems, we propose two simplifications of the physical model also based on a mechanical analysis of the system. We carry out a set of damage recognition experiments showing that cepstra extracted from these models are more discriminative than other previously used methods such as the LPC cepstrum (all-pole model) or a simple FFT cepstrum.

This paper proposes a novel approach for noise-robust speech recognition which combines a missing-data (MD) derived spectral reconstruction technique and uncertainty decoding based on the weighted Viterbi algorithm (WVA). First, the noisy feature vectors are compensated by using a novel MD imputation technique based on the integration of truncated Gaussian pdfs. Although the proposed MD estimator has both the advantages of MD techniques and the use of cepstral features, it may still be affected by a number of uncertainty sources. In order to deal with these uncertainties, WVA-based uncertainty decoding is proposed. Our experiments on the Aurora-2 and Aurora-4 tasks show that the proposed MD estimator outperforms other MD imputation techniques. Also, we show that the combination of MD imputation with WVA provides better results than the combination with other uncertainty processing techniques such as the use of evidence pdfs for the estimated features.

Anticipating and characterizing damages in layered carbon fiber-reinforced polymers is a challenging problem. Non-destructive evaluation using ultrasonic signals is a well-established method to obtain physically relevant parameters to characterize damages in isotropic homogeneous materials. However, ultrasonic signals obtained from composites require special care in signal interpretation due to their structural complexity. In this paper, some enhancements on the interpretation are done by adapting classical parametrization techniques to extract relevant features from the ultrasonic signals. Thus, a cepstral-based feature extractor is firstly designed and optimized by using a classification system based on cepstral distances. Then, this feature extractor is applied in an analysis-by-synthesis scheme which, by using a numerical model of the specimen, infers the values of the damage parameters.

This paper presents a noise estimation technique based on knowledge of pitch information for robust speech recognition. In the first stage the noise is estimated by means of extrapolating the noise from frames where speech is believed to be absent. These frames are detected with a proposed pitch based VAD (Voice Activity Detector). In the second stage the noise estimation is revised in voiced frames using harmonic tunnelling technique. The tunnelling noise estimation is used at high SNRs as an upper bound of the noise rather than a suitable estimation. A spectrogram MD (Missing Data) recognition system is chosen to evaluate the proposed noise estimation. The proposed system is compared in Aurora-2 with other similar techniques like cepstral SS (Spectral Subtraction).

This paper presents a feature compensation framework based on minimum mean square error (MMSE) estimation and stereo training data for robust speech recognition. In our proposal, we model the clean and noisy feature spaces in order to obtain clean feature estimates. However, unlike other well-known MMSE compensation methods such as SPLICE or MEMLIN, which model those spaces with Gaussian mixture models (GMMs), in our case every feature space is characterized by a set of prototype vectors which can be alternatively considered as a vector quantization (VQ) codebook. The discrete nature of this feature space characterization introduces two significative advantages. First, it allows the implementation of a very efficient MMSE estimator in terms of accuracy and computational cost. On the other hand, time correlations can be exploited by means of hidden Markov modeling (HMM). In addition, a novel subregion-based modeling is applied in order to accurately represent the transformation between the clean and noisy domains. In order to deal with unknown environments, a multiple-model approach is also explored. Since this approach has been shown quite sensitive to incorrect environment classification, we adapt two uncertainty processing techniques, soft-data decoding and exponential weighting, to our estimation framework. As a result, environment miss-classifications are concealed, allowing a better performance under unknown environments. The experimental results on noisy digit recognition show a relative improvement of 87.93% in word accuracy regarding the baseline when clean acoustic models are used, while a 4.54% is achieved with multi-style trained models.

In this paper, we propose two estimators for the autocorrelation sequence of a periodic signal in additive noise. Both estimators are formulated employing tables which contain all the possible products of sample pairs in a speech signal frame. The first estimator is based on a pitch-synchronous averaging. This estimator is statistically analyzed and we show that the signal-to-noise ratio (SNR) can be increased up to a factor equal to the number of available periods. The second estimator is similar to the former one but it avoids the use of those sample products more likely affected by noise. We prove that, under certain conditions, this estimator can remove the effect of an additive noise in a statistical sense. Both estimators are employed to extract mel frequency cepstral coefficients (MFCCs) as features for robust speech recognition. Although these estimators are initially conceived for voiced speech frames, we extend their application to unvoiced sounds in order to obtain a coherent feature extractor. The experimental results show the superiority of the proposed approach over other MFCC-based front-ends such as the higher-lag autocorrelation spectrum estimation (HASE), which also employs the idea of avoiding those autocorrelation coefficients more likely affected by noise.

This paper presents a feature compensation technique based on the minimum mean square error (MMSE) estimation for robust speech recognition. Similarly to other MMSE compensation methods based on stereo data, our approach models the differences between clean and noisy feature spaces, and the resulting MMSE estimate of the clean feature vector is obtained as a piece-wise linear transformation of the noisy one. However, unlike other well-known MMSE techniques such as SPLICE or MEMLIN, which model the feature spaces with GMMs, in our proposal each feature space is characterized by a set of cells obtained by means of VQ quantization. This VQ-based approach allows a very efficient implementation of the MMSE estimator. Also, the possible degradation inherent to any VQ process is overcome by a strategy based on considering different subregions inside each cell and a subregion-based mean and variance compensation. The experimental results show that, along with a a very efficient MMSE estimator, our technique achieves even better recognition accuracies than SPLICE and MEMLIN.

In this paper, we analyze the performance of network speech recognition (NSR) over IP networks, adapting and proposing new solutions to the packet loss problem for code excited linear prediction (CELP) codecs. NSR has a client-server architecture which places the recognizer at the server side using a standard speech codec for speech transmission. Its main advantage is that no changes are required for the existing client devices and networks. However, the use of speech codecs degrades its performance, mainly in the presence of packet losses. First, we study the degradations introduced by CELP codecs in lossy packet networks. Later, we propose a reconstruction technique based on minimum mean square error (MMSE) estimation using hidden Markov models. This approach also allows us to obtain reliability measures associated to each estimate. We show how to use this information to improve the recognition performance by means of soft-data decoding and weighted Viterbi algorithm. The experimental results are obtained for two well-known CELP codecs, G.729 and AMR 12.2 kbps, carrying out recognition from decoded speech. Finally, we analyze an efficient and improved implementation of the proposed techniques using an NSR system which extracts speech recognition features directly from the bit-stream parameters. The experimental results show that the different proposed NSR systems achieve a comparable performance to distributed speech recognition (DSR).

The widely used code-excited linear prediction (CELP) paradigm relies on a strong interframe dependency which renders CELP-based codecs vulnerable to packet loss. The use of long-term prediction (LTP) or adaptive codebooks (ACB) is the main source of interframe dependency in these codecs, since they employ the excitation from previous frames. After a frame erasure, previous excitation is unavailable and a desynchronization between the encoder and the decoder appears, causing an additional distortion which is propagated to the subsequent frames. In this paper, we propose a novel media-specific Forward Error Correction (FEC) technique which retrieves LTP-resynchronization with no additional delay at the cost of a very small bit of overhead. In particular, the proposed FEC code contains a multipulse signal which replaces the excitation of the previous frame (i.e., ACB memory) when this has been lost. This multipulse description of the previous excitation is optimized to minimize the perceptual error between the synthesized speech signal and the original one. To this end, we develop a multipulse formulation which includes the additional CELP processing and, in addition, can cope with the presence of advanced LTP filters and the usual subframe segmentation applied in modern codecs. Finally, a quantization scheme is proposed to encode pulse parameters. Objective and subjective quality tests applied to our proposal show that the propagation error due to LTP filter can practically be removed with a very little bandwidth increase.

While VoIP (voice over IP) is gaining importance in comparison with other types of telephony, packet loss remains as the main source of degradation in VoIP systems. Traditional speech codecs, such as those based on the CELP (code excited linear prediction) paradigm, can achieve low bit-rates at the cost of introducing interframe dependencies. As a result, the effect of a packet loss burst is propagated to the frames correctly received after the burst. iLBC (internet low bit-rate codec) alleviates this problem by removing the interframe dependencies at the cost of a higher bit-rate. In this paper we propose a combination of iLBC with an ACELP (algebraic CELP) codec in which a variable number of ACELP-coded frames is inserted between every two iLBC-coded frames. The experimental results show that the combined codec can achieve a performance close to that of iLBC at different loss conditions but with a smaller bit-rate. Also, scalability is achieved by modifying the number of inserted ACELP-coded frames.

This paper focuses on the application of a previously proposed interleaving, derived from the Ramsey convolutional class, in a voice streaming context with bursty packet losses. This kind of interleaving has already shown significant improvements in a distributed speech recognition context, in comparison with the widely used minimum latency block interleavers (MLBI). Here, the effectiveness of these interleavers is evaluated with an internet-oriented speech codec, such as iLBC. Since iLBC avoids error propagation due to lost frames and only uses the previously received frame to recover from those, Ramsey inter-leaving turns out especially suitable for this codec. In order to measure the performance of the system, the ITU PESQ algo-rithm is applied along with an intelligibility criterion based on the accuracy obtained through an automatic speech recognizer (ASR). In addition, an informal subjective test is carried out to corroborate the ASR scores. Results show that the proposed Ramsey-derived interleaving provides at least the same quality and better intelligibility than the MLBI ones when it is applied to iLBC codec. 1.

This paper presents a framework for the error concealment of multimedia signals transmitted over wireless and IP networks. This framework is based on two elements. First, the replacements for the erroneous channel outputs are obtained from a minimum mean square error (MMSE) estimation which takes into account both the source and all the available information from the transmission channel. Then, hidden Markov models (HMMs) provide the required stochastic modeling which allows the estimation. This modeling makes it possible a natural integration of the two types of information (source and available data from the channel). This framework is developed for both, wireless channels with errors at the bit level and packet channels degraded by packet loss. In the first case, a very high performance can be obtained even without the need of soft decision or any type of channel SNR estimation. In the second case, the lack of channel outputs during packet losses must be compensated either by enhancing the source model (increasing the model order) or by introducing media-specific FEC codes which may substitute the missing packets. This second option has yielded higher performance than the first one.

This paper proposes a method for the remote recognition of speech coded with the iLBC codec, which is employed by a number of VoIP systems. While the usual way of performing recognition of coded speech is to decode first the speech signal and use it as input to the recognition engine, our system directly converts the iLBC parameters into recognition features. The main advantage of this approach is to avoid any type of decoding post-processing which, although originally conceived to improve the speech perception, can be harmful for a recognition system. Our method ensures the compatibility between the speech spectra provided by the iLBC codec and those employed for cepstrum computation and introduces a robust and suitable packet loss concealment strategy. Our experimental results show that the proposed system achieves a performance better than that obtained from iLBC-decoded speech and similar to that of a distributed speech recognition system over a clean or degraded transmission channel.

In this work we present a complete set of techniques devoted to offer robustness against frame losses in distributed speech recognition over packet-switched networks. The proposed scheme is composed of tree techniques, two of them are applied at the sender and the last one in the recognizer itself. On one hand, a media-specific forward error correction (FEC) technique is used to allow the recovery of information within the bursts. On the other hand, a recognizer-based technique well known by its remarkable ability to reduce the effects of long consecutive frame losses during recognition, the weighted Viterbi algorithm (WVA), is used to handle the additional information introduced by FEC codes. Moreover, a double stream strategy whereby interleaving can be applied along with FEC codes without any delay increase, is also applied. The application of interleaving allows to reduce the perceived burst length at the receiver, further improving the recognition performance. As a result, the proposed scheme can provide an acceptable performance even under extremely adverse channel conditions.

This paper focuses on the application of the Ramsey class of interleavers to the problem of achieving robust distributed speech recognition in the presence of burst-like packet loss. Although block interleavers of minimal latency have been already proposed by other authors in order to cope with this problem, the Ramsey class offers a more powerful kind of interleavers. Taking into account the behavior of the error concealment techniques commonly used in these recognition systems, it is possible to design Ramsey interleavers which, in comparison with the block interleavers of minimal latency, can counteract longer bursts with the same latency and thus improve the robustness of the recognition system. The effectiveness of the proposed design criterion is verified by means of simulations and experimental comparisons

Abstract Interleaving has shown to be a useful technique to provide robustdistributed speech recognition over IP networks. This is due toits ability to disperse consecutive losses. However, this ability isrelated to the delay introduced by the interleaver. In this work,we propose a novel multi-flow block interleaver which exploitsthe presence of several streams and allows to reduce the involveddelay. Experimental results have shown that this interleaver ap-proximates the performance of end-to-end interleavers but with afraction of their delay. As disadvantage, this interleaver must beplaced in a common node where more than one flow are available. Index Terms : distributed speech recognition, IP networks, inter-leaving, active networks. 1. Introduction Since its beginning, Internet has been growing in size, incorporat-ing many new networks, as well as in functionality, adding newservices. As many other features have been integrated into In-ternet, such as mailing, instant messaging, telephony and so on,speech enabled services (SES) are also being incorporated. Theseservices provide ubiquitous speech recognition, allowing multipleusers to remotely access and share high performance recognitionengines.A very attractive approach to speech recognition over IP net-works is the distributed speech recognition (DSR) solution [1]. Asmany other services over Internet, it is based on a client-serverarchitecture. On one hand, a simple and low power client (

This paper presents a mixed recovery scheme for robust distributed speech recognition (DSR) implemented over a packet channel which suffers packet losses. The scheme combines media-specific forward error correction (FEC) and error concealment (EC). Media-specific FEC is applied at the client side, where FEC bits representing strongly quantized versions of the speech vectors are introduced. At the server side, the information provided by those FEC bits is used by the EC algorithm to improve the recognition performance. We investigate the adaptation of two different EC techniques, namely minimum mean square error (MMSE) estimation, which operates at the decoding stage, and weighted Viterbi recognition (WVR), where EC is applied at the recognition stage, in order to be used along with FEC. The experimental results show that a significant increase in recognition accuracy can be obtained with very little bandwidth increase, which may be null in practice, and a limited increase in latency, which in any case is not so critical for an application such as DSR

Network-based speech recognition (NSR) and distributed speech recognition (DSR) have been proposed as solutions to translate speech recognition technologies to mobile environments. NSR is the most straightforward solution since it does not require any modification in the mobile phone, however DSR offers higher robustness against codec compression and transmission channel degradation. This paper explores an alternative approach for remote speech recognition which combines the advantages of NSR and DSR. In this scheme, a standard speech codec is used for speech transmission but the recognition is performed from the received codec parameters. In particular, we focus on the effect of transmission channel errors, which can cause a more severe performance reduction on speech recognition than codec distortion. First, we show that an NSR solution can approach DSR through a reconstruction technique along with an adapted noise reduction technique originally proposed for acoustic noise. Then, these results are improved by working with recognition features directly extracted from the codec bitstream by means of parameter transcoding. Required modifications on current networks in order to access the bitstream are described. The network upgrading with the tandem free operation (TFO) protocol is an attractive solution. This upgrade not only offers an overall improvement on the end-to-end speech quality, but would also allow a recognition performance similar, and even higher in poor channel conditions, to that obtained by DSR when parameter transcoding along with the proposed mitigation techniques are applied

This paper proposes a new packet loss concealment technique based on the inclusion in each packet of a few FEC bits, representing data replicas, combined with a minimum mean square error estimation (MMSE). This technique is developed for an Aurora-2 distributed speech recognition system working over an IP network. In addition to the data representing the transmitted speech frames, each packet includes some FEC bits representing a strongly VQ-quantized version (replicas) of previous and subsequent frames. When a loss burst occurs, the lost frames can be reconstructed from the VQ replicas. In order to mitigate the degradation introduced by the coarse VQ quantization of the replicas, a model-based MMSE estimation is applied. The experimental results show that, under a strongly degraded channel, it is possible to obtain up to 83.31 % of word accuracy with only 4 FEC bits or 88.47 % with 8 FEC bits per packet, when the Aurora mitigation algorithm only obtains 76.98 %.

This paper describes a method of compensating for nonlinear distortions in speech representation caused by noise. The method described here is based on the histogram equalization method often used in digital image processing. Histogram equalization is applied to each component of the feature vector in order to improve the robustness of speech recognition systems. The paper describes how the proposed method can be applied to robust speech recognition and it is compared with other compensation techniques. The recognition experiments, including results in the AURORA II framework, demonstrate the effectiveness of histogram equalization when it is applied either alone or in combination with other compensation techniques.

This work addresses the mitigation of channel errors by means of efficient minimum mean-square-error (MMSE) estimation. Although powerful model-based implementations have been recently proposed, the computational burden involved can make them impractical. We propose two new approaches that maintain a good level of performance with a low computational complexity. These approaches keep the simple structure and complexity of a raw MMSE estimation, although they enhance it with additional source a priori knowledge. The proposed techniques are built on a distributed speech recognition system. Different degrees of tradeoff between recognition performance and computational complexity are obtained.

Network-based speech recognition (NSR) using the conventional speech channel with the enhanced full rate (EFR) or the adaptive multi-rate (AMR) codec is a very attractive approach since no change to existing mobile phones is needed. However, NSR reveals a degrading performance due to both transmission channel errors and the speech encoding process in comparison with distributed speech recognition (DSR), where speech features are efficiently coded and transmitted on a data channel. We focus on the degradation of the speech features caused by channel errors in an NSR system and propose methods to improve the quality of these features. Applying these methods, it turns out that the performance of an NSR system based on EFR coding is comparable to that based on DSR.

A study of combined source and channel coding applied to LSP parameters in wideband speech coding is presented. The traditional approach to protect against channel errors is to increase the bit-rate for channel coding, decreasing the bit-rate of the source coding according the channel conditions. Joint source-channel coding is an alternative that provides a technique to mitigate channel errors without an increase of the bit-rate due to channel coding. This paper presents a study of channel optimized vector quantizer and channel optimized matrix quantizer applied to line spectral pairs (LSP) parameters in wideband speech coding. Gaussian and slow-fading Rayleigh channels are considered and GMSK (Gaussian minimum shift-keying) is used as the modulation technique. In addition, for comparison purposes, the performance of other schemes (split vector quantization, split matrix quantization and split multistage vector quantization) for quantizing the LSP parameters are evaluated.

Distributed speech recognition (DSR) has been recently proposed as an efficient way of translating automatic speech recognition technologies to mobile and IP network application. In this paper we propose a channel error mitigation technique with a low computational complexity that improves the mitigation technique proposed in the ETSI standard for DSR (ETSI-ES-201-108 v1.12) for bad channel conditions. We also study the influence of the vector quantization index assignment on the proposed mitigation technique and design a new index assignment that gets some improvement on the proposed technique.

In this paper, we develop a new mitigation technique for a distributed speech recognition system over IP. We have designed and tested several methods to improve the interpolation used in the Aurora DSR ETSI standard without any significant increase of computational cost at the decoder. These methods make use of the information contained in the data-source, because, in IP networks, unlike in cellular networks, no information is received during packet losses. When a packet loss occurs, the lost information can be reconstructed through estimations from the N nearest received packets. Due to the enormous amount of combinations from previous and next received speech vector sequences, we have developed a methodology that drastically reduces the amount of required estimations.

The noise usually produces a non-linear distortion of the feature space considered for Automatic Speech Recognition. This distortion causes a mismatch between the training and recognition conditions which significantly degrades the performance of speech recognizers. In this contribution we analyze the effect of the additive noise over cepstral based representations and we compare several approaches to compensate this effect. We discuss the importance of the non-linearities introduced by the noise and we propose a method (based on the histogram equalization technique) specifically oriented to the compensation of the non-linear transformation caused by the additive noise. The proposed method has been evaluated using the AURORA-2 database and task. The recognition results show significant improvements with respect to other compensation methods reported in the bibliography and reveals the importance of the non-linear effects of the noise and the utility of the proposed method.

Distributed Speech Recognition involves the development of techniques to mitigate the degradations that the transmission channel introduces in the speech features. This work proposes an HMM framework from which different mitigation techniques oriented to bursty channels can be derived. In particular, two MMSE-based and a new Viterbi-based mitigation procedures are derived under this framework. Several implementation issues such as the channel SNR estimation or the application of hard decision on the received signal vectors are dealt with. Also, different boundary conditions suitable for the speech recognition application are studied for the different mitigation procedures. The experimental results show that the HMM-based techniques can effectively mitigate channel errors, even in very poor channel conditions.

Recently, the first version of an ETSI standard for Distributed Speech Recognition has been proposed. The main benefit of this approach is the possibility of maintaining a high recognition performance when accessing remote information systems. The use of a digital channel for transmission of the encoded speech parameters implies the introduction of several channel distortions. Our paper deals with the mitigation of such distortions. We study the application of MMSE estimation to this problem and propose a new MMSE procedure that obtains the probabilities needed for MMSE from a forward-backward algorithm. We show that MMSE estimation obtains better performance than the mitigation algorithm described in the ETSI standard under different channel conditions.

This work presents a quantization technique for LSP parameters which results in a low bit-rate transmission while providing protection against channel errors. As a generalization of the so called Channel Optimized Vector Quantization (COVQ), Channel Optimized Matrix Quantization (COMQ) can remove intraframe and interframe LSP redundancy with the target of protecting the information sent through a channel in the presence of noise. Split COMQ is used in order to reduce storage requirements and complexity. Results show that Split COMQ gives better performance under certain error conditions and a lower bit rate transmission in all channel conditions compared to the reference quantization techniques.

In this paper we apply a model-based compensation method to cancel the effect of the additive noise in Automatic Speech Recognition systems. The method is formulated in a statistical framework in order to perform the optimal compensation of the noise effect given the observed noisy speech, a model describing the statistics of the speech recorded in a clean reference environment and the estimation of the noise in the noisy recognition environment. The noise is estimated using the first frames of the sentence to be recognized and a frame-by-frame noise compensation algorithm is performed, so that the compensation procedure does not constrain real-time speech recognition systems and is compatible with emerging technologies based on distributed speech recognition.

The algorithms for the reduction of the number of features without degrading the performance of pattern recognition systems play an important role in real applications. A new algorithm for feature selection is proposed. This algorithm is based on the discriminative feature extraction (DFE) technique and has been applied to speech recognition. The experimental results show that the recognition systems accept important reductions of the number of features without a degradation of the performance. For the representation used in our experiments, the recognition error-rate is not significantly increased when the number of components in the feature vector is reduced from 42 to 20.

This work presents the STACC, Sistema Telef onico Autom atico de Consulta de Calificaciones (Automatic Telephone System for Consulting Marks). This system has been developed at our laboratory during 1996 and implements a service through telephone line that allows the students to consult by speech their marks after the exams by means of a simple phone call. This experience provided us an interesting point of view about the problems of real applications of speech technology. In this work we describe the system and some statistics about the use of STACC by the students are presented.

Signal representation is crucial for designinga speechrecognizer. The feature extractor selects the information to be used by the classifier to perform the recognition. In noisy environments, the data vectors representing the speech signal are changed and the recognizer performance is degraded by two main facts: (1) the mismatch between the training and the recognition conditions and (2) the degradation of the signal to be recognized. In such a situation, the representation of the speech signal plays an important role. In this paper, we analyze the importance of the representation for speechrecognition in noise. We apply the Discriminative Feature Extraction (DFE) method to optimize the representation. The experiments presented in this work show that the DFE method, which has been successfully applied in clean environments, leads also to improvements of the speech recognizers in noise.

Research on multiple vector quantization (MVQ) has shown the suitability of such a technique for speech recognition. Basically, MVQ proposes the use of one separate VQ codebook for each recognition unit. Thus, a MVQHMM model is composed of a VQ codebook and a discrete HMM model. This technique allows the incorporation in the recognition dynamics of the input sequence information wasted by discrete HMM models in the VQ process. The use of distinct codebooks also allows one to train them in a discriminative manner. We propose a new VQ codebook design method for MVQ-based systems, obtained from a modified maximum mutual information estimation. This method provides meaningful error reductions and is performed independently from the estimation of the discrete HMM part of the MVQ model. The results show that the proposed discriminative design turns the MVQHMM technique into a powerful acoustic modeling tool in comparison with other classical methods such as discrete or semicontinuous HMMs.

Although the continuous HMM (CHMM) technique seems to be the most flexible and complete tool for speech modeling, it is not always used for the implementation of speech recognition systems due to several problems related to training and computational complexity. Besides, it is not clear the superiority of continuous models over other well-known types of HMMs, such as discrete (DHMM) or semicontinuous (SCHMM) models, or multiple vector quantization (MVQ) models, a new type of HMM modeling. The authors propose a new variant of HMM models, the SCMVQ, HMM models (semicontinuous multiple vector quantization HMM), that uses one VQ codebook per recognition unit and several quantization candidates, Formally, SCMVQ modeling is the closest one to CHMM, although requiring less computation than SCHMMs. Besides, the authors show that SCMVQs can obtain better recognition results than DHMMs, SCHMMs or MVQs.>