The use of speech-based solutions is an appealing alternative to communicate in human-robot interaction (HRI). An important challenge in this area is processing distant speech which is often noisy, and affected by reverberation and time-varying acoustic channels. It is important to investigate effective speech solutions, especially in dynamic environments where the robots and the users move, changing the distance and orientation between a speaker and the microphone. This paper addresses this problem in the context of speech emotion recognition (SER), which is an important task to understand the intention of the message and the underlying mental state of the user. We propose a novel setup with a PR2 robot that moves as target speech and ambient noise are simultaneously recorded. Our study not only analyzes the detrimental effect of distance speech in this dynamic robot-user setting for speech emotion recognition but also provides solutions to attenuate its effect. We evaluate the use of two beamforming schemes to spatially filter the speech signal using either delay-and-sum (D&S) or minimum variance distortionless response (MVDR). We consider the original training speech recorded in controlled situations, and simulated conditions where the training utterances are processed to simulate the target acoustic environment. We consider the case where the robot is moving (dynamic case) and not moving (static case). For speech emotion recognition, we explore two state-of-the-art classifiers using hand-crafted features implemented with the ladder network strategy and learned features implemented with the wav2vec 2.0 feature representation. MVDR led to a signal-to-noise ratio higher than the basic D&S method. However, both approaches provided very similar average concordance correlation coefficient (CCC) improvements equal to 116% with the HRI subsets using the ladder network trained with the original MSP-Podcast training utterances. For the wav2vec 2.0-based model, only D&S led to improvements. Surprisingly, the static and dynamic HRI testing subsets resulted in a similar average concordance correlation coefficient. Finally, simulating the acoustic environment in the training dataset provided the highest average concordance correlation coefficient scores with the HRI subsets that are just 29% and 22% lower than those obtained with the original training/testing utterances, with ladder network and wav2vec 2.0, respectively.

To make the proper planning of bus public transportation systems, especially with the introduction of electric buses to the fleets, it is essential to characterize the routes, patterns of traffic, speed, constraints, and presence of high slopes. Currently, GPS (Global Position System) is available worldwide in the fleet. However, they often produce datasets of poor quality, with low data rates, loss of information, noisy samples, and eventual paths not belonging to regular bus routes. Therefore, extracting useful information from these poor data is a challenging task. The current paper proposes a novel method based on an unsupervised competitive density clustering algorithm to obtain hot spot clusters of any density. The clusters are a result of their competition for the GPS samples. Each cluster attracts GPS samples until a maximum radius from its centroid and thereafter moves toward the most density areas. The winning clusters are sorted using a novel distance metric with the support of a visual interface, forming a sequence of points that outline the bus trajectory. Finally, indicators are correlated to the clusters making a trajectory characterization and allowing extensive assessments. According to the actual case studies, the method performs well with noisy GPS samples and the loss of information. The proposed method presents quite a fixed parameter, allowing fair performance for most GPS datasets without needing custom adjustments. It also proposes a framework for preparing the input GPS dataset, clustering, sorting the clusters to outline the trajectory, and making the trajectory characterization.

In this paper, a method based on Long Short-Term Memory is presented to address the problem of earthquake magnitude estimation for earthquake early warning (EEW) and tsunami early warning (TW) purposes using a seismic station. An end-to-end based scheme is adopted, and a particular attention is paid to compute the magnitude of seismic events larger than M6 and reduce the effective time for TW. More so, these earthquakes are the ones that cause more fear or uncertainty in the population with provision of the most significant destructive potential. However, the occurrence of large earthquakes is low, but to counteract the drawback of limited training data, engineered features were also proposed. The earthquake magnitude relative error estimation reported here in experiments with Chilean seismic data was 4.01% and 8.04% with earthquakes M4.0 or larger (up to M8.1) and M4.0 or smaller, respectively, by employing seismic traces in the nearest station to the corresponding seismic event. The average earthquake-nearest station distance was 196 km and in 26% of the data this distance was greater than 200 km. These results are competitive with those published elsewhere and suggest the possibility to reduce the time required for EEW and specially TW.

This article presents a stand-alone automatic speech recognition system that accounts for listener movement, time-varying reverberation effects, environmental noise, and user position information for beamforming approaches in an HRI setting. We raise the importance of replacing the classical black-box integration of automatic speech recognition technology in HRI applications with the incorporation of the acoustic environment representation and modeling, and of the target source direction. Test data were recorded on a real robot under various moving conditions. For addressing the time-varying acoustic channel problem and incorporating environmental effect during training, clean speech samples were passed through estimated static channel responses and noise was added. Beamforming is investigated regarding oracle source tracking using, for instance, image processing. The proposed strategy is interesting for the robotics community, because it allows the development of voice-based HRI with limited training data and without relying on third-party technologies or Internet access eliminating the need to upload data to the cloud. In our mobile HRI scenario, the resulting speech recognition engine provided an average word error rate that is at least 19% and 34% lower than publicly available speech recognition APIs with the playback (i.e., loudspeaker) and human testing modalities, respectively.

In this paper, we propose to replace the classical black box integration of automatic speech recognition technology in HRI applications with the incorporation of the HRI environment representation and modeling, and the robot and user states and contexts. Accordingly, this paper focuses on the environment representation and modeling by training a deep neural network-hidden Markov model based automatic speech recognition engine combining clean utterances with the acoustic-channel responses and noise that were obtained from an HRI testbed built with a PR2 mobile manipulation robot. This method avoids recording a training database in all the possible acoustic environments given an HRI scenario. Moreover, different speech recognition testing conditions were produced by recording two types of acoustics sources, i.e. a loudspeaker and human speakers, using a Microsoft Kinect mounted on top of the PR2 robot, while performing head rotations and movements towards and away from the fixed sources. In this generic HRI scenario, the resulting automatic speech recognition engine provided a word error rate that is at least 26% and 38% lower than publicly available speech recognition APIs with the playback (i.e. loudspeaker) and human testing databases, respectively, with a limited amount of training data.

In this paper an uncertainty weighting scheme for DNN–HMM-based speech recognition is proposed to increase discriminability in the decoding process. To this end, the DNN pseudo-log-likelihoods are weighted according to the uncertainty variance assigned to the acoustic observation. The results presented here suggest that substantial reduction in WER is achieved with clean training. Moreover, modelling the uncertainty propagation through the DNN is not required and no approximations for non-linear activation functions are made. The presented method can be applied to any network topology that delivers log-likelihood-like scores. It can be combined with any noise removal technique and adds a minimal computational cost. This technique was exhaustively evaluated and combined with uncertainty-propagation-based schemes for computing the pseudo-log-likelihoods and uncertainty variance at the DNN output. Two proposed methods optimized the parameters of the weighting function by leveraging the grid search either on a development database representing the given task or on each utterance based on discrimination metrics. Experiments with Aurora-4 task showed that, with clean training, the proposed weighting scheme can reduce WER by a maximum of 21% compared with a baseline system with spectral subtraction and uncertainty propagation using the unscented transform. The uncertainty weighting method reduced the gap between clean and multi-noise/multi-condition training. This can be useful when it is not easy to train a DNN–HMM system in conditions that are similar to the testing ones. Finally, the presented results on the use of uncertainty are very competitive with those published elsewhere using the same database as the one employed here.

This letter describes modifications to locally normalized filter banks (LNFB), which substantially improve their performance on the Aurora-4 robust speech recognition task using a Deep Neural Network-Hidden Markov Model (DNN-HMM)-based speech recognition system. The modified coefficients, referred to as LNFB features, are a filter-bank version of locally normalized cepstral coefficients (LNCC), which have been described previously. The ability of the LNFB features is enhanced through the use of newly proposed dynamic versions of them, which are developed using an approach that differs somewhat from the traditional development of delta and delta-delta features. Further enhancements are obtained through the use of mean normalization and mean-variance normalization, which is evaluated both on a per-speaker and a per-utterance basis. The best performing feature combination (typically LNFB combined with LNFB delta and delta-delta features and mean-variance normalization) provides an average relative reduction in word error rate of 11.4% and 9.4%, respectively, compared to comparable features derived from Mel filter banks when clean and multinoise training are used for the Aurora-4 evaluation. The results presented here suggest that the proposed technique is more robust to channel mismatches between training and testing data than MFCC-derived features and is more effective in dealing with channel diversity.

This paper proposes the use of reference models to detect emotional prominence in the energy and F0 contours. The proposed framework aims to model the intrinsic variability of these prosodic features. We present a novel approach based on Functional Data Analysis (FDA) to build reference models using a family of energy and F0 contours, which are implemented with lexicon-independent models. The neutral models are represented by bases of functions and the testing energy and F0 contours are characterized by their projections onto the corresponding bases. The proposed system can lead to accuracies as high as 80.4% in binary emotion classification in the EMODB corpus, which is 17.6% higher than the one achieved by a benchmark classifier trained with sentence level prosodic features. The approach is also evaluated with the SEMAINE corpus, showing that it can be effectively used in real applications. Index Terms: Emotion detection, prosody modeling, emotional speech analysis, expressive speech, functional data analysis.

This research was funded by Conicyt-Chile under grants Fondecyt 1100195 and 11110391, and Team Research in Science and Technology ACT 1120.

This paper proposes a novel feature-space VTLN (vocal tract length normalization) method that models frequency warping as a linear interpolation of contiguous Mel filter-bank energies. The presented technique aims to reduce the distortion in the Mel filter-bank energy estimation due to the harmonic composition of voiced speech intervals and DFT (discrete Fourier transform) sampling when the central frequency of band-pass filters is shifted. This paper also proposes an analytical maximum likelihood (ML) method to estimate the optimal warping factor in the cepstral space. The presented interpolated filter-bank energy-based VTLN leads to relative reductions in WER (word error rate) as high as 11.2% and 7.6% when compared with the baseline system and standard VTLN, respectively, in a medium-vocabulary continuous speech recognition task. Also, the proposed VTLN scheme can provide significant reductions in WER when compared with state-of-the-art VTLN methods based on linear transforms in the cepstral feature-space. The warping factor estimated with the proposed VTLN approach shows more dependence on the speaker and more independence of the acoustic-phonetic content than the warping factor resulting from standard and state-of-the-art VTLN methods. Finally, the analytical ML-based optimization scheme presented here achieves almost the same reductions in WER as the ML grid search version of the technique with a computational load 20 times lower.

Americanae nace como un proyecto conjunto que surge dentro de la Red Europea de Información y Documentación sobre América Latina (REDIAL), y que ha afrontado la Biblioteca de la Agencia Española de Cooperación Internacional para el Desarrollo (AECID). Esta nueva biblioteca virtual hace más accesibles los libros digitales de tema americanista a los investigadores y usuarios interesados de cualquier parte del mundo.

This correspondence presents a novel feature-space channel compensation technique that models the convolutional distortion in the log-energy mel-filter domain by means of a polynomial approximation. The proposed parametric distortion model generates appropriate constraints in the spectral domain that help to improve the channel cancelling estimation with limited data. In a text-dependent speaker verification task, the polynomial-based channel estimation scheme can lead to reductions in equal error rate (EER) as great as 22% and 8% when compared with the baseline system and with the standard cepstral bias removal approach, respectively, with no significant increase in computational load.

Americanae nace como un proyecto conjunto que surge dentro de la Red Europea de Información y Documentación sobre América Latina (REDIAL), y que ha afrontado la Biblioteca de la Agencia Española de Cooperación Internacional para el Desarrollo (AECID). Esta nueva biblioteca virtual hace más accesibles los libros digitales de tema americanista a los investigadores y usuarios interesados de cualquier parte del mundo.

This correspondence proposes a novel feature transform for channel robustness with short utterances. In contrast to well-known techniques based on feature trajectory filtering, the presented procedure aims to reduce the time-varying component of channel distortion by applying a bandpass filter along the Mel frequency domain on a frame-by-frame basis. By doing so, the channel cancelling effect due to conventional feature trajectory filtering methods is enhanced. The filtering parameters are defined by employing a novel version of relative importance analysis based on a discriminant function. Experiments with telephone speech on a text-dependent speaker verification task show that the proposed scheme can lead to reductions of 8.6% in equal error rate when compared with the baseline system. Also, when applied in combination with cepstral mean normalization and RASTA, the presented technique leads to further reductions of 9.7% and 4.3% in equal error rate, respectively, when compared with those methods isolated.

In this paper, a novel confidence-based reinforcement learning (RL) scheme to correct observation log-likelihoods and to address the problem of unsupervised compensation with limited estimation data is proposed. A two-step Viterbi decoding is presented which estimates a correction factor for the observation log-likelihoods that makes the recognized and neighboring HMMs more or less likely by using a confidence score. If regions in the output delivered by the recognizer exhibit low confidence scores, the second Viterbi decoding will tend to focus the search on neighboring models. In contrast, if recognized regions exhibit high confidence scores, the second Viterbi decoding will tend to retain the recognition output obtained at the first step. The proposed RL mechanism is modeled as the linear combination of two metrics or information sources: the acoustic model log-likelihood and the logarithm of a confidence metric. A criterion based on incremental conditional entropy maximization to optimize a linear combination of metrics or information sources online is also presented. The method requires only one utterance, as short as 0.7 s, and can lead to significant reductions in word error rate (WER) between 3% and 18%, depending on the task, training-testing conditions, and method used to optimize the proposed RL scheme. In contrast to ordinary feature compensation and model parameter adaptation methods, the confidence-based RL method takes place in the frame log-likelihood domain. Consequently, as shown in the results presented here, it is complementary to feature compensation and to model adaptation techniques.

In this paper an automatic word stress assessment system is proposed based on a top-to-bottom scheme. The method presented is text and language independent. The utterance pronounced by the student is directly compared with a reference one. The trend similarity of F0 and energy contours are compared frame-byframe by using DTW alignment. The stress assessment evaluation system gives an EER equal to 21.5%, which in turn is similar to the error observed in phonetic quality evaluation schemes. These results suggest that the proposed system can be employed in real applications and applicable to any language.

In this paper the application of automatic speech recognition (ASR) technology in CAPT (Computer Aided Pronunciation Training) is addressed. A method to automatically generate the competitive lexicon, required by an ASR engine to compare the pronunciation of a target word with its correct and wrong phonetic realization, is presented. In order to enable the efficient deployment of CAPT applications, the generation of this competitive lexicon does not require any human assistance or a priori information of mother language dependent errors. The method presented here leads to averaged subjective-objective score correlation equal to 0.82 and 0.75 depending on the task. Index Terms: second language learning, computer aided pronunciation training, speech recognition and competing vocabulary

This paper describes a two-step Viterbi decoding based on reinforcement learning and information theory with telephone speech.The idea is to strength or weaken HMM's by using Bayesbased confidence measure (BBCM) and distances between models.If HMM's in the N-best list show a low BBCM, the second Viterbi decoding will prioritize the search on neighboring models according to their distances to the N-best HMM's.The current reinforcement learning mechanism is modeled as the linear combination of two metrics or information sources.Moreover, a criterion based on incremental conditional entropy maximization to optimize a linear combination of metrics or information sources is also presented.As shown here, the method requires only one adapting utterance and can lead to a reduction in WER as high as 10.9%.

A novel framework based on Bayes-based confidence measure (BBCM) for Multiple Classifier System (MCS) fusion is proposed.As shown here, BBCM based MCS combination scheme corresponds to the ordinary Bayes fusion weighted by the reliability of each individual classifier.BBCM provides a formal model for heuristic weighting functions employed elsewhere.When compared with the ordinary Bayesian fusion, the proposed method leads to reductions as high as 20% and 50% in EER and the area below the ROC curve, respectively, in speaker verification.

This paper proposes a new paradigm to compensate for mismatch condition in speech recognition. A two-step Viterbi decoding based on reinforcement learning is described. The idea is to strength or weaken HMM’s by using Bayes-based confidence measure (BBCM) and distances between models. If HMM’s in the N-best list show a low BBCM, the second Viterbi decoding will prioritize the search on neighboring models according to their distances to the N-best HMM’s. As shown here, a reduction of 6% in WER is achieved in a task which results difficult for standard MAP and MLLR adaptation.

In this paper a compensation method is proposed to address the problem of limited enrolling data in speaker verification.Instead of adapting the client HMM, the technique presented here modifies the verification speech signals by maximizing the a posteriori p.d.f. in order to optimize the reduction in intra-speaker variability.The proposed approach can lead to reductions of 38.9% and 61.8% in EER and in the integral below the falseacceptation / false-rejection ROC curve, respectively.

A solution to the problem of speech recognition with signals distorted by low-bit rate coders is presented in this paper. A model for the coding-decoding distortion, a HMM compensation method to include this model, and an EM-based adaptation algorithm to estimate this distortion are proposed here. Medium vocabulary continuous-speech speaker-independent recognition experiments with 8 kbps G.729(CS-CELP), 13 kbps RPE-LTP (GSM), 5.3 kbps G723.1, 4.8 kbps FS-1016 and 32 kbps G.726(ADPCM) coders show that the approach described in this paper is able to dramatically reduce the effect of the coding distortion and, in some cases, gives a word accuracy higher than the baseline system with uncoded speech. Finally, the EM estimation algorithm requires only one adapting utterance and the approach described is certainly suitable for dialogue systems where just a few adapting utterances are available.

In this letter, Bayes-based confidence measure (BBCM) in speech recognition is proposed. BBCM is applicable to any standard word feature and makes use of information about the speech recognition engine performance. In contrast to ordinary confidence measures, BBCM is a probability, which is interesting itself from the practical and theoretical point of view. If applied with word density confidence measure (WDCM), BBCM dramatically improves the discrimination ability of the false acceptance curve when compared to WDCM itself.

Several mismatch conditions can be modeled as an additive bias. This bias is considered independent of the observation vectors, although this approximation is not always accurate. In this paper the dependence of the bias on the observation vectors is taken into consideration in the context of compensating the GSM coding distortion in speech recognition. However, the results presented here can easily be generalized to deal with other types of mismatch. The coding-decoding distortion is modeled here as feature-dependent. This model is employed to propose an Expectation-Maximization (EM) estimation algorithm of the codingdecoding distortion that is able to cancel the effect of GSM coder with as few as one adapting utterance. Finally, the feature-dependent adaptation can give word error rate (WER) 26 % lower than the featureindependent model. 1.

A solution to the problem of speech recognition with signals corrupted by additive noise and distorted by low-bit rate coders is presented in this paper. The additive noise and the coding distortion are cancelled according to the following scheme: firstly, the pdf of the clean coded-decoced speech is estimated with an additive noise model; second, the pdf of the clean uncoded signal is also estimated with a coding distortion model; and finally, the HMM is compensated by using the expected value of the observation pdf in the context of the stochastic weighted Viterbi (SWV) algorithm. The approach leads to reductions as high as 50 % or 60 % in word error rate. 1.

Generally, real-time applications based on the User Datagram Protocol (UDP) generate large volumes of data and are not sensitive to network congestion. In contrast, Transmission Control Protocol (TCP) traffic is considered "well-behaved" because it prevents the network becoming congested by means of closed-loop control of packet-loss and round-trip-time. The integration of both sorts of traffic is a complex problem, and depends on solutions such as admission control that have not yet been deployed on the Internet. Moreover, the problem of quality-of-service (QoS) and resource allocation is extremely relevant from the point of view of convergence of streaming media and data transmission on the Internet. In this paper an adaptive real-time protocol based on the least mean square (LMS) algorithm is proposed to estimate the application UDP bandwidth in order to reduce the quadratic error between the packet loss and a target. Moreover, the LMS algorithm is also applied to make sure that the reduction in the average bandwidth allocated to each TCP process will not be higher than a given percentage of the average bandwidth allocated before the beginning of the UDP application.

This paper presents a framework for maximum a posteriori (MAP) speaker adaptation of state duration distributions in hidden Markov models (HMM). Four key issues of MAP estimation, namely analysis and modeling of state duration distributions, the choice of prior distribution, the specification of the parameters of the prior density and the evaluation of the MAP estimates, are tackled. Moreover, a comparison with an adaptation procedure based on maximum likelihood (ML) estimation is presented, and the problem of truncation of the state duration distribution is addressed from the statistical point of view. The results shown in this paper suggest that the speaker adaptation of temporal restrictions substantially improves the accuracy of speaker-independent (SI) HMM with clean and noisy speech. The method requires a low computational load and a small number of adapting utterances, and can be useful to follow the dynamics of the speaking rate in speech recognition.

This paper proposes the replacement of the ordinary output probability with its expected value if the addition of noise is modeled as a stochastic process, which in turn is merged with the hidden Markov model (HMM) in the Viterbi algorithm. This new output probability is analytically derived for the generic case of a mixture of Gaussians and can be seen as the definition of a stochastic version of the weighted Viterbi algorithm. Moreover, an analytical expression to estimate the uncertainty in noise canceling is also presented. The method is applied in combination with spectral subtraction to improve the robustness to additive noise of a text-dependent speaker verification system. Reductions as high as 30% or 40% in the error rates and improvements of 50% in the stability of the decision thresholds are reported.

This paper addresses the problem of temporal constraints in the Viterbi algorithm in speaker-dependent and independent tasks. The results here presented suggest that in a speaker-dependent task the introduction of temporal constraints can lead to a high improvement with additive or convolutional noise, the statistical modeling of state durations is not relevant if the max and min state duration restrictions are imposed, and truncated probability densities give better results than a metric previously proposed. Finally, word position dependent and independent temporal restrictions are compared in connected word speech recognition experiments and it is shown that the former leads to better results with the same computational load. However the duration model effect could be much less significant when the acoustic model is optimized and when the training and testing conditions are matched.

A weighted Viterbi algorithm (HMM) is proposed and applied in combination with spectral subtraction and cepstral mean normalization to cancel both additive and convolutional noise in speech recognition. The weighted Viterbi approach is compared and used in combination with state duration modelling. The results presented show that a proper weight on the information provided by static parameters can substantially reduce the error rate, and that the weighting procedure improves better the robustness of the Viterbi algorithm than the introduction of temporal constraints with a low computational load. Finally, it is shown that the weighted Viterbi algorithm in combination with temporal constraints leads to a high recognition accuracy at moderate SNRs without the need of an accurate noise model.

Addresses the problem of speech recognition with signals corrupted by additive noise at moderate signal-to-noise ratio (SNR). A model for additive noise is presented and used to compute the uncertainty about the hidden clean signal so as to weight the estimation provided by spectral subtraction. Weighted dynamic time warping (DTW) and Viterbi (HMM) algorithms are tested, and the results show that weighting the information along the signal can substantially increase the performance of spectral subtraction, an easily implemented technique, even with a poor estimation for noise and without using any information about the speaker. It is also shown that the weighting procedure can reduce the error rate when cepstral mean normalization is also used to cancel the convolutional noise.

This paper addresses the problem of speech recognition with signals corrupted by additive noise at moderate SNR. A technique based on spectral subtraction and noise cancellation reliability weighting in acoustic pattern matching algorithms is studied. A model for additive noise is proposed and used to compute the variance of the hidden clean signal information and the reliability of the spectral subtraction process. The results presented show that a proper weight on the information provided by static parameters can substantially reduce the error rate.

The problems of ecacy estimation in noise cancelling by a neural net ) and the use of this information in weighting matching algorithms are focused.Since the eect of noise on the speech signal is variable and the backpropagation training algorithm is essentially stochastic (most common patterns have more inuence in the weights re-estimation process), it is reasonable to suppose that the LIN ecacy depends on the input and each noisy frame could be associated to a reliability coecient that attempts to measure how reliable is the result of the neural net processing.Isolated word recognition experiments have shown that reliability weighting can result in a mean error rate reduction as high as 96, 80, 58 and 36 % at SNR=12, 6, 3 and 0dB, respectively, when the noise is white Gaussian.