We propose an approach for cognitive coding of speech by unsupervised extraction of contextual representations in two hierarchical levels of abstraction. Speech attributes such as phoneme identity that last one hundred milliseconds or less are captured in the lower level of abstraction, while speech attributes such as speaker identity and emotion that persist up to one second are captured in the higher level of abstraction. This decomposition is achieved by a two-stage neural net-work, with a lower and an upper stage operating at different time scales. Both stages are trained to predict the content of the signal in their respective latent spaces. A top-down path-way between stages further improves the predictive capability of the network. With an application in speech compression in mind, we investigate the effect of dimensionality reduction and low bitrate quantization on the extracted representations. The performance measured on the LibriSpeech and EmoV-DB datasets reaches, and for some speech attributes even exceeds, that of state-of-the-art approaches.

This paper proposes a novel approach for speech signal prediction based on a recurrent neural network (RNN). Unlike existing RNN-based predictors, which operate on parametric features and are trained offline on a large collection of such features, the proposed predictor operates directly on speech samples and is trained online on the recent past of the speech signal. Optionally, the network can be pre-trained offline to speed-up convergence at start-up. The proposed predictor is a single end-to-end network that captures all sorts of dependencies between samples, and therefore has the potential to outperform classicallinear/non-linear and short-termllong-term speech predictor structures. We apply it to the packet loss concealment (PLC) problem and show that it outperforms the standard ITU G.711 Appendix I PLC technique.

Until recently, there was no emotional speech dataset available in Canadian French. This was a limiting factor for research activities not only in Canada, but also elsewhere. This paper introduces the newly released Canadian French Emotional (CaFE) speech dataset and gives details about its design and content. This dataset contains six different sentences, pronounced by six male and six female actors, in six basic emotions plus one neutral emotion. The six basic emotions are acted in two different intensities. The audio is digitally recorded at high-resolution (192 kHz sampling rate, 24 bits per sample). This new dataset is freely available under a Creative Commons license (CC BY-NC-SA 4.0).

Conventional feature-based classification methods do not apply well to automatic recognition of speech emotions, mostly because the precise set of spectral and prosodic features that is required to identify the emotional state of a speaker has not been determined yet. This paper presents a method that operates directly on the speech signal, thus avoiding the problematic step of feature extraction. Furthermore, this method combines the strengths of the classical source-filter model of human speech production with those of the recently introduced liquid state machine (LSM), a biologically-inspired spiking neural network (SNN). The source and vocal tract components of the speech signal are first separated and converted into perceptually relevant spectral representations. These representations are then processed separately by two reservoirs of neurons. The output of each reservoir is reduced in dimensionality and fed to a final classifier. This method is shown to provide very good classification performance on the Berlin Database of Emotional Speech (Emo-DB). This seems a very promising framework for solving efficiently many other problems in speech processing.

Although emotions play a major role in voice communication, the quality of their reproduction by low bit rate voice coders has never been investigated so far. This paper shows that the emotional state of a speaker can be identified automatically, with reasonable precision and accuracy, using conventional voice coding parameters (pitch, voicing, energy and LPC coefficients). It also shows that the performance of this identification degrades when these parameters are quantized, especially at lower rates (1200 bits/s). This suggests that quantization of speech parameters could be improved by targeting the faithful reproduction of important higher-level voice communication attributes such as emotions, rather than simply optimizing objective measures such as the signal-to-noise ratio, mean squared error and spectral distortion.

Stop consonants are an important constituent of the speech signal. They contribute significantly to its intelligibility and subjective quality. However, because of their dynamic and unpredictable nature, they tend to be difficult to encode using conventional approaches such as linear predictive coding and transform coding. This paper presents a system to detect, segment, and modify stop consonants in a speech signal. This system is then used to assess the following hypothesis: Muting the burst phase of stop consonants has a negligible impact on the subjective quality of speech. The muting operation is implemented and its impact on subjective quality is evaluated on a database of speech signals. The results show that this apparently drastic alteration has in reality very little perceptual impact. The implications for speech coding are then discussed.

Traditionally, speech coding and audio coding were separate worlds. Based on different technical approaches and different assumptions about the source signal, neither of the two coding schemes could efficiently represent both speech and music at low bitrates. This paper presents a unified speech and audio codec, which efficiently combines techniques from both worlds. This results in a codec that exhibits consistently high quality for speech, music and mixed audio content. The paper gives an overview of the codec architecture and presents results of formal listening tests comparing this new codec with HE-AAC(v2) and AMR-WB+. This new codec forms the basis of the reference model in the ongoing MPEG standardization activity for Unified Speech and Audio Coding.

In CELP coders, the past excitation signal used to build the adaptive codebook is known to be the main source of error propagation when a frame is lost. This paper presents a novel resynchronization technique using very low bit rate side information to correct the past excitation signal after a frame erasure, the novelty being that the correction is computed in a closed loop fashion, based on the actual error introduced by the concealment. Subjective test results show that this approach is a promising area for future research on frame loss recovery.

The adaptive codebook used in code-excited linear prediction (CELP)-like speech codecs is very effective for modeling the quasi-periodic component of the excitation signal but, unfortunately, introduces a strong interframe dependency that renders the decoder vulnerable to frame erasures. For voiced speech, the error affects not only the erased frame but also all the subsequent frames. In this paper, a technique to improve the recovery after a frame erasure is proposed. The technique consists in a constrained excitation search at the encoder and a resynchronization procedure at the decoder. The constraint aims at reducing the contribution of the adaptive codebook by making the innovation codebook partially model the pitch excitation. Further, for highly voiced frames, the pitch-related information contained in the innovation excitation is exploited at the decoder to speed up the resynchronization of the adaptive codebook after a frame erasure. When applied to the adaptive multirate wideband (AMR-WB) codec, the method brings a significant improvement in the case of frame erasures, at the cost of a minor quality loss compared to the standard codec at the same bit rate. The method does not need additional delay and has the advantage of maintaining full interoperability between the standard codec and its modified version.

The adaptive codebook used in CELP (code excited linear prediction) codecs to model the pitch excitation allows the attainment of a high quality of synthesized speech but introduces a strong inter-frame dependency and consequently causes error propagation in case of frame erasure. In a previous work we showed that the error propagation can be greatly reduced by constraining, at the encoder side, the innovative codebook to partially model the pitch excitation. In this paper we extend this work by exploiting, at the decoder side, the pitch-related information present in the innovative excitation to speed up the recovery of the decoder. The method consists in adequately shifting the last pitch pulse present within the corrupted adaptive codebook memory so that it is resynchronized with the excitation parameters of the frame that follows the erased one

This paper describes how a CELP speech decoder such as the VMR-WB decoder can be modified in order to deliver decoded speech frames of variable length, a feature that is required for adaptive jitter buffering. This method is shown to be successful for playout delay adaptation in terms of both subjective quality and reactivity. Moreover, it requires almost no additional complexity provided some clever limitations are imposed on time scaling.

We propose in this paper a two-stage lossless audio coder for stereo signals. The first stage is based on a hybrid approach that uses either a stereo linear prediction or a multilayer perceptron-based (MLP) nonlinear prediction. The second stage is based on two cascaded normalized least mean square (NLMS) filters that remove the remaining redundancy in the residuals of the stereo prediction. The obtained compression ratios are equal or superior to the best state-of-the-art coders

The concealment procedure used by CELP speech decoders to regenerate lost frames introduces an error that propagates into the following frames. Within the context of voice transmission over packet networks, some packets arrive too late to be decoded and must also be concealed. Once they arrive however, those packets can be used to update the internal state of the decoder, which stops error propagation. Yet, care must be taken to ensure a smooth transition between the concealed frame and the following “updated” frame computed with properly updated internal states. During voiced or quasi-periodic segments, the pitch phase error that is generally introduced by the concealment procedure makes it difficult and detrimental to quality to use the traditional fade-in, fade-out approach. This paper presents a method to handle that pitch phase error. Specifically, the transition is done in such a way that the natural pitch periodicity of the speech signal is not broken. Index Terms: speech coding, robustness, late packets

The adaptive codebook used in CELP-like speech coders is extremely effective on voiced signals. Unfortunately, it is also the main source of error propagation at the decoder when a frame is lost. In this paper, we study several ways of limiting the energy contribution of the adaptive codebook to the synthesized speech signal. We show that a constrained search of the adaptive and innovative codebooks significantly improves the recovery time of the decoder after a lost frame, at the cost of only minor quality degradation in a clear channel. When applied to a standard codec such as the AMR-WB, this constraint only affects the encoder, and the modified codec remains fully interoperable with the standard codec.

The description and design of the source-controlled variable-rate multimode wideband (VMR-WB) codec recently selected by the 3/sup rd/ Generation Partnership Project 2 (3GPP2) for the cdma2000/spl reg/ system in Rate-Set II are presented. The paper gives an overview of the codec and the methodologies that enable high quality wideband coding at average data rates ranging from TIA/EIA/IS-733 ADR (average data rate) to that of TIA/EIA/IS-127. The codec has three modes of operation at different average data rates and a fourth mode that is interoperable with 3GPP/AMR-WB (ITU-T G.722.2). Despite the interoperability constraint, the codec is capable of meeting the aggressive performance requirements through the use of novel techniques such as noise suppression, efficient signal classification, new coding types optimized for stable voiced and unvoiced frames, novel post-processing technique for periodicity enhancement in the lower frequency band, and improved frame erasure concealment mechanisms.

We present an objective and subjective comparison of alternative methods for improving the robustness of speech coders in packet networks. The two approaches are considered: 1) adding redundancy in the packets to improve the robustness of a baseline encoder; 2) reducing (or eliminating) inter-frame dependencies at the encoder. It is shown that both approaches have to trade bit rate and/or delay for quality over lossy channels. Formal subjective tests clearly show that, using relatively simple forward error correction methods, standard coders such as ITU-T recommendation G.729 can be made significantly more robust than "frame-independent" coders, at a lower or similar bit rate.

The paper presents a method to improve the recovery of a speech decoder after the reception of one or several late frames. Rather than considering a late frame as "lost", we propose to use it in order to update the internal state of the decoder. This limits, and, in some cases, stops, the error propagation caused by the concealment. Evaluation results show that there is much to be gained in a voice over IP environment, where late frames can be used to improve the robustness against jitter without increasing the overall end-to-end delay.