We consider audio decoding as an inverse problem and solve it through diffusion posterior sampling. Explicit conditioning functions are developed for input signal measurements provided by an example of a transform domain perceptual audio codec. Viability is demonstrated by evaluating arbitrary pairings of a set of bitrates and task-agnostic prior models. For instance, we observe significant improvements on piano while maintaining speech performance when a speech model is replaced by a joint model trained on both speech and piano. With a more general music model, improved decoding compared to legacy methods is obtained for a broad range of content types and bitrates. The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie’s mean. Combining Tweedie’s mean with our conditioning functions improves the objective performance. An audio demo is available at https://dpscodec-demo.github.io/

We provide a speech coding scheme employing a generative model based on SampleRNN that, while operating at significantly lower bitrates, matches or surpasses the perceptual quality of state-of-the-art classic wide-band codecs. Moreover, it is demonstrated that the proposed scheme can provide a meaningful rate-distortion trade-off without retraining. We evaluate the proposed scheme in a series of listening tests and discuss limitations of the approach.

In this paper, Gaussian mixture modeling is applied to error concealment for block-based packet video. A Gaussian mixture model for video data is obtained offline and is thereafter utilized online in order to restore lost blocks from spatial and temporal surrounding information. We propose estimators on closed form for missing data in the case of varying available neighboring contexts. Our error concealment strategy increases peak signal-to-noise ratio compared to previously proposed schemes. Examples of improved subjective visual quality by means of the proposed method are also supplied.

We have developed a Gaussian mixture model-based technique for the compensation of lost pixel blocks during real time transmission of video. In this paper, we pursue an argument in order to better understand how the Gaussian mixture model-based estimator works. The discussion is supported with subjective evaluations and examples. Naive viewers preferred the result of our proposed technique 72 percent of the time in comparison with the linear minimum mean square error estimator. Our scheme increases performance measured in peak signal-to-noise ratio for all the 11 standard evaluation movie clips that were used

We have developed a statistical prediction technique to compensate for lost pixel blocks during real time transmission of video over packet-switched networks. The approach could as well be used for standard inter-frame coding. The method is based on the joint modeling of pixel statistics by Gaussian mixtures. Different EM variants for decreasing computational complexity and storage space are proposed. In the case of 4/spl times/4 luminance blocks, our method increases performance by 2.3 dB when augmenting the number of mixture components from 1 to 64. We show that these results are statistically significant.

Here, we propose speech-coding procedures achieving high subjective quality, avoiding speech-specific processing and interframe exploitation. Thus, the scheme is tractable for packet-based voice communication, and has the capability of coding generic audio. The architecture is based on an modified discrete cosine transform (MDCT) representation of the signal, and combines efficient vector quantization (VQ) techniques with psychoacoustic principles. Weighted quantization of MDCT coefficients is performed, using a codebook based on a statistical model of the multidimensional MDCT pdf. The weighting and the codebook are adapted for each frame to account for masking thresholds given by a psychoacoustic analysis. Actual quantization is performed using lattices, thereby, achieving close to rate independent complexity. The result is a coding scheme operational at a range of rates. Here, a particular instance at 16 kbits/s, using a sampling frequency of 8 kHz, is shown to perform better than an LD-CELP operating at the same rate, even though no interframe memory is exploited.

We present a fixed rate encoding scheme for the line spectrum pair (LSP) representation of an LPC-filter, based on Gaussian mixture (GM) modeling. For each mixture component, we construct a codebook by a union of product quantizers. Each local codebook is trained, independently, using a clustering scheme similar to the generalized Lloyd algorithm (GLA), over synthetic data. The training algorithm iterates fast, due to low complexity encoding, and converges in few iterations. The overall codebook is a combination of local codebooks, and inherits their high performance, while having a moderate complexity. We provide numerical results for average spectral distortion (SD) of the proposed encoder, and benchmark them by a lower bound, according to high-rate theory. We achieve an average SD (full-band measure) of 1 dB at 23 b/frame, for speech signals sampled at 8 kHz and LPC of order 10. By tolerating additional complexity, we reach a SD within 0.01 dB of the lower bound.

An encoding scheme is presented based on random coding theory in conjunction with companding techniques. This combination provides an easy design, and a low-storage quantizer. The groundwork of the study is Gaussian sources. The motivation is the attractive properties of Gaussian and Gaussian mixtures (GMs), and their applications in data compression. Here, codebooks are generated by companding a union of randomly translated cubic lattices. The compander functions operate scalar wise, providing an affordable encoding complexity that is rate independent. For such a codebook, an optimal search, and the indexing of the codevectors are addressed. Performance close to finite-dimensional random coding is achieved, while the complexity is not far from scalar coding. The structure of the codebook balances between complexity and performance. In comparison to some other methods, the advantage of the proposed scheme is more distinct for correlated sources.

In this paper, Gaussian mixture (GM) models are used to design variable-dimension quantizers according to a weighted distortion criterion. A general method for combining a variable-to-fixed dimension transform, with GM modeling and quantization, is proposed. The method provides a convenient and efficient way to encode the amplitudes in a sinusoidal speech coder. Quantizers designed according to the proposed scheme are evaluated both according to weighted distortion criteria, and with respect to a high-rate bound approximation of the distortion. Informal listening tests suggest that the amplitudes can be encoded without subjective loss in a wideband harmonic coder, at a rate around 40 bits per frame (for the amplitudes only).

We propose an error protection and packet loss concealment system, based on a modified sinusoidal vocoder. By means of a few additional phase parameters, the vocoder is able to partially track the original signal during voiced segments. This signal tracking allows us to switch seamlessly between the vocoded signal and a primary encoded signal. A packetized speech transmission system, using PCM data as primary encoding, and the modified vocoder as a sub-coder, is simulated and evaluated using subjective preference testing. Packet loss concealment based on extrapolation of the vocoder parameters is found to perform well, especially at high error rates.

We address speech coding architectures, suited for channels such as packetized networks. We adopt the analysis-by-synthesis paradigm. To achieve operability at a continuum of rates, all explicit perceptual filtering in the codebook search is eliminated, and quantization is performed with lattice quantizers. In this particular implementation, rate-distortion performance is improved by entropy coding of the lattice indices. The result is a competitive coder applicable to both speech and music, where subjective as well as objective performance scales well with the rate of the coder.

In this paper, a packet loss concealment scheme based on sinusoidal extrapolation is proposed. It is to be implemented at the receiver end of a packetized speech transmission system and conceals frame-erasures in a PCM waveform. The scheme operates on the source-filter components of the speech signal, which are obtained through Linear Prediction (LP) analysis. When one or more speech frames are missing, the LP residual and filter coefficients are extrapolated separately. The residual is extrapolated based on sinusoidal modeling of the last correct residual frame, whereas the latest filter is repeated and bandwidth expanded. The proposed scheme is compared to two benchmark systems by means of subjective testing. The benchmark systems are based on repetitions of the last pitch period; one operates on the source-filter components, the other directly on the speech signal. It is noted that the systems operating on the source-filter components are preferred by the listeners, and that the proposed system is clearly superior to both benchmark systems.

Low-complexity scalable methods are of importance to achieve multi-rate and variable rate speech coding. Adopting the CELP concept, we focus on the innovation coding, suggesting an adaptive companding VQ. We present a scheme having essentially no increase in complexity as the rate is increased, yet having a competitive distortion performance. This is acheived by avoiding an explicit perceptual filtering step in the coding, still utilizing close to optimal VQ techniques in a perceptual domain. Subjective and objective distortion performance is better than, or in parity with, that of conventional white noise excitation methods or multipulse structures.

Recent advances in spectral coding for speech suggest Gaussian mixture modeling (GMM) as a tool for designing close to optimal, single stage quantizers. This combined with companding techniques promises to give flexible, no-memory methods that are essentially rate universal. We present methods for designing high performance companding structures based on a random union of Z-lattices. With this choice of basic lattice, VQ search is brought to the level of scalar quantization. OUT results show performance that clearly outperforms conventional VQ companding. OUT results are discussed in terms of the shape of the Vornoi cells of the overall quantizer. We apply the overall procedure to predictive spectrum quantization in two scenarios in terms of two audio bandwidths. For 16 kHz sampling we reach a spectral distortion (SD) of 1 at 33 bits per frame.

A theoretical analysis of recursive speech spectrum coding, where predictive and finite state schemes are special cases, is presented. We evaluate the spectral distortion (SD) theoretically and design coders that minimize the SD. The analysis rests on three cornerstones: high-rate theory, PDF modeling, and an approximation of SD. A derivation of the mean L/sub 2/-norm distortion of a recursive quantizer operating at high rate is provided. Also, the distortion distribution is supplied. The evaluation of the distortion expressions requires a model of the joint PDF of two consecutive spectrum vectors. The LPC spectrum source considered here has outcomes in a bounded region, and this is taken into account in the choice of model and modeling algorithm. It is further shown how to approximate the SD with an L/sub 2/-norm measure. Combining the results, we show theoretically that 16 bits are needed to achieve an average SD of 1 dB when quantizing ten-dimensional (10-D) spectrum vectors using a first-order recursive scheme. A gain of six bits per frame is noted compared to memoryless quantization. These results rely on high-rate assumptions which are validated in experiments. There, actual high-rate optimal coders are designed and evaluated.

In this paper, we generalize the standard blockwise linear predictive (LP) coding by introducing low-pass filtering and downsampling of the LPC vectors at the encoder side, accompanied by interpolation at the decoder. Several concepts in LP coding, such as overlapping frames, interpolation and long analysis frames, can be described in the proposed framework. We also note that the proposed methods are in agreement with previous work on spectral dynamics, which indicates that spectral dynamics is (at least) as important as spectral distortion for speech quality. We have applied the proposed method to low-rate quantization of spectrum parameters. Objective and subjective performance of the new approach is compared to a standard blockwise LP coding, at various rates. We show that both the subjective and objective quality of the new method compare favorably with standard methods. Even at as low rate as 500 bit/s, the proposed method has a good quality.

We propose a recursive coding scheme for spectral parameters. The coder utilizes a Gaussian mixture model describing the PDF of pairs of source vectors. From this model we can, at each coding instant, calculate a conditional PDF for the vector to be coded given the previously quantized vector. Knowing this conditional PDF, we can use high rate theory to design an optimal codebook. To reduce complexity we employ a classified vector quantizer structure. The mixture components with highest probability around the incoming vector are detected and the corresponding codebooks are searched. A pair of indices, representing the chosen component and codevector, is transmitted. Experiments show a small degradation in performance due to the imposed structural constraint. When compared with a standard predictive vector quantization (PVQ), slightly superior performance, both objectively and subjectively, is observed. Further, close to transparent spectral coding is achieved at 16 bits per frame when employed in a sinusoidal vocoder.

We model the underlying probability density function of vectors in a database as a Gaussian mixture (GM) model. The model is employed for high rate vector quantization analysis and for design of vector quantizers. It is shown that the high rate formulas accurately predict the performance of model-based quantizers. We propose a novel method for optimizing GM model parameters for high rate performance, and an extension to the EM algorithm for densities having bounded support is also presented. The methods are applied to quantization of LPC parameters in speech coding and we present new high rate analysis results for band-limited spectral distortion and outlier statistics. In practical terms, we find that an optimal single-stage VQ can operate at approximately 3 bits less than a state-of-the-art LSF-based 2-split VQ.

This paper presents a method for obtaining numerical estimates of high rate vector quantization (VQ) performance suitable for sources for which the PDF is not analytically available. In the proposed method, the VQ point density is described from a Gaussian mixture model optimized for the data. Employing this method for LPC spectrum quantization, we obtain high rate expressions for both the average spectral distortion (SD) and the distribution function of the SD. We estimate the minimum bits required for a quantizer to obtain an average SD of 1 dB and the outlier statistics for that quantizer. We find that approximately 3 bits can be saved as compared to a 2-split LSF-based vector quantizer.

In this paper we propose a novel technique for vector quantizer design where the reconstruction vectors are given by a linear mapping of a binary block code (LMBC). The LMBC framework provides a relation between the index bits and the reconstruction vectors through mapping properties. We define a framework, show its flexibility, and give optimality conditions. We consider source optimized vector quantization (VQ), where the objective is to directly obtain a VQ with inherent good channel robustness properties. Several instructive theoretical results and properties of the distortion experienced due to channel noise are demonstrated. These results are used to guide the design process. Both optimization algorithms and a block code selection procedure are devised. Experimental results for Gauss-Markov sources show that quantization performance close to an unconstrained VQ is obtained with a short block code which implies a constrained VQ. The resulting VQs have better channel noise robustness than conventional VQs designed with the generalized Lloyd algorithm (GLA) and splitting initialization, even when a post-processing index assignment algorithm is applied to the GLA-based VQ. We have, thus, demonstrated a unique method for direct design resulting in an inherent good index assignment combined with small losses in quantization performance.

We present an estimator-based, or soft, vector quantizer decoder for communication over a noisy channel. The decoder is optimal according to the mean-square error criterion, and Hadamard-based in the sense that a Hadamard transform representation of the vector quantizer is utilized in the implementation of the decoder. An efficient algorithm for optimal decoding is derived. We furthermore investigate suboptimal versions of the decoder, providing good performance at lower complexity. The issue of joint encoder-decoder design is considered both for optimal and suboptimal decoding. Results regarding the channel distortion and the structure of a channel robust code are also provided. Through numerical simulations, soft decoding is demonstrated to outperform hard decoding in several aspects.

A procedure is presented where the conventional innovation codebook approach of CELP coding is replaced by an interpolative scheme. A generalized LTP-codebook constitutes the basis for the interpolation. A pre-requisite to efficient interpolation concerns establishing the neighboring vectors of a given LTP entry. We discuss several working approximations for establishing a suitable neighborhood concept. By simulations we have found that the interpolative scheme leads to 17-20 bits for the excitation coding of one block. In addition to this, some 3-4 bits are required for block gain. A standard CELP typically spends 25 bits or more for one block of excitation coding. The subjective quality of our proposed coding scheme compares favorably with standard CELP. In particular, the interpolation improves the pitch-related properties giving a less noisy subjective impression.

Efficient coding of spectral parameters constitutes a major concern for speech compression. It is generally agreed that a single-stage vector quantizer, operating in the log-spectral domain would give superior performance, as compared to any other block coding procedure. Such a coder has, however, been considered impractical due to its complexity. The authors demonstrate that high quality encoding of speech spectra is feasible around 20 bits/spectrum with a single-stage approach. The experimental results are accurately predicted by a theoretical performance analysis. The author suggests a straightforward tree-based procedure that can be employed for the codebook search such that the computational complexity is acceptable.>

A soft decision decoder is presented. The soft decision decoder is optimal in the mean square sense, if the encoder entropy is full. A source vector estimate is obtained as a linear mapping of a soft Hadamard column. The soft Hadamard column is formed as a generally nonlinear mapping of soft information bits. It is shown that the best index assignment, on the encoder, is obtained in the special case of a linear mapping from the soft information bits. Simulations indicate that the jointly trained system performs better than channel optimized VQ with hard decisions. The interesting case, for applications, of using an ordinary VQ codebook as encoder, together with our soft decision decoder, is also investigated. In our examples this approach gives comparable performance to channel optimized VQ with hard decisions.>

The author addresses methods for high-performance innovation coding in a code-excited linear prediction (CELP) speech coder context. He argues that care should be taken in order to ensure that innovations do not extend outside the Voronoi regions of the LTP-stage. With this cell-conditioned approach, it is shown that innovation gain can be coded with 1 bit only (i.e. sign only). The previous method is extended to the case of delayed decision (tree search methods). It is demonstrated that, in this case, the innovation vectors should be placed near the boundaries of the LTP-Voronoi regions. This is in contrast to the strict one-at-a-time (OAT) procedure presented by P. Heddin and A. Bergstrom (1991), where innovations should be placed well inside the regions. Simulation studies clearly demonstrate that high-quality speech coding is indeed feasible using this 1 bit gain coding procedure.>

The authors propose a probability based optimization (PBO) algorithm which exploits the inherent statistical properties present in the speech signal and effectively minimizes the error probability during the classification stages of the system. The performance of this algorithm has been systematically evaluated using a MAP (maximum a posteriori) estimator and a Kohonen (1988) map. In the evaluation of the MAP classifier, the PBO algorithm contributes to a decrease of the recognition error rate with respect to both training and test data. The experimental results on applying PBO to a Kohonen map classifier are compared with the results obtained by using LVQ2, revealing a decrease of the classification error rate for both the broad class database and the vowel database.>

The authors present a computationally expensive method for tuning the innovation coding of a CELP (code excited linear prediction) coder to the LTP coding. Evidence is given, both from vector quantization (VQ) theory and from simulations, that coding the innovation gain with only one bit is feasible by introducing an adaptive procedure. A stochastic codebook of innovations is selected for the simulations. Simulations demonstrate that the density function for the amplitude of an innovation vector is distinctly bimodal. In an analogy to coding of Gaussian vectors, it is shown that this property is in agreement with classical VQ theory.>

The use of line spectral pair (LSP)-frequencies for approaching the performance of a full vector quantizer for speech coding is discussed. An algorithm is presented that attains close to optimal performance if an accumulative constraint is introduced in the parameter encoding. Numerical results are presented from the training of code-books as well as from the evaluation of the quantizer. Some listening tests are discussed.>

The problem of pitch determination in aperiodic speech signals and its relevance for practical computer speech applications is discussed. Four patterns of aperiodic voice excitation are distinguished systematically with respect to their acoustical characteristics and their distributional properties between different speakers (female and male) and different kinds of text boundaries. Several pitch determination algorithms (PDAs), including both time-domain and short-term analysis approaches such as the Gold-Rabiner algorithm, the SIFT algorithm, and the cepstrum method, are evaluated for their capacity to detect and identify these patterns correctly in continuous human speech. A PDA is presented which outperforms the other methods regarding correct voicing decision and pitch estimation in quasi-periodic as well as in aperiodic speech signals.>

A code-book approach is developed for glottal-pulse modeling of speech. The authors extend previous methods of glottal pulse analysis, suggesting a practical scheme suitable for speech coding and compression. The scheme is based on using a codebook of glottal pulse signals, thereby allowing for efficient minimization of an objective measure of distortion, consistent with ordinary CELP (code-excited linear prediction) analysis. Subjective and objective quality, as well as analysis complexity, compare favorably with established methods, such as stochastic 1024-CELP coders. Moreover, the proposed coder is robust. >

A CELP (code-excited linear predictive) coder using code books populated with pulses generated using a higher time resolution than a classical multi-pulse system is proposed. The pulses in the code book are generated as sampled sinc pulses, which are allowed to attain their corresponding maximum values at positions not necessarily equal to sample positions. The proposed code book performs equally well as CELP coders using multipulse code books in terms of robustness toward corrupted speech. However, the speech obtained is smoother, and the number of pulses required per subblock can be reduced without degrading the speech quality. A preliminary estimation of the bit rate required indicates that a reasonable choice lies in the range 8-9 kb/s.>

General methods are discussed for estimating a phase model in the context of all-pole analysis of speech. An optimal estimator is presented for the case in which the phase model is a strict all-pass filter. It is shown that the solution to this estimation problem is very close to that obtained by established methods of linear prediction. As a complement, discrete Fourier transform methods for the phase modeling are briefly discussed and evaluated. Experience with the algorithms is presented, including results for applications to glottal-inverse filtering and speech coding.>

This paper presents an analysis procedure for a parametric coder, capable of coding speech at 2-3 kbit/s. The production model used is essentially that of the LPC-vocoder, however augmented with a variable glottal pulse model for voiced excitation. Starting with formulating a suitable error criterion, a scheme is discussed in detail for extracting close to optimal synthesis parameters. The error measure is in the time-domain essentially defined as for waveform coders. A medium size data base has been used to evaluate the coder. For high quality input speech the resulting speech quality is far above that of standard vocoders. For most of the data base, even trained listeners cannot tell the coded signal from the original signal when listening over loudspeakers.

A procedure is suggested for improving LPC speech quality. The central theme is to introduce a parametric model of voiced excitation - a glottal source model. In the analysis this allows for a different method than the AR-estimation used in conventional LPC. Here, a method known as AR-X-estimation is used. A complete analysis and coding method is presented. It is found that the additional glottal parameters can be coded effectively such that the total bit rate is in the same range as for conventional LPC. The glottal LPC-vocoder does significantly improve synthesis quality as compared to standard LPC. It should be emphasized, however, that the glottal vocoder requires high quality speech as input, recorded in a phase linear system. Moreover, the computational complexity is high.

Residual-excited base-band vocoders (RELP-vocoders) are known to be one of the better alternatives for speech vocoding in the range 4.8 to 9.6 kbit/s. This paper proposes non-uniform decimation as one candidate for improving the quality of RELP vocoders. Both fixed and adaptive schemes for the time-varying decimation are discussed. It is shown that an adaptive decimation allows for coding the full band residual if desired. In fact the decimation can be used to adaptively allocate bits in accordance with the instantaneous spectrum. The work is concentrated on developing the possibilities of non-uniform down-sampling and a fully optimized implementation requires further work. Preliminary listening tests indicate that the methods proposed do improve the subjective quality significantly.

Statistical methods for estimating the parameters of a system are often based on assuming that the system inputs are Gaussian. As a result least-squares criteria are commonly used for estimating the system parameters. In this paper we will describe methods that are based on uniformly distributed system inputs. The estimates of the system parameters will then be found from min/max operations on linear combinations of the output samples. A new method for solving the resulting min/max problems is described and the min/max criterion is used in an application where it performs better than the commonly used least-squares criterion.

A complete text-to-speech system has been developed. This paper, however, is limited to a discussion of some extensions to the area function approach to speech synthesis. The voiced source uses a pulse, shaped by two sinusoids, to represent the opening and the closing phases of one glottal period. Moreover, to model the motion of the vocal cords, one reflection coefficient is made time-varying in accordance with the glottal wave. Thereby the formants are time-varying over a pitch cycle. For the production of fricative sounds, the unvoiced source can be moved to different positions in the vocal tract.

An LPC base-band vocoder is developed. The novel feature concerns the coding of the base-band. A model is set up for the base-band as a set of modulated tones. Algorithms are presented for the extraction of amplitude and phase/frequency of the tones. Implementation aspects as well as simulation results are discussed. Total bit rates in the order of 3,2-4.8 kbits are possible where approximately one half of the bits represents the base-band coding. Experiments have shown the coder to be robust to background noise.

Statistical methods for parameter estimation are often based on a Gaussian assumption. In this paper we describe a signal model based on an assumption of uniform distributions. This signal model leads to a simple, non-linear algorithm, called QD, for parameter estimation. The QD algorithm can be applied to adaptive, inverse filtering of speech signals. The technique used is to sequentially estimate the reflection coefficients of a lattice structure. Some of the results of one such application are discussed and compared to those obtained with other methods.

The problem of constructing low complexity suboptimal fixed-lag estimators is discussed. First, it is shown that any optimal fixed-lag estimator structure generally performs better if more delay is accepted than the designed lag. As this holds for any design lag including zero-lag, it is demonstrated that the zero-lag filter treated as a suboptimal fixed-lag smoother has an error considerably lower than the conventional zero-lag error. A near optimal fixed-lag smoother is then found by a straightforward extention of the zero-lag filter. Several examples are considered. Finally, the usefulness of the conventional fixed-lag mmse criterion is discussed.