We consider audio decoding as an inverse problem and solve it through diffusion posterior sampling. Explicit conditioning functions are developed for input signal measurements provided by an example of a transform domain perceptual audio codec. Viability is demonstrated by evaluating arbitrary pairings of a set of bitrates and task-agnostic prior models. For instance, we observe significant improvements on piano while maintaining speech performance when a speech model is replaced by a joint model trained on both speech and piano. With a more general music model, improved decoding compared to legacy methods is obtained for a broad range of content types and bitrates. The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie’s mean. Combining Tweedie’s mean with our conditioning functions improves the objective performance. An audio demo is available at https://dpscodec-demo.github.io/

We consider source coding of audio signals with the help of a generative model. We use a construction where a waveform is first quantized, yielding a finite bitrate representation. The waveform is then reconstructed by random sampling from a model conditioned on the quantized waveform. The proposed coding scheme is theoretically analyzed. Using SampleRNN as the generative model, we demonstrate that the proposed coding structure provides performance competitive with state-of-the-art source coding tools for specific categories of audio signals.

We provide a speech coding scheme employing a generative model based on SampleRNN that, while operating at significantly lower bitrates, matches or surpasses the perceptual quality of state-of-the-art classic wide-band codecs. Moreover, it is demonstrated that the proposed scheme can provide a meaningful rate-distortion trade-off without retraining. We evaluate the proposed scheme in a series of listening tests and discuss limitations of the approach.

The coding of audio-visual signals is generally based on different paradigms for high and low rates. At high rates the signal is approximated directly and at low rates only signal features are transmitted. The recently introduced distribution preserving quantization (DPQ) paradigm provides a seamless transition between these two regimes. In this paper we present a simplified scheme that preserves the power spectral density (PSD) rather than the probability distribution. In a practical system the PSD must be estimated. We show that both forward adaptive and backward adaptive PSD estimation are possible. Our experimental results confirm that preservation of PSD at finite precision leads to a unified coding paradigm that provides effective coding at both high and low rates. An audio coding application shows the perceptual benefits of PSD preserving quantization.

This paper studies the tight rate-distortion bound for L-channel symmetric multiple-description coding of a scalar Gaussian source with two levels of receivers. Each of the first-level receivers obtains κ of the L descriptions (κ < L). The second-level receiver obtains all L descriptions. We find that if the central distortion (corresponding to the second-level receiver) is much smaller than the side distortion (corresponding to the first-level receivers), the product of a function of the side distortions and the central distortion is asymptotically independent of the redundancy between the descriptions. Using this property, we analyze the asymptotic behavior of a practical multiple-description lattice vector quantizer (MDLVQ). Our analysis includes the treatment of the MDLVQ system from a new geometric viewpoint, which results in an expression for the side distortions using the normalized second moment of a sphere of higher dimensionality than the quantization space. The expression of the distortion product derived from the lower bound is then applied as a criterion to assess the performance loss of the considered MDLVQ system. In principle, the efficiency of other practical MD systems can also be evaluated using the derived distortion product.

Modern networks are highly variable and, as a result, source coders are commonly used under conditions that they were not designed for. We address this problem with a source-coding philosophy that aims at the instantaneous re-optimization of a source coder to match a wide range of constraints on rate or quality and a wide range of packet-loss rates. We present a number of technologies that can be reconfigured by solving analytic relations that use the current conditions and a statistical description of the source as input. The technologies include distribution-preserving quantizers, flexible multiple-description quantizers, and a rate distribution scheme. Based on the generic technologies, we created a complete audio coder. Formal listening tests show that the resulting audio coding scheme with full flexibility provides a quality that is on-par with the best standardized codecs for any particular rate.

A new quantization scheme that preserves the probability distribution of the source signal is presented. The distribution preserving quantization (DPQ) achieves the optimal trade-off between mean square error and bit rate asymptotically. It provides a continuum ranging from rate-distortion optimal signal quantization to parametric coding. The method can be used as a core component for scalable coding. Its efficacy is illustrated by applying the scheme to audio coding.

This paper studies the tight rate-distortion bound for K-channel symmetric multiple-description coding for a memory less Gaussian source. We find that the product of a function of the individual side distortions (for single received descriptions) and the central distortion (for K received descriptions) is asymptotically independent of the redundancy among the descriptions. Using this property, we analyze the asymptotic behaviors of two different practical multiple-description lattice vector quantizers (MDLVQ). Our analysis includes the treatment of a MDLVQ system from a new geometric viewpoint, which results in an expression for the side distortions using the normalized second moment of a sphere of higher dimensionality than the quantization space. The expression of the distortion product derived from the lower bound is then applied as a criterion to assess the performance losses of the considered MDLVQ systems.

We consider the rate allocation problem for multiple-description quantization of the signal described by an adaptive model with a fixed structure. The source modeling in coding generally results in a two-stage description of the data, where one of the stages describes the model parameters, and the other describes the signal. Such a setup implies the existence of a trade-off between the rate spent on the parameters and the rate spent on the signal. We optimize this trade-off analytically for the multiple-description case using a method inspired by Minimum Description Length principle. We also provide an algorithm for optimizing the rate allocation between the components of the model-based multiple description coder. Finally we experimentally confirm our results. Our method facilitates the rate-adaptive multiple-description coding.

We consider adaptive two-channel multiple-description coding. We provide an analytical method for designing a resolution-constrained symmetrical multiple-description coder that uses an index assignment matrix. We use existing index assignment algorithms that are known for their good properties within our adaptive multiple-description coding architecture. These existing index assignment algorithms are parameterized and the coefficient of quantization of the side coders is described by a rational function. This leads to an analytical solution for the design problem, facilitating real-time adaptation based on information provided by feedback channels or rate conditions imposed by network management. Our experimental results show that the practical performance closely approximates theoretically obtained optimal behavior.