This paper is concerned with the quantization setting where the encoder and the decoder have misaligned objectives. We first motivate the problem via a toy example which demonstrates the intricacies of the strategic quantization problem, specifically shows that iterative optimization of the decoder and the encoder mappings may not converge to a local optimum. As a remedy, we propose a dynamic programming based optimal optimization method, inspired by the early works in the quantization theory. We then extend our approach to variable-rate (entropy-coded) quantization. We finally present numerical results obtained via the proposed algorithms.

The hidden Markov model (HMM) is widely used to model processes in several real world applications, including speech processing and recognition, image understanding and sensor networks. A problem of concern is that of quantization of the sequence of observations generated by an HMM, which is referred as a hidden Markov source (HMS). Despite the importance of the problem, and the well-defined structure of the process, there has been very limited work addressing the optimal quantization of HMS, and conventional approaches focus on optimization of parameters of known quantization schemes. This paper proposes a method that directly tracks the state probability distribution of the underlying source and optimizes the encoder structure according to the estimated HMS status. Unlike existing approaches, no stationarity assumption is needed, and code parameters are updated on they: with each observation, both the encoder and the decoder refine the estimated probability distribution over the states. The main approach is then specialized to a practical variant involving switched quantizers, and an algorithm that iteratively optimizes the quantize codebooks is derived. Numerical results show superiority of the proposed approach over prior methods.

Randomized (dithered) quantization is a method capable of achieving white reconstruction error independent of the source. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization. In this paper we extend conventional dithered quantization to nonuniform quantization, via a subterfage: dithering is performed in the companded domain. Closed form necessary conditions for optimality of the compressor and expander mappings are derived for both fixed and variable rate randomized quantization. Numerically, mappings are optimized by iteratively imposing these necessary conditions. The resulting quantizer renders the reconstruction error white with negligible performance loss compared to the optimal quantizer. The framework is extended to include an explicit constraint that deterministic or randomized quantizers yield reconstruction error that is uncorrelated with the source. Surprising theoretical results show direct and simple connection between the optimal constrained quantizers and their unconstrained counterparts. Numerical results for the Gaussian source provide strong evidence that the proposed constrained randomized quantizer outperforms the conventional dithered quantizer, as well as the constrained deterministic quantizer.

This paper is concerned with transform coding of correlated sources in conjunction with variable rate quantization at high resolution. The approach builds on our prior work on optimality conditions for transform coding in the point-to-point setting. The first contribution involves transform coding with decoder side information. In this setting, side information is only available to the decoder, whereas the encoder knows the joint statistics. The necessary and sufficient condition for optimality of a unitary transform in the side information setting is derived, namely, such transform minimizes a conditional divergence-based measure of inter-dependence of the transform coefficients, given the side information. This optimality result subsumes prior, known results that were restricted to the Gaussian case, where the conditional Karhunen-Loeve transform is optimal. The second contribution involves distributed transform coding, where two correlated sources are to be transform coded separately, but decoded jointly. The necessary and sufficient condition for optimality of unitary transforms in the distributed coding setting is derived. It is then specialized to produce closed form optimal transforms for specific source densities, including the case of jointly Gaussian sources.

This paper focuses on optimal analog mappings for zero-delay, distributed source-channel coding. The objective is to obtain the optimal vector transformations that map between m-dimensional source spaces and k-dimensional channel spaces, subject to a prescribed power constraint and assuming the mean square error distortion measure. Closed-form necessary conditions for optimality of encoding and decoding mappings are derived. An iterative de- sign algorithm is proposed, which updates encoder and decoder mappings by sequentially enforcing the complementary optimality conditions at each iteration. The obtained encoding functions are shown to be a continuous relative of, and in fact subsume as a special case, the Wyner-Ziv mappings encountered in digital distributed source coding systems, by mapping multiple source intervals to the same channel interval. Example mappings and performance results are presented for Gaussian sources and channels.

This paper is concerned with optimal transform coding in conjunction with dithered quantization. While the optimal deterministic quantizer's error is uncorrelated with the reconstructed value, the dithered quantizer yields quantization errors that are correlated with the reconstruction but are white and independent of the source. These properties offer potential benefits, but also have implications on the optimization of the rest of the coder. We derive the optimal transform for consequent dithered quantization. For fixed rate coding, we show that the transform derived for dithered quantization is universally optimal (for all sources), unlike the conventional quantization case where optimality of the Karhunen-Loeve transform is guaranteed for Gaussian sources. Moreover, we establish variable rate coding optimality for Gaussian sources.

Summary form only given: Dithered quantization has useful properties such as producing quantization noise independent of the source and continous reconstruction at the decoder side. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization framework. A uniformly distributed (with step size matched to the quantization interval) dither signal is added before quantization and the same dither signal is subtracted from the quantized value at the decoder side (only subtractive dithering is considered in this paper). The quantized values are entropy coded conditioned on the dither signal. This work proposes and analyzes optimal (non-uniform) dithered quantization. One immediate problem with nonuniform dithered quantization is how to apply dithering for unequal quantization intervals. This problem is circumvented by performing dithering in the companded domain. After appropriate companding, uniform dither can be applied. The quantization problem is defined as finding the optimal compander mapping that minimizes the mean square error. To solve the problem, some approximations of rate and distortion expressions and to the compander mapping are used. First, we only consider the piecewise linear compander which is also used in deterministic quantizers. Also, we assume the error in each half quantization interval (the interval between the decision boundary and the reconstruction) is constant and only a function of the intervals length, i.e., identical to the case where for each interval a uniform dithered quantizer is applied. We also assume that rate can be approximated by the rate of the deterministic quantizer which uses the same compander. We derive the necessary conditions for optimality and design the compander by iterating between the necessary conditions.