A 3-node lossy source coding problem for a 2-DMS (X1, X2) is considered. Source nodes 1 and 2 observe X1and X2, respectively, and each wishes to reconstruct the other source with a prescribed distortion. To achieve these goals, nodes 1 and 2 send descriptions of their sources to relay node 3. The relay node then broadcasts a joint description to the source nodes. A cutset outer bound and a compress-linear code inner bound are established and shown to coincide in several special cases. A compute-compress inner bound is then presented and shown to outperform the compress-linear code in some cases. An outer bound based on Kaspi's converse for the two-way source coding problem is shown to be strictly tighter than the cutset outer bound.

In a discrete denoising problem, if the denoiser knows the clean source distribution, the Bayes optimal denoiser is the Bayes response of the posterior distribution of the source given the noisy observations. However, in many applications the source distribution is unknown.We consider the Bayes response based on the approximate posterior distribution induced by a universal lossless compression code. Motivated by this approach, we present the empirical conditional entropy-based denoiser. Simulations show that when the source alphabet is small, the proposed denoiser achieves the performance of the Universal Discrete DEnoiser (DUDE). Furthermore, if the alphabet size increases, the proposed denoiser degrades more gracefully than the DUDE.

The geometric mean decomposition (GMD) transform coder (TC) was recently introduced and was shown to achieve the optimal coding gain without bit loading under the high bit rate assumption. However, the performance of the GMD transform coder is degraded in the low rate case. There are mainly two reasons for this degradation. First, the high bit rate quantizer model becomes invalid. Second, the quantization error is no longer negligible in the prediction process when the bit rate is low. In this letter, we introduce dithered quantization to tackle the first difficulty, and then redesign the precoders and predictors in the GMD transform coders to tackle the second. We propose two dithered GMD transform coders: the GMD subtractive dithered transform coder (GMD-SD) where the decoder has access to the dither information and the GMD nonsubtractive dithered transform coder (GMD-NSD) where the decoder has no knowledge about the dither. Under the uniform bit loading scheme in scalar quantizers, it is shown that the proposed dithered GMD transform coders perform significantly better than the original GMD coder in the low rate case.

In this paper, an information theoretic formulation of the distributed averaging problem previously studied in computer science and control is presented. We assume a network with$m$nodes each observing a white Gaussian noise (WGN) source. The nodes communicate and perform local processing with the goal of computing the average of the sources to within a prescribed mean squared error distortion. The network rate distortion function$R^{\ast }(D)$for a two-node network with correlated Gaussian sources is established. A general cutset lower bound on$R^{\ast }(D)$is established and shown to be achievable to within a factor of$2$via a centralized protocol over a star network. A lower bound on the network rate distortion function for distributed weighted-sum protocols, which is larger in order than the cutset bound by a factor of$\log m$, is established. An upper bound on the network rate distortion function for gossip-base weighted-sum protocols, which is only$\log \log m$larger in order than the lower bound for a complete graph network, is established. The results suggest that using distributed protocols results in a factor of$\log m$increase in order relative to centralized protocols.

We investigate distributed source coding of two correlated sources X and Y where messages are passed to a decoder in a cascade fashion. The encoder of X sends a message at rate R1 to the encoder of Y. The encoder of Y then sends a message to the decoder at rate R2based both on Y and on the message it received about X. The decoder's task is to estimate a function of X and Y. For example, we consider the minimum mean squared-error distortion when encoding the sum of jointly Gaussian random variables under these constraints. We also characterize the rates needed to reconstruct a function of X and Y losslessly. Our general contribution toward understanding the limits of the cascade multiterminal source coding network is in the form of inner and outer bounds on the achievable rate region for satisfying a distortion constraint for an arbitrary distortion function d(x, y, z). The inner bound makes use of a balance between two encoding tactics—relaying the information about X and recompressing the information about X jointly with Y. In the Gaussian case, a threshold is discovered for identifying which of the two extreme strategies optimizes the inner bound. Relaying outperforms recompressing the sum at the relay for some rate pairs if the variance of X is greater than the variance of Y.

An information theoretic formulation of distributed averaging is presented. We assume a network with m nodes each observing an i.i.d. source; the nodes communicate and perform local processing with the goal of computing the average of the sources to within a prescribed mean squared error distortion. The network rate distortion function R* (D) for a 2-node network with correlated Gaussian sources is established. A general cutset lower bound on R* (D) with independent Gaussian sources is established and shown to be achievable to within a factor of 2 via a centralized protocol. A lower bound on the network rate distortion function for distributed weighted-sum protocols that is larger than the cutset bound by a factor of log m is established. An upper bound on the expected network rate distortion function for gossip-based weighted-sum protocols that is only a factor of log log m larger than this lower bound is established. The results suggest that using distributed protocols results in a factor of log m increase in communication relative to centralized protocols.