We explore the duality between lossy compression and channel coding in the operational sense: whether a capacity-achieving encoder-decoder sequence achieves the rate-distortion function of the dual problem when the channel decoder [encoder] is the source compressor [decompressor, resp.], and vice versa. We show that, if used as a lossy compressor, the maximum-likelihood channel decoder of a randomly chosen capacity-achieving codebook achieves the rate-distortion function almost surely. However, operational duality does not hold for every capacity achieving encoder-decoder sequence, or rate-distortion achieving compressor-decompressor sequence. We show that there exist optimal channel coding [lossy compression] schemes, which fail when used for the dual lossy compression [channel coding resp.] problem.

This paper considers the problem of identifying the support set of a high-dimensional sparse vector, from noise-corrupted 1-bit measurements. We present passive and adaptive algorithms for this problem, both requiring no more than O(d log(D)) measurements to recover the unknown support. The adaptive algorithm has the additional benefit of robustness to the dynamic range of the unknown signal.

We explore the duality between the Gelfand-Pinsker problem of channel coding with side information at the transmitter and the Wyner-Ziv problem of lossy compression with side information at the decompressor in the operational sense: whether a capacity-achieving encoder-decoder sequence achieves the rate distortion function of the dual problem when the channel decoder (resp. encoder) is the source compressor (resp. decompressor). We show that there exist capacity-achieving channel coding schemes that also achieve the rate-distortion function for the dual problem. However, this duality does not hold for every capacity-achieving channel coding scheme. In particular, we show that the original capacity-achieving encoder-decoder scheme of Gelfand-Pinsker operates far from the Wyner-Ziv rate-distortion function.

We propose a scheme for lossy compression of discrete memoryless sources: The compressor is the decoder of a nonlinear channel code, constructed from a sparse graph. We prove asymptotic optimality of the scheme for any separable (letter-by-letter) bounded distortion criterion. We also present a suboptimal compression algorithm, which exhibits near-optimal performance for moderate block lengths.

We present two results related to the computational complexity of lossy compression. The first result shows that for a memoryless source Ps with rate-distortion function R(D), the rate-distortion pair (R(D) + gamma, D + isin) can be achieved with constant decoding time per symbol and encoding time per symbol proportional to C1(gamma)isin-C2(gamma). The second results establishes that for any given R, there exists a universal lossy compression scheme with O(ng(n)) encoding complexity and O(n) decoding complexity, that achieves the point (R,D(R)) asymptotically for any ergodic source with distortion-rate function D(.), where g(n) is an arbitrary non-decreasing unbounded function. A computationally feasible implementation of the first scheme outperforms many of the best previously proposed schemes for binary sources with blocklengths of the order of 1000.