This paper extends the well-known source coding problem of multiple descriptions, in its general and basic setting, to analog source-channel coding scenarios. Encoding-decoding functions that optimally map between the (possibly continuous valued) source and the channel spaces are numerically derived. The main technical tool is a non-convex optimization method, namely, deterministic annealing, which has recently been successfully used in other mapping optimization problems. The obtained functions exhibit several interesting structural properties, map multiple source intervals to the same interval in the channel space, and consistently outperform the known competing mapping techniques.

This paper studies the problem of global optimization of zero-delay source-channel codes that map between the source space and the channel space, under a given transmission power constraint and for the mean-square-error distortion. Particularly, we focus on two well-known network settings: the Wyner-Ziv setting where only a decoder has access to side information and the distributed setting where independent encoders transmit over independent channels to a central decoder. Prior work derived the necessary conditions for optimality of the encoder and decoder mappings, along with a greedy optimization algorithm that imposes these conditions iteratively, in conjunction with the heuristic noisy channel relaxation method to mitigate poor local minima. While noisy channel relaxation is arguably effective in simple settings, it fails to provide accurate global optimization in more complicated settings considered in this paper. We propose a powerful nonconvex optimization method based on the concept of deterministic annealing-which is derived from information theoretic principles and was successfully employed in several problems including vector quantization, classification, and regression. We present comparative numerical results that show strict superiority of the proposed method over greedy optimization methods as well as prior approaches in literature.

This paper studies the optimization of zero-delay analog mappings in a network setting that involves distributed coding. The cost surface is known to be non-convex, and known greedy methods tend to get trapped in poor locally optimal solutions that depend heavily on initialization. We derive an optimization algorithm based on the principles of “deterministic annealing”, a powerful global optimization framework that has been successfully employed in several disciplines, including, in our recent work, to a simple zero-delay analog communications problem. We demonstrate strict superiority over the descent based methods, as well as present example mappings whose properties lend insights on the workings of the solution and relations with digital distributed coding.

This paper proposes an optimization method, based on information theoretic ideas, to a class of distributed control problems. As a particular test case, the well-known and numerically “over-mined” problem of decentralized control and implicit communication, commonly referred to as Witsenhausen's counterexample, is considered. The key idea is to randomize the zero-delay mappings. which become “soft”, probabilistic mappings to be optimized in a deterministic annealing process, by incorporating a Shannon entropy constraint in the problem formulation. The entropy of the mapping is controlled and gradually lowered to zero to obtain deterministic mappings, while avoiding poor local minima. For the particular test case, our approach obtains new mappings that shed light on the structure of the optimal solution, as well as achieving a small improvement in total cost over the state of the art in numerical approaches to this problem. Proposed method is general and applicable to any problem of similar nature.

This paper studies optimization of zero-delay source-channel codes, and specifically the problem of obtaining globally optimal transformations that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on the setting where the decoder has access to side information, whose cost surface is known to be riddled with local minima. Prior work derived the necessary conditions for optimality of the encoder and decoder mappings, along with a greedy optimization algorithm that imposes these conditions iteratively, in conjunction with the heuristic “noisy channel relaxation” method to mitigate poor local minima. While noisy channel relaxation is arguably effective in simple settings, it fails to provide accurate global optimization results in more complicated settings including the decoder with side information as considered in this paper. We propose a global optimization algorithm based on the ideas of “deterministic annealing” - a non-convex optimization method, derived from information theoretic principles with analogies to statistical physics, and successfully employed in several problems including clustering, vector quantization and regression. We present comparative numerical results that show strict superiority of the proposed algorithm over greedy optimization methods as well as over the noisy channel relaxation.