We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of m samples containing both variables and n samples missing one fixed variable. We adopt the minimax framework with $l_p^p$ loss functions, and we show that the composition of uni-variate minimax estimators achieves minimax risk with the optimal first-order constant for p ≥ 2, in the regime m = o(n).

The Gray-Wyner network subject to a fidelity criterion is studied. Upper and lower bounds for the trade-offs between the private sum-rate and the common rate are obtained for arbitrary sources subject to mean-squared error distortion. The bounds meet exactly, leading to the computation of the rate region, when the source is jointly Gaussian. They meet partially when the sources are modeled via an additive Gaussian “channel”. The bounds are inspired from the Shannon bounds on the rate-distortion problem.

This paper presents explicit solutions for two related non-convex information extremization problems due to Gray and Wyner in the Gaussian case. The first problem is the Gray-Wyner network subject to a sum-rate constraint on the two private links. Here, our argument establishes the optimality of Gaussian codebooks and hence, a closed-form formula for the optimal rate region. The second problem is Wyner’s common information and a generalization thereof, where conditional independence is generalized to a limit on the conditional mutual information. We present full explicit solutions for the scalar as well as the vector case.

An important notion of common information between two random variables is due to Wyner. In this paper, we derive a lower bound on a relaxed variant of Wyner's common information for continuous random variables. The new bound reduces to the lower bound on Wyner's common information of Liu (2018). We also show that the new lower bound is tight for a special case of the so-called “Gaussian channels”, namely, when the joint distribution of the random variables can be written as the sum of a single underlying random variable and Gaussian noises. We motivate this work from the recent variations of Wyner's common information and applications to network data compression problems such as the Gray-Wyner network.

The feedback sum-rate capacity is established for the symmetric J -user Gaussian multiple-access channel (GMAC). The main contribution is a converse bound that combines the dependence-balance argument of Hekstra and Willems (1989) with a variant of the factorization of a convex envelope of Geng and Nair (2014). The converse bound matches the achievable sum-rate of the Fourier-Modulated Estimate Correction strategy of Kramer (2002).

In the problem of coded caching for media delivery, two separate coding opportunities have been identified. The first opportunity is a multi-user advantage and crucially hinges on a public broadcast link in the delivery phase. This has been explored in a plethora of works. The second opportunity has received far less attention and concerns similarities between files in the database. Here, the paradigm is to cache “the similarity” between the files. Upon the request, the encoder refines this by providing the specific details for the requested files. Extending Gray and Wyner's work (1974), it follows that the right measure of file similarity is Wyner's Common Information and its generalizations. The present paper surveys and extends the role of Wyner's Common Information in caching. As a novel result, explicit solutions are found for the Gaussian case under mean-squared error, both for the caching problem as well as for the network considered by Gray and Wyner. Our solution leverages and extends the recent technique of factorization of convex envelopes.

We present a practical strategy that aims to attain rate points on the dominant face of the multiple access channel capacity using a standard low complexity decoder. This technique is built upon recent theoretical developments of Zhu and Gastpar on compute-forward multiple access which achieves the capacity of the multiple access channel using a sequential decoder. We illustrate this strategy with off-the-shelf LDPC codes. In the first stage of decoding, the receiver first recovers a linear combination of the transmitted codewords using the sum-product algorithm (SPA). In the second stage, by using the recovered sum-of-codewords as side information, the receiver recovers one of the two codewords using a modified SPA, ultimately recovering both codewords. The main benefit of recovering the sum-of-codewords instead of the codeword itself is that it allows to attain points on the dominant face of the multiple access channel capacity without the need of rate-splitting or time sharing while maintaining a low complexity in the order of a standard point-to-point decoder. This property is also shown to be crucial for some applications, e.g., interference channels. For all the simulations with single-layer binary codes, our proposed practical strategy is shown to be within 1.7 dB of the theoretical limits, without explicit optimization on the off-the-self LDPC codes.

The feedback sum-rate capacity is established for the symmetric three-user Gaussian multiple-access channel (GMAC). The main contribution is a converse bound that combines the dependence-balance argument of Hekstra and Willems (1989) with a variant of the “doubling trick” of Geng and Nair (2014). The converse bound matches the achievable sum-rate of the Fourier-Modulated Estimate Correction strategy of Kramer (2002). The proof arguments extend to GMACs with more than three users.

Inspired by the compute-and-forward scheme from Nazer and Gastpar, a novel multiple-access scheme introduced by Zhu and Gastpar makes use of nested lattice codes and sequential decoding of linear combinations of codewords to recover the individual messages. This strategy, coined compute-forward multiple access (CFMA), provably achieves points on the dominant face of the multiple-access capacity region while circumventing the need of time sharing or rate splitting. For a two-user multiple-access channel (MAC), we propose a practical procedure to design suitable codes from off-the-shelf LDPC codes and present a sequential belief propagation decoder with complexity comparable with that of point-to-point decoders. We demonstrate the potential of our strategy by comparing several numerical evaluations with theoretical limits.