Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. This approach is referred to as (). s differ from other approaches in the literature as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. To test the performance across various s, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. We consider three scenarios for benchmarking: (i) the classic AWGN channel, (ii) channels continuous inputs– the optical intensity and peak-power constrained AWGN channel (iii) channels with a discrete output– i.e., the Poisson channel. We also consider the extension to the multi-terminal case with (iv) the AWGN and optical MAC models. We argue that benchmarking a certain across these four scenarios provides a substantive test of performance. We study the performance ofmutual information neural estimator(MINE),smoothed mutual information lower-bound estimator(SMILE), anddirected information neural estimator(DINE) and provide insights into the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowledge of closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity-approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as accuracy, stability, and sensitivity to initialization.

This paper provides a numerical framework for computing the achievable rate region of memoryless multiple access channel (MAC) with a continuous alphabet from data. In particular, we use recent results on variational lower bounds on mutual information and KL-divergence to compute the boundaries of the rate region of MAC using a set of functions parameterized by neural networks. Our method relies on a variational lower bound on KL-divergence and an upper bound on KL-divergence based on the f-divergence inequalities. Unlike previous work, which computes an estimate on mutual information, which is neither a lower nor an upper bound, our method estimates a lower bound on mutual information. Our numerical results show that the proposed method provides tighter estimates compared to the MINE-based estimator at large SNRs while being computationally more efficient. Finally, we apply the proposed method to the optical intensity MAC and obtain a new achievable rate boundary tighter than prior works.