This paper is concerned with the computation of the capacity region of a continuous, Gaussian vector broadcast channel (BC) with covariance matrix constraints. Since the decision variables of the corresponding optimization problem are Gaussian distributed, they can be characterized by a finite number of parameters. Consequently, we develop new Blahut-Arimoto (BA)-type algorithms that can compute the capacity without discretizing the channel. First, by exploiting projection and an approximation of the Lagrange multiplier, which are introduced to handle certain positive semidefinite constraints in the optimization formulation, we develop the Gaussian BA algorithm with projection (GBA-P). Then, we demonstrate that one of the subproblems arising from the alternating updates admits a closed-form solution. Based on this result, we propose the Gaussian BA algorithm with alternating updates (GBA-A) and establish its convergence guarantee. Furthermore, we extend the GBA-P algorithm to compute the capacity region of the Gaussian vector BC with both private and common messages. All the proposed algorithms are parameter-free. Lastly, we present numerical results to demonstrate the effectiveness of the proposed algorithms.

In this paper, we investigate the fundamental tradeoff between computation and communication for the full-duplex (FD) wireless MapReduce distributed computing network. Specifically, a coded interference alignment and neutralization (CIAN) scheme is proposed to significantly reduce the achievable normalized delivery time (NDT) for any given computation load, which jointly exploits both the coding and interference management technologies. In particular, a novel coding strategy is designed to create the coded message desired by multiple nodes, thereby providing the coded multicasting gain. Furthermore, the Shuffle phase is molded as a special cooperative X-multicast network. For this network, a novel IAN scheme is proposed to improve the achievable sum degree of freedom (SDoF), thereby providing the IAN gain. In the proposed CIAN scheme, the fundamental tradeoff between the coded multicasting gain and IAN gain is characterized, and the achievable NDT is minimized by carefully optimizing these two gains. Furthermore, a tight information-theoretic lower bound on the NDT is derived, demonstrating the optimality of the CIAN scheme in some cases. In other cases, the achievable NDT of the CIAN scheme and the lower bound are within a multiplicative gap of 2. Theoretical analysis and numerical results indicate the superior performance of the CIAN scheme compared to existing schemes, particularly by providing additional coded multicasting gain and improved IAN gain.

Bistatic sensing refers to scenarios where the transmitter (illuminating the target) and the sensing receiver (estimating the target state) are physically separated, in contrast to monostatic sensing, where both functions are co-located. In practical settings, bistatic sensing may be required either due to inherent system constraints or as a means to mitigate the strong self-interference encountered in monostatic configurations. A key practical challenge in bistatic radio-frequency radar systems is the synchronization and calibration of the separate transmitter and sensing receiver. In this paper, we are not concerned with these signal processing aspects and take a complementary information-theoretic perspective on bistatic integrated sensing and communication (ISAC). Namely, we aim to characterize the capacity-distortion function—the fundamental tradeoff between communication capacity and sensing accuracy. We consider a general discrete channel model for a bistatic ISAC system and derive a multi-letter representation of its capacity-distortion function. Then, we establish single-letter upper and lower bounds and provide exact single-letter characterizations for degraded bistatic ISAC channels. Numerical examples illustrate the theoretical results, highlighting the benefits of ISAC over separate communication and sensing, as well as the role of leveraging communication to assist sensing in bistatic systems.

McKean conjectured that Gaussian random variables are optimal for the$n\text{th}$order derivative of differential entropy along the heat flow, and verified this for$n=1,2$. Recently, Zhang, Anantharam and Geng introduced the linear matrix inequality approach to show that this conjecture holds for$n \leq 5$under the log-concavity assumption. In this work, with the same assumption, we improve their method using the positive semidefinite reformulation and validate McKean's conjecture for$n \leq 9$, and also the completely monotone conjecture for$n\leq 11$.

We design an algorithm from the perspective of information theory to calculate the capacity region of the Gaussian vector broadcast channel with private messages. For a continuous channel, a common method to approximately calculate its capacity is to apply the Blahut-Arimoto algorithm after discretization. In this work, we derive an equivalent form of the objective function and decouple the coupled variables in the original problem by exploiting the property that a Gaussian distribution is uniquely determined by its mean and variance. And thus develop a Gaussian Blahut-Arimoto algorithm without discretization.

This paper begins a systematic study of the zeroerror capacity problem of broadcast channels, where the message sent can be decoded by each receiver with zero error. A graph set is used to represent the broadcast channel. We particularly consider a set of two graphs, where each graph in the set contains only one edge. The corresponding zero-error capacity is determined in this paper.

Recently, it was shown that the Fisher information is log-convex along the heat flow. The main tools involved were the Cauchy-Schwarz inequality and clever observations on the sum of squares. In this work, we reformulate their method as a rank-one constrained semidefinite programming problem. Then we show that the rank-one matrix can be determined through investigating the first diagonal entry. We apply this new approach to recover existing results, as well as to obtain new results on the derivative of the negative Fisher information along the heat flow: the derivative is convex if it is raised to the power of three-eighths, and is log-convex if the input distribution is log-concave.

We consider a model of broadcast channel where a pair of base-user operates in the full-duplex mode. A partial decode-forward strategy together with Marton's coding are adopted to obtain an inner bound. An outer bound is also presented. Numerical evaluations are performed on a particular set of discrete memoryless channels to compare the sum-rates of these two bounds and the one of time division duplex.

Bistatic Integrated Sensing and Communication (ISAC) systems circumvent the issue of strong self-interference present in monostatic ISAC systems by employing a pair of physically separated sensing transceivers. They maintain the advantage of co-designing radar sensing and communications on shared spectrum and hardware. Motivated by the favorable attributes of bistatic radar, this paper investigates bistatic ISAC. In this setup, a transmitter sends messages to a communication receiver, while a sensing receiver at another location conducts a “decoding-and-estimation” (DnE) operation to obtain the state of the communication receiver. We propose three achievable DnE strategies based on the degree of information decoding at the sensing receiver: blind estimation, partial decoding-based estimation, and full decoding-based estimation. We explore the corresponding rate-distortion regions associated with each strategy. Furthermore, we provide a specific example to illustrate the comparison of the rate-distortion regions among the three DnE strategies and demonstrate the advantage of ISAC over independent communication and sensing.

In this paper, for a general full-duplex wireless MapReduce distributed computing network, we investigate the minimization of the communication overhead for a given computation overhead. The wireless MapReduce framework consists of three phases: Map phase, Shuffle phase and Reduce phase. Specifically, we model the Shuffle phase into a cooperative X network based on a more general file assignment strategy. Furthermore, for this cooperative X network, we derive an information-theoretic upper bound on the sum degree of freedom (SDoF). Moreover, we propose a joint interference alignment and neutralization (IAN) scheme to characterize the achievable SDoF. Especially, in some cases, the achievable SDoF coincides with the upper bound on the SDoF, hence, the IAN scheme provides the optimal SDoF. Finally, based on the SDoF, we present an information-theoretic lower bound on the normalized delivery time (NDT) and achievable NDT of the wireless distributed computing network, which are less than or equal to those of the existing networks. The lower bound on the NDT shows that 1) there is a tradeoff between the computation load and the NDT; 2) the achievable NDT is optimal in some cases, hence, the proposed IAN scheme can reduce the communication overhead effectively.

The Blahut-Arimoto algorithm was recently extended to computing inner and outer bounds for broadcast channels through the exchange of max-min. However, the convergence analysis is still limited, especially for the minimization part. In this work, we first simplify the algorithm for the superposition coding region, then extend the algorithm to the general supporting hyperplanes of Marton’s inner bound, and finally provide a detailed treatment of the convergence analysis. Numerical experiments on the superposition coding region and Marton’s inner bound validate the effectiveness of our algorithms.

The computation of inner and outer bounds on capacity regions of broadcast channels is difficult due to the non-convexity of expressions. In this work, with the help of a Terkelsen-type minimax theorem, we develop a Blahut-Arimoto algorithm to evaluate the supporting hyperplanes of the superposition coding region. Then we extend the algorithm to calculate the sum-rate of Marton’s inner bound, and the supporting hyperplanes of UV outer bound.

We propose a new learning paradigm, Local to Global Learning (LGL), for Deep Neural Networks (DNNs) to improve the performance of classification problems. The core of LGL is to learn a DNN model from fewer categories (local) to more categories (global) gradually within the entire training set. LGL is most related to the Self-Paced Learning (SPL) algorithm but its formulation is different from SPL. SPL trains its data from simple to complex, while LGL from local to global. In this paper, we incorporate the idea of LGL into the learning objective of DNNs and explain why LGL works better from an information-theoretic perspective. Experiments on the toy data, CIFAR-10, CIFAR-100, and ImageNet dataset show that LGL outperforms the baseline and SPL-based algorithms.

In this paper, we consider the coded-caching broadcast network with user cooperation, where a server connects with multiple users and the users can cooperate with each other through a cooperation network. We propose a centralized coded caching scheme based on a new deterministic placement strategy and a parallel delivery strategy. It is shown that the new scheme optimally allocate the communication loads on the server and users, obtaining cooperation gain and parallel gain that greatly reduces the transmission delay. Furthermore, we show that the number of users who parallelly send information should decrease when the users' caching size increases. In other words, letting more users parallelly send information could be harmful. Finally, we derive a constant multiplicative gap between the lower bound and upper bound on the transmission delay, which proves that our scheme is order optimal.

Let Z be a standard Gaussian random variable, X be independent of Z, and t be a strictly positive scalar. For the derivatives in t of the differential entropy of X+√tZ, McKean noticed that Gaussian X achieves the extreme for the first and second derivatives, and he conjectured that this holds for general orders of derivatives. Here we show that, when the probability density function of X+√tZ is log-concave, this conjecture holds for orders up to at least five. We also recover Toscani's result on the non-negativity of the third derivative of the entropy power of X+√tZ for log-concave densities, using a much simpler argument.

Let X be an arbitrary continuous random variable and Z be an independent Gaussian random variable with zero mean and unit variance. For t > 0, Costa proved that e2h(X+√t Z)is concave in t, where the proof hinged on the first and second order derivatives of h(X + √t Z). In particular, these two derivatives are signed, i.e., (∂/∂t)h(X + √tZ) ≥ 0 and (∂2/∂t2)h(X + √tZ) ≤ 0. In this paper, we show that the third order derivative of h(X + √tZ) is nonnegative, which implies that the Fisher information J(X + √tZ) is convex in t. We further show that the fourth order derivative of h(X +√tZ) is nonpositive. Following the first four derivatives, we make two conjectures on h(X +√tZ): the first is that (∂n/∂tn)h(X +√tZ) is nonnegative in t if n is odd, and nonpositive otherwise; the second is that log J(X + √tZ) is convex in t. The first conjecture can be rephrased in the context of completely monotone functions: J(X + √tZ) is completely monotone in t. The history of the first conjecture may date back to a problem in mathematical physics studied by McKean in 1966. Apart from these results, we provide a geometrical interpretation to the covariance-preserving transformation and study the concavity of h(√t X +√1 - t Z), revealing its connection with Costa's entropy power inequality.

Marton's inner bound is the tightest known inner bound on the capacity region of the broadcast channel. It is not known, however, if this bound is tight in general. One approach to settle this key open problem in network information theory is to investigate the multiletter extension of Marton's bound, which is known to be tight in general. This approach has become feasible only recently through the development of a new method for bounding cardinalities of auxiliary random variables by Gohari and Anantharam. This paper undertakes this long overdue approach to establish several new results, including 1) establishing the optimality of Marton's bound for new classes of product broadcast channels, 2) showing that the best-known outer bound by Nair and El Gamal is not tight in general, and 3) finding sufficient conditions for a global maximizer of Marton's bound that imply that the 2-letter extension does not increase the achievable rate. Motivated by the new capacity results, we establish a new outer bound on the capacity region of product broadcast channels.

A novel method for establishing the optimality of Gaussian auxiliary random variables in multiterminal information theory problems is developed. This method is then employed to show that Marton's inner bound achieves the capacity region of the two-receiver Gaussian vector broadcast channel with private and common messages.

We establish an information inequality concerning five random variables. This inequality is motivated by the sum-rate evaluation of Marton's inner bound for two receiver broadcast channels with a binary input alphabet. We establish that randomized time-division strategy achieves the sum rate of Marton's inner bound for all binary input broadcast channels. We also obtain an improved cardinality bound for evaluating the maximum sum rate given by Marton's inner bound for all broadcast channels. Using these tools we explicitly evaluate the inner and outer bounds for the binary skew-symmetric broadcast channel and demonstrate a gap between the bounds.

We establish capacity regions for some classes of broadcast channels with binary inputs and symmetric outputs. We investigate the more capable partial order and establish that the binary erasure channel and the binary symmetric channel form the two extremes for channels having the same capacity. Further, we apply the results to identify a class of broadcast channels for which the best-known inner and outer bounds on the capacity region differ.

We develop a new method for showing the optimality of the Gaussian distribution in multiterminal information theory problems. As an application of this method we show that Marton's inner bound achieves the capacity of the vector Gaussian broadcast channels with common message.

We establish a new outer bound for the capacity region of product broadcast channels. This outer bound matches Marton's inner bound for a variety of classes of product broadcast channels whose capacity regions were previously unknown. These classes include product of reversely semi-deterministic and product of reversely more-capable channels. A significant consequence of this new outer bound is that it establishes, via an example, that the previously best known outer-bound is strictly suboptimal for the general broadcast channel. Our example is comprised of a product broadcast channel with two semi-deterministic components in reverse orientation.

We study the capacity regions of broadcast channels with binary inputs and symmetric outputs. We study the partial order induced by the more capable ordering of broadcast channels for channels belonging to this class. In particular this leads to some surprising connections regarding various notions of dominance of receivers. This study also helps us isolate some classes of symmetric channels where the best known inner and outer bounds differ.

We establish an information inequality that is intimately connected to the evaluation of the sum rate given by Marton's inner bound for two-receiver broadcast channels with a binary input alphabet. This generalizes a recent result where the inequality was established for a particular channel, the binary skew-symmetric broadcast channel. The inequality implies that randomized time-division strategy indeed achieves the sum rate of Marton's inner bound for all binary input broadcast channels.