Investigation of a relationship between the mutual information and variational distance, started in Pinsker paper [1], where an upper bound for the mutual information via variational distance was obtained, is here continued. We present a simple lower bound which is optimal or asymptotically optimal in some cases. An uniform upper bound for the mutual information via variational distance is also derived for random variables with a finite number of values. For such random variables, the asymptotic behaviour of the maximum of mutual information is also investigated in the case where the variational distance tends to zero or to its maximum value.

The thinnest coverings of ellipsoids are studied in the Euclidean spaces of an arbitrary dimension n. Given any ellipsoid, we obtain a tight asymptotic bound on the minimum size of its covering by the balls of radius /spl epsi/. This bound holds for all but the most oblong ellipsoids. The results can be applied to vector quantization when different data streams are bundled together in one block.

The thinnest coverings of ellipsoids are studied in the Euclidean spaces of an arbitrary dimension n. Given any ellipsoid, the main goal is to find its /spl epsiv/-entropy, which is the logarithm of the minimum number of the balls of radius /spl epsiv/ needed to cover this ellipsoid. A tight asymptotic bound on the /spl epsiv/-entropy is obtained for all but the most oblong ellipsoids, which have very high eccentricity. This bound depends only on the volume of the sub-ellipsoid spanned over all the axes of the original ellipsoid, whose length (diameter) exceeds 2/spl epsiv/. The results can be applied to vector quantization performed when data streams from different sources are bundled together in one block.

A formula for the second-order expansion of the input-output mutual information of multidimensional channels as the signal-to-noise ratio (SNR) goes to zero is obtained. While the additive noise is assumed to be Gaussian, we deal with very general classes of input and channel distributions. As special cases, these channel models include fading channels, channels with random parameters, and channels with almost Gaussian noise. When the channel is unknown at the receiver, the second term in the asymptotic expansion depends not only on the covariance matrix of the input signal but also on the fourth mixed moments of its components. The study of the second-order asymptotics of mutual information finds application in the analysis of the bandwidth-power tradeoff achieved by various signaling strategies in the wideband regime.

In some channels subject to crosstalk or other types of additive interference, the noise is the sum of a dominant Gaussian noise and a relatively weak non Gaussian contaminating noise. Although the capacity of such channels cannot be evaluated in general, the authors analyze the decrease in capacity, or sensitivity of the channel capacity to the weak contaminating noise. The main result is that for a very large class of contaminating noise processes, explicit expressions for the sensitivity of a discrete-time channel capacity do exist. Moreover, in those cases the sensitivity depends on the contaminating process distribution only through its autocorrelation function and so it coincides with the sensitivity with respect to a Gaussian contaminating noise with the same autocorrelation function.

An asymptotic expression is derived for the Shannon mutual information between the input and output signals for a relatively large class of continuous alphabet memoryless channels in the case of weak input signals, when the input space is multidimensional. The authors extend a result of Ibragimov and Khas'minskii (1972) from the one-dimensional to the N-dimensional case. The asymptotic expression obtained relates the Shannon (1948) mutual information function and the Fisher information matrix. This expression is used to derive an asymptotic expression for the capacity of continuous alphabet memoryless channels with vector-valued weak input signals. This asymptotic capacity involves the largest eigenvalue of the Fisher information matrix evaluated at the zero input signal.>

Necessary and sufficient conditions are derived for the transmission of two arbitrarily correlated sources over a discrete memoryless asymmetric multiple-access channel. It is shown that in this situation the classical separation principle of Shannon (the factorization of the joint source-channel transmission problem into separate source and channel coding problems) applies. This asymmetric case is the first non-trivial situation of a multiple-access channel with arbitrarily correlated sources in which the sufficient conditions found for the reliable transmission of the sources over the channel turn out to be necessary as well. Furthermore, it is demonstrated that these necessary and sufficient conditions continue to hold if feedback is available to one or both of the encoders.