In this correspondence, we point out the incorrectness of an upper bound in van der Meulen (Advances in Applied Probability, vol. 3, no. 1, pp. 120-154, 1971) and report the correct relay capacity for one of the examples given there. We also show that, when viewed as a relay-without-delay channel, the capacity of this channel is strictly larger than its classical relay capacity.

This correspondence considers the class P of discrete bivariate distributions p with marginals a(p), b(p) and the subclass Pa,b /spl sub/ P of distributions with given marginals a and b. For an arbitrary q /spl isin/ P we study the distributions p/spl tilde/ /spl isin/ P/sub a,b/ minimizing on P/sub a,b/ various /spl phi/-divergences D/sub /spl phi//(p, q) for convex /spl phi/ with finite /spl phi/(0).

By combining techniques of Chang and Cantor and Mills, we construct signature codes for a T-user binary adder multiple access channel, when any subset of users can be active at any time, for new values of the parameters N (the codeword length) and T, different from the ones which can be obtained directly from the Cantor and Mills construction.

We investigate a nonparametric estimator of the probability density introduced by Barron (1988, 1989). Earlier papers established its consistency in a strong sense, e.g., in the expected information divergence or expected chi-square divergence. This paper pays main attention to the expected chi-square divergence criterion. We give a new motivation of the Barron estimator by showing that a maximum-likelihood estimator (MLE) of a density from a family important in practice is consistent in expected information divergence but not in expected chi-square divergence. We also present new and practically applicable conditions of consistency in the expected chi-square divergence. Main attention is paid to optimization (in the sense of the mentioned criterion) of the two objects specifying the Barron estimator: the dominating probability density and the decomposition of the observation space into finitely many bins. Both problems are explicitly solved under certain regularity assumptions about the estimated density. A simulation study illustrates the results in exponential, Rayleigh, and Weibull families.

By extending the information-theoretic arguments of previous papers dealing with the Barron-type density estimates, and their consistency in information divergence and chi-square divergence, the problem of consistency in Csiszar's /spl phi/-divergence is motivated for general convex functions /spl phi/. The problem of consistency in /spl phi/-divergence is solved for all /spl phi/ with /spl phi/(0)

Shows that a discrete infinite distribution with finite entropy cannot be estimated consistently in information divergence. As a corollary the authors show that there is no universal source code for an infinite source alphabet over the class of all discrete memoryless sources with finite entropy.>

An asymptotic expression is derived for the Shannon mutual information between the input and output signals for a relatively large class of continuous alphabet memoryless channels in the case of weak input signals, when the input space is multidimensional. The authors extend a result of Ibragimov and Khas'minskii (1972) from the one-dimensional to the N-dimensional case. The asymptotic expression obtained relates the Shannon (1948) mutual information function and the Fisher information matrix. This expression is used to derive an asymptotic expression for the capacity of continuous alphabet memoryless channels with vector-valued weak input signals. This asymptotic capacity involves the largest eigenvalue of the Fisher information matrix evaluated at the zero input signal.>

The problem of the nonparametric estimation of a probability distribution is considered from three viewpoints: the consistency in total variation, the consistency in information divergence, and consistency in reversed-order information divergence. These types of consistencies are relatively strong criteria of convergence, and a probability distribution cannot be consistently estimated in either type of convergence without any restrictions on the class of probability distributions allowed. Histogram-based estimators of distribution are presented which, under certain conditions, converge in total variation, in information divergence, and in reversed-order information divergence to the unknown probability distribution. Some a priori information about the true probability distribution is assumed in each case. As the concept of consistency in information divergence is stronger than that of convergence in total variation, additional assumptions are imposed in the cases of informational divergences.>

The deterministic relay channel is analyzed and explicit code constructions for all binary and all ternary/binary channels are given. An explicit set of equivalence conditions is used to make a classification of all such relay channels, for which also the capacity is evaluated. The coding problem is then reduced to finding all possible output sequences of a certain finite-state channel determined by the relay coding strategy. The channel states correspond to the possible relay memory contents. For some relay channels capacity is reached by using simple uniquely decodable codes, thus establishing the zero-error capacity of those channels with finite-memory relay strategies. For other relay channels the relay memory must be arbitrarily large to achieve zero-error rates arbitrarily close to capacity. One such code construction is given. It is not known whether there exist relay channels for which the zero-error capacity is strictly smaller than the average-error capacity. The code construction problem for the semideterministic relay channel and for the nonsynchronized relay channel is briefly considered.>

An inner bound to the capacity region for identification via a broadcast channel with feedback and a soft outer bound to the capacity region of a general broadcast channel without feedback are established, allowing randomized encoding in both cases. These results can be combined to determine the identification capacity region for the deterministic broadcast channel; the set of achievable second-order rate pairs (R/sub 1/,R/sub 2/) for the deterministic broadcast channel is given by the constraints 0>

Necessary and sufficient conditions are derived for the transmission of two arbitrarily correlated sources over a discrete memoryless asymmetric multiple-access channel. It is shown that in this situation the classical separation principle of Shannon (the factorization of the joint source-channel transmission problem into separate source and channel coding problems) applies. This asymmetric case is the first non-trivial situation of a multiple-access channel with arbitrarily correlated sources in which the sufficient conditions found for the reliable transmission of the sources over the channel turn out to be necessary as well. Furthermore, it is demonstrated that these necessary and sufficient conditions continue to hold if feedback is available to one or both of the encoders.

It is shown that for an asymmetric multiple-access channel, it is possible, starting with one codeword at each of the encoders, to add the codewords for one message at a time (first for the elements in the set of private messages and thereafter for the messages in the set of common messages) and to construct this way, iteratively, a code for this channel with the average error probability upper-bounded by an exponentially small quantity. Using this result, it is subsequently shown that a code for the asymmetric multiple-access channel can also be generated by only. a few (linear in the block length) permutations, still keeping the average error probability upper-bounded by the same error bound and in this way gaining storage space.

The capacity regions are determined for various communication situations in which one or both encoders for a multiple access channel crib from the other encoder and learn the channel input(s) (to be) emitted by this encoder. Most of the achievability proofs in this paper hinge upon the new concept of backward decoding. Also, the notion of Shannon strategies seems to be of crucial importance. It is demonstrated that in some situations parts of the total cooperation line are achievable. Moreover, it is proved that if the encoders and the decoder are allowed to be nondeterministic, the capacity regions are not increased.

The region Cover and Leung found for the discrete memoryless multiple access channel with feedback to both encoders is proved achievable also with feedback to only one encoder. The novel ideas of nonrandom partitions and restricted decoding are used to avoid list coding techniques.

A simple proof using random partitions and typicality is given for Marton's coding theorem for broadcast channels.

The advances in the area of multi-way channels during the period 1961-1976 are described. Shannon's two-way channel, the multiple-access channel, the interference channel, the broadcast channel, and the relay channel are treated successively. Only channel coding aspects are discussed. Thirty-nine open problems are mentioned.

Three different communication situations are considered for the general nondegraded discrete memoryless broadcast channel with two components. In the most general situation, common and separate information is sent to both receivers. In another situation, only separate information is sent, and in a third, one Common and one separate message is sent. For each communication situation a random coding inner bound on the capacity region is derived. An example is presented which Shows that in the most general situation the inner bound strictly dominates the family of rates obtained by time-sharing. The capacity region for the general situation is characterized by a limiting expression. The relationship with the degraded broadcast channel and the connection with other multiway channels, such as the channel with two senders and two receivers, is shown.