In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.

The Slepian-Wolf scheme for source coding with side information at the receiver, assures that the sender can send the source X at a rate of only the conditional entropy H(X|Y-) bits per source symbol, which is the minimal possible rate even if the sender knew the side information Y. However, the Slepian-Wolf result requires knowledge of the optimal required rate. In this paper we consider a situation where this rate is not known, possibly since the source is broadcasted to many heterogeneous receivers. The approach is based on recent results regarding sending a common information over a broadcast channel.

An improved upper bound on the error probability (first error event) of time-invariant convolutional codes, and the resulting error exponent, is derived. The improved error bound depends on both the delay of the code K and its width (the number of symbols that enter the delay line in parallel) b. Determining the error exponent of time-invariant convolutional codes is an open problem. While the previously known bounds on the error probability of time-invariant codes led to the block-coding exponent, we obtain a better error exponent (strictly better for b>1). In the limit b/spl rarr//spl infin/ our error exponent equals the Yudkin-Viterbi (1967, 1971, 1965) exponent derived for time-variant convolutional codes. These results are also used to derive an improved error exponent for periodically time-variant codes.

This work provides techniques to apply the channel coding theorem and the resulting error exponent, which was originally derived for totally random block-code ensembles, to ensembles of codes with less restrictive randomness demands. As an example, the random coding technique can even be applied for an ensemble that contains a single code. For a specific linear code, we get an upper bound for the error probability, which equals Gallager's (1968) random coding bound, up to a factor determined by the maximum ratio between the weight distribution of the code, and the expected random weight distribution.