For the discrete-time additive white generalized Gaussian noise channel with a generalized input power constraint, with the respective shape and power parameters ≥ 1, we derive an upper bound on the optimal block error exponent. Explicit asymptotic upper bounds in the limit of a large block lengthnare given for three special cases: the Laplace noise channel and the Gaussian noise channel with the average absolute value constraint, and for the Laplace noise channel with the average squared value constraint. The derivation uses the method of types with finite alphabets of sizes depending on the block lengthnand with the number of types sub-exponential inn1.

For the discrete-time AWGN channel with a power constraint, we give an alternative derivation for the sphere-packing upper bound on the optimal block error exponent. The derivation uses the method of types with finite alphabets of sizes depending on the block length$n$and with the number of types sub-exponential in$n^{1}$.

For a discrete memoryless channel with finite input and output alphabets, we prove convergence of an iterative computation of the optimal correct-decoding exponent as a function of communication rate, for a fixed rate and for a fixed slope.

We consider a generalization of the discrete memoryless channel, in which the channel probability distribution is replaced by a uniform distribution over clouds of channel output sequences. For a random ensemble of such channels, we derive an achievable error exponent and a converse bound on the correct-decoding exponent. As a corollary of these results, we obtain the channel ensemble capacity.

We propose an on-line algorithm for adapting the input of an unknown or slowly varying channel, while keeping reliable communication at some fixed rate R during the adaptation process. The purpose of the algorithm is to push the generating distribution of an i.i.d. random code toward the input that achieves the channel capacity C. The algorithm uses one bit feedback per each transmission block, that acknowledges whether the decoded codeword crossed some pre-determined threshold T > R, with respect to some “fitness” metric. In the rare event of threshold crossing, the encoder and decoder update the input distribution according to the type of the current codeword, while the decoder updates the fitness metric. We show that for a large block length, this algorithm simulates computation of the channel correct-decoding exponent, and it leads to the capacity-achieving input if we set T = C.

We propose a source/channel duality in the exponential regime, where success/failure in source coding parallels error/correctness in channel coding, and a distortion constraint becomes a log-likelihood ratio (LLR) threshold. We establish this duality by first deriving exact exponents for lossy coding of a memoryless source P, at distortion D, for a general i.i.d. codebook distribution Q, for both encoding success (RR(P, Q, D)). We then turn to maximum likelihood (ML) decoding over a memoryless channel P with an i.i.d. input Q, and show that if we substitute P = QP, Q = Q, and D = 0 under the LLR distortion measure, then the exact exponents for decoding-error (RI(Q, P)) follow as special cases of the exponents for source encoding success/failure, respectively. Moreover, by letting the threshold D take general values, the exact random-coding exponents for erasure (D > 0) and list decoding (D1.

The Arimoto algorithm computes the Gallager function maxQE0(ρ, Q) for a given channel P (y | x) and parameter ρ, by means of alternating maximization. Along the way, it generates a sequence of input distributions Q1(x), Q2(x), ..., that converges to the maximizing input Q*(x). We propose a stochastic interpretation for the Arimoto algorithm. We show that for a random (i.i.d.) codebook with a distribution Qk(x), the next distribution Qk+1(x) in the Arimoto algorithm is equal to the type (Q') of the feasible transmitted codeword that maximizes the conditional Gallager exponent (conditioned on a specific transmitted codeword type Q'). This interpretation is a first step toward finding a stochastic mechanism for on-line channel input adaptation.

Shannon's capacity and rate-distortion function, combined with the separation principle, provide tight bounds for the minimum possible distortion in joint source-channel coding. These bounds, however, are usually achievable only in the limit of a large block length. In their 1973 paper, Ziv and Zakai introduced a family of alternative capacity and rate-distortion functions, based on functionals satisfying the data-processing inequality, which potentially give tighter bounds for systems with a small block length. There is a considerable freedom as to how to choose those functionals, and the ways of finding the best possible functionals yielding the best bounds for a given source-channel combination are not specified. We examine recently conjectured high SNR asymptotic expressions for the Ziv-Zakai bounds, based on the Rényi-divergence functional. We derive nonasymptotic bounds on the Ziv-Zakai-Rényi rate-distortion function and capacity for a broad class of sources and additive noise channels, which hold for arbitrary SNR and prove the conjectured asymptotic expressions in the limit of a small distortion/high SNR. The results lead to new bounds on the best achievable distortion in finite dimensional joint source-channel coding. Examples are presented where the new bounds achieve significant improvement upon Shannon's original bounds.

Shannon's capacity and rate-distortion function, combined with the separation principle, provide tight bounds for the minimum possible distortion in joint source-channel coding. These bounds, however, are usually achievable only in the limit of large block length. In their 1973 paper, Ziv and Zakai provide a family of alternative capacity and rate-distortion functions, based on functionals satisfying the data-processing inequality, which potentially give tighter bounds for systems with a small block length, e.g., for scalar modulation. We examine a recently proposed approximation for the Ziv-Zakai bounds based on the Rényi-divergence functional. For the specific case of a uniform source, we derive explicit bounds on the Ziv-Zakai-Rényi rate- distortion function, which prove this approximation in the limit of small distortion. Our results can be extended, using the same technique, to more general sources.