In lossless source coding, one-to-one codes refer to encodings without the prefix constraint. In this paper, we consider Campbell’s generalized expected length for such codes, i.e., the normalized cumulant generating function of codeword lengths. We show that the $\rho$-th order generalized expected length of any one-to-one code for a discrete random variable $X$ is at least \[H_{\frac{1}{1+\rho}}(X) - \log\big(H(X_{\frac{1}{1+\rho}}) + 1\big) - \log e, \] where $H_{\frac{1}{1+\rho}}(X)$ is the Rényi entropy of $X$, and $H(X_{\frac{1}{1+\rho}})$ is the Shannon entropy of the corresponding tilted (escort) distribution of order $1/(1+\rho)$. This result generalizes a bound by Alon and Orlitsky concerning the expected length, which is recovered by setting $\rho = 0$. Moreover, we show that the same bound also applies to the $\rho$-th guessing moment, yielding a lower bound that is valid for countably infinite supports.

We study the exponential strong converse for the Gelfand-Pinsker channel, i.e., the exponential speed at which the decoding error probability converges to 1 at rates above capacity. We establish the exact convergence speed, known as the strong converse exponent, by deriving matching upper and lower bounds. The upper bound is derived by applying and analyzing the likelihood decoder. The lower bound follows from single-letterizing a KL-divergence based on the weak converse.

The information bottleneck channel, also known as oblivious relaying, is a two-hop channel where a transmitter sends messages to a remote receiver via an intermediate relay node. A codeword sent by the transmitter passes through a discrete memoryless channel to reach the relay, which then processes the noisy channel output and forwards it to the receiver through a noiseless rate-limited link. The relay is oblivious, in the sense that it has no knowledge of the channel codebook used in transmission. Previous works on oblivious relaying focus on characterizing achievable rates. In this work, we study error exponents and explore connections to lossless source coding with a helper, also known as the Wyner-Ahlswede-Körner (WAK) problem. We first establish an achievable error exponent for oblivious relaying under constant compositions codes. A key feature of our analysis is the use of the type covering lemma to design the relay’s compress-forward scheme. We then show that employing constant composition code ensembles does not improve the rates achieved with their IID counterparts. We also derive a sphere packing upper bound for the error exponent. In the second part of this paper, we establish a connection between the information bottleneck channel and the WAK problem. We show that good codes for the latter can be produced through permuting codes designed for the former. This is accomplished by revisiting Ahlswede’s covering lemma, and extending it to achieve simultaneous covering of a type class by several distinct sets using the same sequence of permutations. We then apply our approach to attain the best known achievable error exponent for the WAK problem, previously established by Kelly and Wagner. As a byproduct of our derivations, we also establish error exponents and achievable rates under mismatched decoding rules.

We study the information bottleneck (IB) source coding problem, also known as remote lossy source coding under logarithmic loss. Based on a rate-limited description of noisy observations, the receiver produces a soft estimate for the remote source, i.e., a probability distribution, evaluated under the logarithmic loss. We focus on the excess distortion probability of IB source coding and investigate how fast it converges to 0 or 1, depending on whether the rate is above or below the rate-distortion function. The latter case is also known as the exponential strong converse. We establish both the exact error exponent and the exact strong converse exponent for IB source coding by deriving matching upper and lower exponential bounds. The obtained exponents involve optimizations over auxiliary random variables. The matching converse bounds are derived through non-trivial extensions of existing sphere packing and single-letterization techniques, which we adapt to incorporate auxiliary random variables. In the second part of this paper, we establish a code-level connection between IB source coding and source coding with a helper, also known as the Wyner-Ahlswede-Körner (WAK) problem. We show that every code for the WAK problem is a code for IB source coding. This requires noticing that IB source coding, under the excess distortion criterion, is equivalent to source coding with a helper available atboththe transmitterandthe receiver; the latter in turn relates to the WAK problem. Through this connection, we re-derive the best known sphere packing exponent of the WAK problem, and provide it with an operational interpretation.

Past works on remote lossy source coding studied the rate under average distortion and the error exponent of excess distortion probability. In this work, we look into how fast the excess distortion probability converges to 1 at small rates, also known as exponential strong converse. We characterize its exponent by establishing matched upper and lower bounds. From the exponent, we also recover two previous results on lossy source coding and biometric authentication.

We derive an achievable error exponent for the information bottleneck channel. The exponent is expressed as a minimum of two terms: a compression error exponent due to the bottleneck, and a channel decoding error exponent. Achievability is established through treating the rate-limited noiseless link between the relay and the receiver asymptotically as a virtual discrete memoryless channel.

We study a lossy variant of the Massy-Arikan guessing problem where instead of guessing the exact value of a discrete random variable, the goal is to guess a good soft reconstruction: a probability distribution under which the true realization has low uncertainty. The remaining uncertainty after guessing is measured through the logarithmic loss. We derive single-shot lower and upper bounds for the corresponding guessing moments. These bounds are exponentially tight in the asymptotic regime. Moreover, we establish a connection between our proposed soft guessing problem and the problem of variable-length lossy source coding under logarithmic loss.

We consider a basic communication and sensing setup comprising a transmitter, a receiver and a sensor. The transmitter sends an encoded sequence to the receiver through a discrete memoryless channel, and the receiver is interested in decoding the sequence. On the other hand, the sensor picks up a noisy version of the transmitted sequence through one of two possible discrete memoryless channels. The sensor knows the transmitted sequence and wishes to discriminate between the two possible channels, i.e. to identify the channel that has generated the output given the input. We study the trade-off between communication and sensing in the asymptotic regime, captured in terms of the coding rate to the receiver against the discrimination error exponent at the sensor. We characterize the optimal rate-exponent trade-off for general discrete memoryless channels with an input cost constraint.