This work considers a binomial noise channel. The paper can be roughly divided into two parts. The first part is concerned with the properties of the capacity-achieving distribution. In particular, for the binomial channel, it is not known if the capacity-achieving distribution is unique since the output space is finite (i.e., supported on integers 0, …, n) and the input space is infinite (i.e., supported on the interval [0, 1 D, and there are multiple distributions that induce the same output distribution. This paper shows that the capacity-achieving input distribution is unique by appealing to the total positivity property of the binomial kernel. In addition, we provide upper and lower bounds on the cardinality of the support of the capacity-achieving distribution. Specifically, an upper bound of order n/2 is shown, which improves on the previous upper bound of order$n$due to Witsenhausen. Moreover, a lower bound of order yin is shown. Finally, additional results about the locations and probability values of the support points are established.

Consider a pair of random vectors$( {\mathbf{X}}, {\mathbf{Y}}) $and the conditional expectation operator$ \mathbb {E}[ {\mathbf{X}}| {\mathbf{Y}}={\mathbf{y}}]$. This work studies analytical properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain$ {\mathbf{U}}\leftrightarrow {\mathbf{X}}\leftrightarrow {\mathbf{Y}}$, a compact expression for the Jacobian matrix of$ \mathbb {E}[ \psi ( {\mathbf{Y}}, {\mathbf{U}})| {\mathbf{Y}}= {\mathbf{y}}]$for a smooth function$\psi $is derived. In the second part of the paper, the main identity is specialized to the exponential family and two main applications are shown. First, it is demonstrated that, via various choices of the random vector$ {\mathbf{U}}$and function$\psi $, one can recover and generalize several known identities (e.g., Tweedie’s formula) and derive some new ones. For example, a new relationship between conditional expectations and conditional cumulants is established. Second, it is demonstrated how the derivative identities can be used to establish new lower bounds on the estimation error. More specifically, using one of the derivative identities in conjunction with a Poincaré inequality, a new lower bound on the minimum mean squared error, which holds for all prior distributions on the input signal, is derived. The new lower bound is shown to be tight in the high-noise regime for the additive Gaussian noise setting.

This paper studies the minimum mean squared error (MMSE) of estimating X ∈ ℝdfrom the noisy observation Y ∈ ℝk, under the assumption that the noise (i.e., Y|X) is a member of the exponential family. The paper provides a new lower bound on the MMSE. Towards this end, an alternative representation of the MMSE is first presented, which is argued to be useful in deriving closed-form expressions for the MMSE. This new representation is then used together with the Poincaré inequality to provide a new lower bound on the MMSE. Unlike, for example, the Cramér-Rao bound, the new bound holds for all possible distributions on the input X. Moreover, the lower bound is shown to be tight in the high-noise regime for the Gaussian noise setting under the assumption that X is sub-Gaussian. Finally, several numerical examples are shown which demonstrate that the bound performs well in all noise regimes.

The Ziv-Zakai bound is a well-known lower bound on the minimum mean squared error. This article analyzes the performance of this bound in the practically relevant high-noise regime for a broad family of observation models. The goal is to understand whether this bound is tight, and in which scenarios it should be used. It is shown that, while the Ziv-Zakai bound is tight for a certain class of symmetric distributions, in general, it isnottight in the high-noise regime.