This paper considers the model of an arbitrarily distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean-square error of the noncausal estimator and the likelihood ratio between y and w are derived. This is followed by an extended version of a recently derived relation between the mutual information I(x;y) and the minimal mean-square error. These results are applied to derive infinite-dimensional versions of the Fisher information and the de Bruijn identity. A comparison between the causal and noncausal estimation errors yields a restricted form of the logarithmic Sobolev inequality. The derivation of the results is based on the Malliavin calculus

New upper and lower bounds for the nonlinear filtering problem are presented. The lower bounds are especially useful in the region of small diffusion coefficients where previously known bounds are inefficient. The upper and lower bounds are shown to be tight. An example demonstrating the tightness of the bounds is presented.>

The minimal mean-square error for the scalar nonlinear filtering problem is considered. Asymptotic lower and upper bounds on the error are derived for the case where the intensity of the observation noise tends to zero.

The system modeldx_{t}=m(x_{t})dt+\sigma_{o}dw_{t}and the observationdy_{t}=h(x_{t})dt+\rho dv_{t}are considered, and a lower bound on the error for the least-square estimate of a function ofx_{t}, sayz(x_{t}), is derived. This bound is then applied to derive a lower bound on the error in estimatingx_{t}in the case where the constant\sigma_{o}in the equation for the system dynamics is replaced by\sigma (x_{t}).

A lower bound on the minimal mean-square error in estimating nonlinear diffusion processes is derived. The bound holds for causal and noncausal filtering.

An improved technique for bounding the mean-square error of signal parameter estimates is presented. The resulting bounds are independent of the bias and stronger than previously known bounds.

Let\xi = \{\xi(t), 0 \leq t \leq T\}be a process with covariance functionK(s,t)andE \int_0^T \xi^2(t) dt < \infty. It is proved that for every\varepsilon > 0the\varepsilon-entropyH_{\varepsilon}(\xi)satisfies \begin{equation} H_{\varepsilon}(\xi_g) - \mathcal{H}_{\xi_g} (\xi) \leq H_{\varepsilon}(\xi) \leq H_{\varepsilon}(\xi_g) \end{equation} where\xi_gis a Gaussian process with the covarianeeK(s,t)and\mathcal{H}_{\xi_g}(\xi)is the entropy of the measure induced by\xi(in function space) with respect to that induced by\xi_g. It is also shown that if\mathcal{H}_{\xi_g}(\xi) < \inftythen, as\varepsilon \rightarrow 0\begin{equation} H_{\varepsilon}(\xi) = H_{\varepsilon}(\xi_g) - \mathcal{H}_{\xi_g}(\xi) + o(1). \end{equation} Furthermore, ff there exists a Gaussian processg = \{ g(t); 0 \leq t \leq T \}such that\mathcal{H}_g(\xi) < \infty, then the ratio betweenH_{\varepsilon}(\xi)andH_{\varepsilon}(g)goes to one as\varepsilongoes to zero. Similar results are given for the rate-distortion function, and some particular examples are worked out in detail. Some cases for which\mathcal_{\xi_g}(\xi) = \inftyare discussed, and asymptotic bounds onH_{\varepsilon}(\xi), expressed in terms ofH_{\varepsilon}(\xi_g), are derived.

Upper and lower bounds on the\varepsilon-entropy of the Wiener process and theRCprocess are derived. The bounds are quite tight for small\varepsilon. The associated rate-distortion function is discussed and upper and lower bounds on this function are derived.

It is shown that the rate-distortion bound(R(d) \leq C)remains true when-\log xin the definition of mutual information is replaced by an arbitrary concave(\cup)nonincreasing function satisfying some technical conditions. Examples are given showing that for certain choices of the concave functions, the bounds obtained are better than the classical rate-distortion bounds.

The optimal nonlinear filtering of certain vector-valued diffusion processes embedded in white noise is considered. We derive upper and lower bounds on the minimal causal mean-square error. The derivation of the lower bound is based on information-theoretic considerations, namely the rate-distortion function (\varepsilon-entropy). The upper bounds are based on linear-filtering arguments. It is demonstrated that for a wide class of high-precision systems, the upper and lower bounds are tight within a factor of 2 or better.

The following model for the white Gaussian channel with or without feedback is considered: \begin{equation} Y(t) = \int_o ^{t} \phi (s, Y_o ^{s} ,m) ds + W(t) \end{equation} wheremdenotes the message,Y(t)denotes the channel output at timet,Y_o ^ {t}denotes the sample pathY(\theta), 0 \leq \theta \leq t. W(t)is the Brownian motion representing noise, and\phi(s, y_o ^ {s} ,m)is the channel input (modulator output). It is shown that, under some general assumptions, the amount of mutual informationI(Y_o ^{T} ,m)between the messagemand the output pathY_o ^ {T}is directly related to the mean-square causal filtering error of estimating\phi (t, Y_o ^{t} ,m)from the received dataY_o ^{T} , 0 \leq t \leq T. It follows, as a corollary to the result forI(Y_o ^ {T} ,m), that feedback can not increase the capacity of the nonband-limited additive white Gaussian noise channel.

Shannon showed that the capacity of a discrete memoryless channel can not be increased by noiseless feedback. It has been conjectured that this should be true for a continuous memoryless channel, provided such a channel is appropriately defined. We precisely define such a channel from two mathematically different points of view and rigorously prove that its capacity can not be increased by feedback.

The problem of estimating the position of a position-modulated rectangular pulse in additive white noise is considered. The maximum likelihood estimation procedure is assumed. Bounds are derived for the probability of large estimation errors.

New bounds are presented for the maximum accuracy with which parameters of signals imbedded in white noise can be estimated. The bounds are derived by comparing the estimation problem with related optimal detection problems. They are, with few exceptions, independent of the bias and include explicitly the dependence on the a priori interval. The new results are compared with previously known results.

LetY(\omega,\alpha)be the Wiener filter designed to yield an output which is the least-square approximation tos(t + \alpha)wheres(t)is the desired signal input. LetY_L(\omega)be the Wiener filter designed to yield an output which is the least-square approximation to some linear operationLon the desired input signal. The following simple relationship has been shown to hold betweenY(\omega, \alpha)andY_L(\omega). Ifs(t)is the desired input signal andL_{\alpha}[s(t + \alpha)is the desired output, whereL_{\alpha}is some linear operation with respect to\alpha, thenY_L(\omega) = L_{\alpha}[Y(\omega, \alpha)].