We pose the problem of approximating optimally a given nonnegative signal with the scalar autoconvolution of a nonnegative signal. The I-divergence is chosen as the optimality criterion being well suited to incorporate nonnegativity constraints. After proving the existence of an optimal approximation, we derive an iterative descent algorithm of the alternating minimization type to find a minimizer. The algorithm is based on the lifting technique developed by Csiszár and Tusnádi and exploits the optimality properties of the related minimization problems in the larger space. We study the asymptotic behavior of the iterative algorithm and prove, among other results, that its limit points are Kuhn-Tucker points of the original minimization problem. Numerical experiments confirm the asymptotic results and exhibit the fast convergence of the proposed algorithm.

We pose the deterministic, nonparametric, approximation problem for scalar nonnegative input/output systems via finite impulse response convolutions, based on repeated observations of input/output signal pairs. The problem is converted into a nonnegative matrix factorization with special structure for which we useCsiszár's I-divergenceas the criterion of optimality. Conditions are given, on the input/output data, that guarantee the existence and uniqueness of the minimum. We propose an algorithm of the alternating minimization type for I-divergence minimization, and study its asymptotic behavior. For the case of noisy observations, we give the large sample properties of the statistical version of the minimization problem. Numerical experiments confirm the asymptotic results and exhibit the fast convergence of the proposed algorithm.

In this paper we consider the approximate realization problem for finite valued hidden Markov models i.e. stochastic processes Y=f(X) where X is a finite state Markov chain and f a many-to-one function. Given the laws p/sub Y/(/spl middot/) of Y the weak realization problem consists in finding a Markov chain X and a function f such that, at least distributionally, Y/spl sim/f(X). The approximate realization problem consists in finding X and f such that Y and f (X) are close. The approximation criterion we use is the informational divergence between properly defined. nonnegative (componentwise) matrices related to the processes. To construct the realization we apply recent results on the approximate factorization of nonnegative matrices.

We consider the problem of estimating the order of a stationary ergodic Markov chain. Our focus is on estimators which satisfy a generalized Neyman-Pearson criterion of optimality. Specifically, the optimal estimator minimizes the probability of underestimation among all estimators with probability of overestimation not exceeding a given value. Our main result identifies the best exponent of asymptotically exponential decay of the probability of underestimation. We further construct a consistent estimator, based on Kullback-Leibler divergences, which achieves the best exponent. We also present a consistent estimator involving a recursively computable statistic based on appropriate mixture distributions; this estimator also achieves the best exponent for underestimation probability.