The problems of Lasso regression and optimal design of experiments share a critical property: their optimal solutions are typically sparse, i.e., only a small fraction of the optimal variables are non-zero. Therefore, the identification of the support of an optimal solution reduces the dimensionality of the problem and can yield a substantial simplification of the calculations. It has recently been shown that linear regression with a squared $\ell_1$-norm sparsity-inducing penalty is equivalent to an optimal experimental design problem. In this work, we use this equivalence to derive safe screening rules that can be used to discard inessential samples. Compared to previously existing rules, the new tests are much faster to compute, especially for problems involving a parameter space of high dimension, and can be used dynamically within any iterative solver, with negligible computational overhead. Moreover, we show how an existing homotopy algorithm to compute the regularization path of the lasso method can be reparametrized with respect to the squared $\ell_1$-penalty. This allows the computation of a Bayes $c$-optimal design in a finite number of steps and can be several orders of magnitude faster than standard first-order algorithms. The efficiency of the new screening rules and of the homotopy algorithm are demonstrated on different examples based on real data.

We focus on motion estimation using a block matching approach and suggest using a minimum-entropy criterion. Many entropy-based estimation procedures exist, such as plug-in estimators based on Parzen windowing. We consider here an alternative that is applicable to data of any dimension and that circumvents the critical issues raised by kernel-based methods. To the best of our knowledge, this criterion has not yet been considered for image processing problems. The inherent robustness property of entropy is expected to provide a robust and efficient estimation of the motion vector of a block of a video sequence. In particular, the minimum-entropy estimator should be robust to occlusions and variations of luminance, for which standard approaches like SSD usually meet their limitations.

Stochastic gradient algorithms are widely used in signal processing. Whereas stopping rules for deterministic descent algorithms can easily be constructed, using for instance the norm of the gradient of the objective function, the situation is more complicated for stochastic methods since the gradient needs first to be estimated. We show how a simple Kalman filter can be used to estimate the gradient, with some associated confidence, and thus construct a stopping rule for the algorithm. The construction is illustrated by a simple example. The filter might also be used to estimate the Hessian, which would open the way to a possible acceleration of the algorithm. Such developments are briefly discussed.

The paper is a continuation of earlier work (Pronzato and Thierry, Proc. 20th Int. Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, p.169-80, 2001; Proc. ICASSP, 2001): we estimate parameters in a regression model, linear or not, by minimizing (an estimate of) the entropy of the symmetrized residuals, obtained by a kernel estimation of their distribution. The objective is to obtain efficiency in the absence of knowledge of the density, f, of the observation errors, which is called adaptive estimation (Stein, C., 1956; Stone, C.J., 1975; Bickel, P.J., 1982;. Manski, C.F, 1984). Connections and differences with previous work are indicated. Numerical results illustrate that asymptotic efficiency is not necessarily in conflict with robustness.

We consider the situation where the parameters /spl theta/ of a linear regression model have to be estimated from observations corrupted by an additive noise with unknown distribution f. Since maximum likelihood estimation cannot be used, we estimate /spl theta/ by minimizing the entropy of a kernel estimate of f, constructed from the residuals. An example of parameter estimation in the presence of interference with random binary signals is presented.

Prediction by kriging does not rely on any specific model structure, and is thus much more flexible than approaches based on parametric behavioural models. Since accurate predictions are obtained for extremely short training sequences, it generally performs better than prediction methods using parametric models. Application to nonlinear system inversion is considered.

We consider the situation where n items have to be selected among a series of N presented sequentially, the information contained in each item being random. The problem is to get a collection of n items with maximal information. We consider the case where the information is additive, and thus need to maximize the sum of n independently identically distributed random variables x/sub k/ observed sequentially in a sequence of length N. This is a stochastic dynamic-programming problem, the optimal solution of which is derived when the distribution of the x/sub k/s is known. The asymptotic behaviour of this optimal solution (when N tends to infinity with n fixed) is considered. A (forced) certainty-equivalence policy is proposed for the case where the distribution is unknown and estimated on-line.

This article presents a new approach to blind equalization of an FIR channel. It is based on a bounded-error assumption and takes into account the fact that the input signal is in a finite alphabet. We show that even in the noisy case, identifiability can be guaranteed in finite time, provided that the support of the noise density is suitably bounded.

In a paper recently published (ibid., vol.12, p. 30-8, 1993), Chiang and Sullivan compare a new criterion for image registration called CBC (coincident bit counting) with two criteria that the authors proposed some years ago, namely SSC and DSC (stochastic and deterministic sign change criteria). The authors' nonparametric approach was demonstrated to outperform the conventional image registration criteria for robust registration in the fact that the value of their similarity measure did not take the specific pixel values into account. In light of this observation, Chiang and Sullivan have built the CBC criterion. The CBC method compares the number of coincident bits between the corresponding pixels in two different frames for a fixed amount of displacement. While the authors consider that the CBC is of interest and deserves to be studied, they feel that the comparison made by Chiang and Sullivan was not entirely accurate. Here the authors comment on this comparison and suggest possible further studies.