Feature extraction is considered as a mean-quare estimation of the Bayes risk vector. The problem is simplified by partitioning the distribution space into local subregions and performing a linear estimation in each subregion. A modified clustering algorithm is used to fimd the partitioning which minimizes the mean-square error.

A random frequency-varying signal is approximated by a finite-parameter model. A maximum {\em a posteriori} estimate of the parameter set leads to an estimation criterion, which when maximized is also the decision statistic obtained with an estimator-correlator detector. An efficient implementation of the frequency-parameter estimate (which corresponds to maximizing the criterion) is presented. The resulting estimate is compared to previous techniques found in the literature. Finally, performance bounds for the resulting detector are evaluated and compared to the asymptotically optimum low-energy coherence detector.

A local distance measure is shown to optimize the performance of the nearest neighbor two-class classifier for a finite number of samples. The difference between the finite sample error and the asymptotic error is used as the criterion of improvement. This new distance measure is compared to the well-known Euclidean distance. An algorithm for practical implementation is introduced. This algorithm is shown to be computationally competitive with the present nearest neighbor procedures and is illustrated experimentally. A closed form for the corresponding second-order moment of this criterion is found. Finally, the above results are extended to

Feature extraction criteria of the formf(D_{l}, \cdots ,D_{M},\Sigma_{1}, \cdots , \Sigma_{M})are considered whereD_{i}and\Sigma_{i}are the conditional means and covariances. The functionfis assumed to be invariant under nonsingular linear transformations and coordinate shifts. For the casef(D_{1},D_{2},\Sigma_{o}),fis shown to depend only upon the distance(D_{2}-D_{1})^{T} \Sigma_{o}^{-1}(D_{2}-D_{1})between classes. TheMclass case,f(D_{1}, \cdots ,D_{M}, \Sigma_{o}), is shown to depend upon the(M-1)M/2between-class distances(D_{j}-D_{k})^{T} \Sigma^{-1}_{O}(D_{j}-D_{k}). This criterion is also shown to be equivalent to the mean-square-error of the general Bayes risk estimate. The most generalfis reduced to a function ofMsets of between-class distances with metrics induced by the conditional covariances. WhenM=2this dependence is reduced to two parameters which may be regarded as different between-class distance measures. Finally, the linear mapping is reduced to a one-parameter problem as in the work of Peterson and Mattson.

The general procedure of conventional clustering is modified for various applications involving problem localization. This modification introduces the concept of clustering criteria which are used for partitioning a training set, and depend upon certain a priori information with regards to the training set. Also, the need of a structure or a mathematical form for the partition boundaries arises naturally from the need to process unknown samples. The general procedure is discussed in detail for applications of piecewise linear classifier design and piecewise linear density estimation. Experimental results are presented for both applications.

The optimal nonlinear features for a criterion function of the general formf(D_{l},\cdots,D_{M},K_{1},\cdots,K_{M})are studied, where theD_{j}and the are the conditional first- and second-order moments. The optimal solution is found to be a parametric function of the conditional densities. By imposing a further restriction on the functional dependence offon theK_{j}, the optimal mapping becomes an intuitively pleasing function of the posterior probabilities. Given a finite number of features\psi_{1}(X),\cdots ,\psi_{L}(X), the problem of finding the best linear mappings tomfeatures is next investigated. The resulting optimum mapping is a linear combination of the projections of the posterior probabilities onto the subspace spanned by the\psi_{j}(X). The problem of finding the best single feature and seqnential feature selection is discussed in this framework. Finally, several examples are discussed.