Two nonparametric methods to estimate the Bayes risk using classified sample sets are described and compared. The first method uses the nearest neighbor error rate as an estimate to bound the Bayes risk. The second method estimates the Bayes decision regions by applying Parzen probability-density function estimates and counts errors made using these regions. This estimate is shown to be asymptotically consistent in mean square. The results of experiments with these estimators on simulated and empirical data imply that the estimators both have acceptable small-sample properties; however, small-sample convergence of both estimators depends strongly on the choice of metric and local area or window size in the measurement space.

An important problem in pattern recognition or signal detection is the recognition of a pattern that is completely characterized statistically except for a finite set of unknown parameters. If a machine is required to solve such a problem on a number of occasions, it is possible to take advantage of this repetition. One can design a machine that will extract more and more of the pertinent information about these unknown parameters as it recognizes the patterns and readjusts itself to be more selective to them; the machine improves in performance as it gains experience on the problem. This paper presents a model suitable for many such problems and evolves a solution in the form of a machine that "learns" to solve the problem without external aid. Such machines are said to "learn without a teacher." The Bayes solution to the model problem requires the computation of the a posteriori probability density of the unknown parameters. A recursive equation for this density is derived. This equation describes the structure of a relatively simple system of finite size that may be realized in a delay-feedback form. The application of the model and the synthesis of a learning system are illustrated by the derivation of a receiver for the detection of signals of unknown amplitude in white Gaussian noise.