Self-organizing probability state variable (PSV) parameter search algorithms possessing long-term memory have been formulated to cope with systems that must avoid high performance-penalty operating regions. The information gained from all previous experiments is efficiently encoded in multivariate probability distribution functions (pdf's). This long-term memory capability enables the PSV algorithms to avoid effectively future experiments in high penalty regions. The systems considered are resource-limited, and catastrophic failure may occur if parameter values lying in high penalty regions are implemented. Those cases in which the high penalty regions are not known in advance were investigated. The PSV algorithms have the capability of adaptively learning the location and hypervolume of these regions as the search proceeds. The algorithms are explicitly guided in their internal strategies as a function of the remaining system resources and the updated probability distribution functions. Clustering analysis is used both in the discovery of new operating regions and for updating the pdf's. As a by-product of this research, clustering was also investigated as a presearch scheme. It is shown that this procedure has great promise as a means of assessing the complexity of an optimization problem. Experimental results are presented to demonstrate the utility of the self-organizing PSV search algorithms.

A new measure of dependence is formulated, and some of its properties are described. The measure is applied to feature evaluation to show the relation of dependence between features and classes to discrimination between class-conditional probability distributions.

An economical technique for approximating a joint N-dimensional probability density function has been described by Sebestyen and Edie [20]. The algorithm searches for clusters of points and considers each cluster as one hyperellipsoidal cell in an N-dimensional histogram. Among the advantages of this scheme are: 1) the histogram cell descriptors-location, shape, and size-can be determined adaptively from sequentially introduced data samples of known classification and, 2) the number of cells required for a good fit can usually be held to a small number. No assumptions are required about the underlying statistical structure of the data. The algorithm requires three types of "control parameters" which critically affect its performance and are dependent upon the number of dimensions. The three factors control the birth, shape, and growth rate of the cells. Guides were presented in [20] for choosing the control parameter values. These guides functioned well for spaces of 3 dimensions or less, but did not yield usable values for spaces of greater dimensionality. This paper presents heuristics which were developed to automate the selection of the control parameters. The properties of these parameters were studied as a function of dimension. Two of the control parameters were found to be linearly related to dimension. This provides a method for determining their value by extrapolation, thereby avoiding a great deal of computation.

The only guaranteed technique for choosing the best subset of N properties from a set of M is to try all (MN) possible combinations. This is computationally impractical for sets of even moderate size, so heuristic techniques are required. This paper presents seven techniques for choosing good subsets of properties and compares their performance on a nine-class vectorcardiogram classification problem.

A pattern recognition network which computes a weighted sum of nonlinear functions of its inputs is considered. An algorithm for training this multithreshold network is presented. The multithreshold device is used for classifying patterns into more than two categories. Experimental results on the recognition of hand-printed characters are shown. In this case, the nonlinear functions consisted of an orthogonal set of property detectors. The network changed its structure by an evolutionary technique which consisted of periodic replacement of the least useful elements by new ones, randomly chosen.