A structured 3D SOM is an extension of a Self-Organizing Map from 2D to 3D in such a way that a pre-defined structure is built into the design of the 3D map. The structured 3D SOM is a 3×3×3 structure that has a distinct core cube in the center and exterior cubes around the core. The current application of the structured SOM, as a digital music archive, only uses the 8 corner cubes among the 26 exterior cubes. Given that the SOM has a built-in structure, the SOM learning algorithm is modified to include a four-phase learning and labeling phase. The first phase is meant to position the music files in their general locations within the core cube. The second phase positions the music files in their respective corner cubes according to their music genre. The second phase is therefore a semi-supervised version of the SOM algorithm which leads to the stability of the trained SOM in terms of the general distribution of the music files in the core cube. The third phase does a fine adjustment of the weight vectors in the core cube and finalizes the training of the 3D SOM. The final fourth phase is the labeling of the core cube and the association (uploading) of music files to specific nodes in the core cube. Based on the pre-defined structure of the 3D SOM, a precise measure is developed to measure the quality of the resulting trained SOM (in this case, the music archive), as well as the quality of the different categories/genres of music albums based on a novel measure of the distortion values of music files with respect to their respective music genres.

In this paper, we present an approach for sample selection using an ensemble of neural networks for credit scoring. The ensemble determines samples that can be considered outliers by checking the classification accuracy of the neural networks on the original training data samples. Those samples that are consistently misclassified by the neural networks in the ensemble are removed from the training dataset. The remaining data samples are then used to train and prune another neural network for rule extraction. Our experimental results on publicly available benchmark credit scoring datasets show that by eliminating the outliers, we obtain neural networks with higher predictive accuracy and simpler in structure compared to the networks that are trained with the original dataset. A rule extraction algorithm is applied to generate comprehensible rules from the neural networks. The extracted rules are more concise than the rules generated from networks that have been trained using the original datasets.

Keywords and key-phrases that concisely represent text documents are integral to many knowledge management and text information retrieval systems, as well as digital libraries in general. Not all text documents, however, are annotated with good keywords; and the quality of these keywords is often dependent on a tedious, sometimes manual, extraction and tagging process. To automatically extract high quality keywords without the need for a semantic analysis of the document, it is shown that artificial neural networks (ANN) can be trained to only consider in-document word features such as word frequency, word distribution in document, use of word in special parts of the document, and use of word formatting features (i.e. bold-faced, italicized, large-font size). Results show that purely local features are adequate in determining whether a word in a document is a keyword or not. Classification performance yields a G mean of a least 0.83, and weighted f-measure of 0.96 for both keywords and non-keywords. Precision for keywords alone, however, is not as high. To understand the basis for classifying keywords, C4.5 is used to extract rules from the ANN. The extracted rules from C4.5, in the form of a decision tree, show the relative importance of the different document features that were extracted.

The accuracy of neural networks can be improved when they are trained with discretized continuous attributes as additional inputs. Such input augmentation makes it easier for the network weights to form more accurate decision boundaries when the data samples of different classes in the data set are contained in distinct hyper-rectangular subregions in the original input space. In this paper we present first how a neural network can be trained with augmented discretized inputs. The additional inputs are obtained by simply dividing the original interval of each continuous attribute into subintervals of equal length. Thermometer encoding scheme is used to represent these discretized inputs. The network is then pruned to remove most of the discretized inputs as well as the original continuous attributes as long as the network still achieves a minimum preset accuracy requirement. We then discuss how MofN rules can be extracted from the pruned network by analyzing the activations of the network's hidden units and the weights of the network connections that remain in the pruned network. For data sets that have sample classes defined by relatively complex boundaries, surprisingly simple MofN rules with very good accuracy rates are obtained.

Neural networks are widely used for classification as they achieve good predictive accuracy. When the class labels are determined by complex interactions of the input variables, neural networks can be expected to provide better predictions than methods that test on the values of one variable at a time such as univariate decision tree classifiers. On the other hand, when no or relatively simple interaction between variables determines the class membership, the neural network may over fit the data and the input-to-output relationship in the data is represented by a function that is more complex than it should be. In this paper, we propose adding discretized values of the continuous variables in the data as input when training the neural networks. Finding out whether the discretized values or the original continuous values of the variables are useful is achieved by pruning. By having only the relevant inputs left in the pruned networks, we are able to extract classification rules from these networks that are accurate, concise and interpretable.

Keyword extraction is vital for Knowledge Management System, Information Retrieval System, and Digital Libraries as well as for general browsing of the web. Keywords are often the basis of document processing methods such as clustering and retrieval since processing all the words in the document can be slow. Common models for automating the process of keyword extraction are usually done by using several statistics-based methods such as Bayesian, K-Nearest Neighbor, and Expectation-Maximization. These models are limited by word-related features that can be used since adding more features will make the models more complex and difficult to comprehend. In this research, a Neural Network, specifically a backpropagation network, will be used in generalizing the relationship of the title and the content of articles in the archive by following word features other than TF-IDF, such as position of word in the sentence, paragraph, or in the entire document, and formats such as heading, and other attributes defined beforehand. In order to explain how the backpropagation network works, a rule extraction method will be used to extract symbolic data from the resulting backpropagation network. The rules extracted can then be transformed into decision trees performing almost as accurate as the network plus the benefit of being in an easily comprehensible format.

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.

The five papers in this special section focus on white-box nonlinear prediction models.

Neural networks are often selected as tool for software effort prediction because of their capability to approximate any continuous function with arbitrary accuracy. A major drawback of neural networks is the complex mapping between inputs and output, which is not easily understood by a user. This paper describes a rule extraction technique that derives a set of comprehensible IF-THEN rules from a trained neural network applied to the domain of software effort prediction. The suitability of this technique is tested on the ISBSG R11 data set by a comparison with linear regression, radial basis function networks, and CART. It is found that the most accurate results are obtained by CART, though the large number of rules limits comprehensibility. Considering comprehensible models only, the concise set of extracted rules outperform the pruned CART tree, making neural network rule extraction the most suitable technique for software effort prediction when comprehensibility is important.

This paper presents the results from a neural network rule extraction algorithm applied to the LED display recognition problem. We show that pruned neural networks with small number of hidden nodes and connections are able to recognize all the 10 digits from 0 to 9. Earlier work by other researchers demonstrated how symbolic fuzzy rules can be extracted from trained neural networks to solve this problem. Our rules in contrast are crisp rules, and they are obtained from smaller networks. As a result, simpler and easier to understand rules are obtained. These rules give us an insight of how neural networks differentiate one digit from the rest in LED display recognition problem.

Even in an increasingly globalized market, the knowledge about individual local markets could still be invaluable. In this cross-national study of brand image perception of cars, survey data from buyers in the top 20 automobile markets have been collected, where each respondent has been asked to associate a car brand with individual brand images and corporate brand images. These data can be used as tool for decision making at the enterprise level. We describe an algorithm for constructing auto-associative neural networks which can be used as a tool for knowledge discovery from this data. It automatically determines the network topology by adding hidden units as they are needed to improve accuracy and by removing irrelevant input attributes. Two market research applications are presented, the first is for classification, and the second is for measuring similarities in the perceptions of the respondents from the different markets. In the first application, the constructed networks are shown to be more accurate than a decision tree. In the second application, the constructed networks are able to reproduce the training data very accurately and could be used to identify country-level (i.e. local) markets which share similar perceptions about the car brands being studied.

In this paper, we present a recursive algorithm for extracting classification rules from feedforward neural networks (NNs) that have been trained on data sets having both discrete and continuous attributes. The novelty of this algorithm lies in the conditions of the extracted rules: the rule conditions involving discrete attributes are disjoint from those involving continuous attributes. The algorithm starts by first generating rules with discrete attributes only to explain the classification process of the NN. If the accuracy of a rule with only discrete attributes is not satisfactory, the algorithm refines this rule by recursively generating more rules with discrete attributes not already present in the rule condition, or by generating a hyperplane involving only the continuous attributes. We show that for three real-life credit scoring data sets, the algorithm generates rules that are not only more accurate but also more comprehensible than those generated by other NN rule extraction methods.

Various benchmarking studies have shown that artificial neural networks and support vector machines often have superior performance when compared to more traditional machine learning techniques. The main resistance against these newer techniques is based on their lack of interpretability: it is difficult for the human analyst to understand the reasoning behind these models' decisions. Various rule extraction (RE) techniques have been proposed to overcome this opacity restriction. These techniques are able to represent the behavior of the complex model with a set of easily understandable rules. However, most of the existing RE techniques can only be applied under limited circumstances, e.g., they assume that all inputs are categorical or can only be applied if the black-box model is a neural network. In this paper, we present Minerva, which is a new algorithm for RE. The main advantage of Minerva is its ability to extract a set of rules from any type of black-box model. Experiments show that the extracted models perform well in comparison with various other rule and decision tree learners.

This paper proposes GRG (Greedy Rule Generation) algorithm for generating classification rules from a data set with discrete attributes. The algorithm is "greedy" in the sense that at every iteration, it searches for the best rule to generate. The criteria for the best rule include the number of samples that it covers, the number of attributes involved in the rule, and the size of the input subspace it covers. This method is applied for extracting rules from neural networks that have been trained and pruned for solving classification problems. Neural networks with one hidden layer are trained and the proposed GRG algorithm is applied to their discretized hidden unit activation values. Our results show that rule extraction with the GRG method produces rule sets that are more accurate and concise compared to those obtained by a decision tree method and an existing neural network rule extraction method.

This paper addresses the problem of finding neural networks with optimal topology such that their generalization capability is maximized. Our approach is to combine the use of a penalty function during network training and a subset of the training samples for cross-validation. The penalty is added to the error function so that the weights of network connections that are not useful have small magnitude. Such network connections can be pruned if the resulting accuracy of the network does not change beyond a preset level. Training samples in the cross-validation set are used to indicate when network pruning is terminated. Our results on 32 publicly available data sets show that the proposed method outperforms existing neural network and decision tree methods for classification.

Learning in self-organizing maps (SOM) is considered unsupervised because training patterns do not need accompanying desired output information. Prior to its use in some real-world applications, however, a trained SOM often has to be labeled. This labeling phase is usually supervised in that labeled patterns need accompanying output information. Because such labeled patterns are not always available or may not even be possible to construct, the supervised nature of the labeling phase restricts the deployment of SOM from a wide range of potential domains of application. This work proposes a methodical and automatic SOM labeling procedure that does not require a set of prelabeled patterns. Instead, nodes in the trained map are clustered and subsets of training patterns associated to each of the clustered nodes are identified. Salient dimensions per node cluster that constitute the bases for labeling each node in the map are then identified. The effectiveness of the method is demonstrated on a SOM-based international market segmentation study.

Recent advances in algorithms that extract rules from artificial neural networks make it feasible to use neural networks as a tool for acquiring knowledge hidden in the data. Findings are reported from the use of such algorithms to separate core and noncore knowledge in a cross-national study of automobile brand image perception. Respondents from five Western European countries have been asked to associate individual and corporate brand associations for a number of well-known automobile brands. Knowledge, expressed as concise and accurate rules that distinguish between the respondents' perceptions of German and Japanese brands, is extracted from trained neural networks. This paper explains how both core knowledge, which captures the perceptions shared by the respondents in all countries, and country-specific noncore knowledge can be acquired and differentiated by a proposed two-step approach to train and extract rules from a multi-neural network system. The experimental results show that, in addition to providing a better understanding of the differences and similarities in the brand image perceptions of consumers in various countries, the proposed approach also yields better predictive accuracy than a decision tree method.

We investigate how knowledge acquired by a neural network from one input environment can be transferred and revised for similar application in a new environment. Knowledge revision is achieved by re-training the neural network. Knowledge common to both environments are retained, while localized knowledge components are introduced during network retraining. Various network performance measures are computed to measure how much knowledge is transferred and revised. Furthermore, because the knowledge acquired by a neural network can be expressed as an accurate set of simple rules, we are able to compare knowledge extracted from one network with that from another. In a cross-national study of car image perceptions, a comparison of the original and revised knowledge gives us insights into the commonalities and differences in brand perceptions across countries.

In many applications, black-box prediction is not satisfactory, and understanding the data is of critical importance. Typically, approaches useful for understanding of data involve logical rules, evaluate similarity to prototypes, or are based on visualization or graphical methods. This paper is focused on the extraction and use of logical rules for data understanding. All aspects of rule generation, optimization, and application are described, including the problem of finding good symbolic descriptors for continuous data, tradeoffs between accuracy and simplicity at the rule-extraction stage, and tradeoffs between rejection and error level at the rule optimization stage. Stability of rule-based description, calculation of probabilities from rules, and other related issues are also discussed. Major approaches to extraction of logical rules based on neural networks, decision trees, machine learning, and statistical methods are introduced. Optimization and application issues for sets of logical rules are described. Applications of such methods to benchmark and real-life problems are reported and illustrated with simple logical rules for many datasets. Challenges and new directions for research are outlined.

Neural networks (NNs) have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how Me problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained NNs for regression. This article presents an approach for extracting rules from trained NNs for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.

An iterative backward selection method for determination of relevant input variables in data-driven fuzzy modeling is presented. The method utilizes parameters of the Takagi-Sugeno model as a factor to determine the significance of input variables. As a result, it is less computationally intensive than most of the existing methods for input variable selection.

We describe a method for multivariate function approximation which combines neural network learning, clustering and multiple regression. Neural networks with a single hidden layer are universal function approximators. However, due to the complexity of the network topology and the nonlinear transfer function used in computing the activation of the hidden units, the predictions of a trained network are difficult to comprehend. On the other hand, predictions from a multiple linear regression equation are easy to understand but not accurate when the underlying relationship between the input variables and the output variable as nonlinear. The method presented in this paper generates a set of multiple linear regression equations using neural networks. The number of regression equations as determined by clustering the weighted input variables. The predictions for samples in the same cluster are computed by the same regression equation. Experimental results on real-world data demonstrate that the new method generates relatively few regression equations from the training data samples. The errors an prediction using these equations are comparable to or lower than those achieved by existing function approximation methods.

This article presents an algorithm that constructs feedforward neural networks with a single hidden layer for pattern classification. The algorithm starts with a small number of hidden units in the network and adds more hidden units as needed to improve the network's predictive accuracy. To determine when to stop adding new hidden units, the algorithm makes use of a subset of the available training samples for cross validation. New hidden units are added to the network only if they improve the classification accuracy of the network on the training samples and on the cross-validation samples. Extensive experimental results show that the algorithm is effective in obtaining networks with predictive accuracy rates that are better than those obtained by state-of-the-art decision tree methods.

This paper describes a system for rapid acquisition of 3D models of objects for use in applications such as CAD and VR. The system consists of an image capturing platform, which comprises a PC-controlled turntable and CCD camera, and associated computer vision and graphics algorithms for 3D model acquisition. An algorithm for accurately recovering 3D points from image sequences is described. Experiments performed on synthetic and real data show that the algorithm is accurate in recovering the coordinates of 3D points and is robust against noise in 2D feature location and 3D object rotation. Results of applying the system to acquiring 3D models of real objects are illustrated.

A simple algorithm for pruning feedforward neural networks with a single hidden layer trained for function approximation is presented. The algorithm assumes that the networks have been trained with more then the necessary number of hidden units and it consists of two stages. In the first stage redundant hidden units are removed, and in the second stage irrelevant input units are removed. Experimental results on seven publicly available data sets show that the proposed algorithm outperforms other methods such as the nearest neighbors, decision trees and regression-based methods.

An effective algorithm for extracting M-of-N rules from trained feedforward neural networks is proposed. Two components of the algorithm distinguish our method from previously proposed algorithms which extract symbolic rules from neural networks. First, we train a network where each input of the data can only have one of the two possible values, -1 or one. Second, we apply the hyperbolic tangent function to each connection from the input layer to the hidden layer of the network. By applying this squashing function, the activation values at the hidden units are effectively computed as the hyperbolic tangent (or the sigmoid) of the weighted inputs, where the weights have magnitudes that are equal one. By restricting the inputs and the weights to binary values either -1 or one, the extraction of M-of-N rules from the networks becomes trivial. We demonstrate the effectiveness of the proposed algorithm on several widely tested datasets. For datasets consisting of thousands of patterns with many attributes, the rules extracted by the algorithm are surprisingly simple and accurate.

Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network re-training. Given a fully connected trained feedforward network, FERNN first identifies the relevant hidden units by computing their information gains. Next, it identifies relevant connections from the input units to the relevant hidden units by checking the magnitudes of their weights. Finally, FERNN generates rules based on the relevant hidden units and weights. Our experimental results show that the size and accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.

Neural networks and decision tree methods are two common approaches to pattern classification. While neural networks can achieve high predictive accuracy rates, the decision boundaries they form are highly nonlinear and generally difficult to comprehend. Decision trees, on the other hand, can be readily translated into a set of rules. In this paper, we present a novel algorithm for generating oblique decision trees that capitalizes on the strength of both approaches. Oblique decision trees classify the patterns by testing on linear combinations of the input attributes. As a result, an oblique decision tree is usually much smaller than the univariate tree generated for the same domain. Our algorithm consists of two components: connectionist and symbolic. A three-layer feedforward neural network is constructed and pruned, a decision tree is then built from the hidden unit activation values of the pruned network. An oblique decision tree is obtained by expressing the activation values using the original input attributes. We test our algorithm on a wide range of problems. The oblique decision trees generated by the algorithm preserve the high accuracy of the neural networks, while keeping the explicitness of decision trees. Moreover, they outperform univariate decision trees generated by the symbolic approach and oblique decision trees built by other approaches in accuracy and tree size.

Selective induction algorithms are efficient in learning target concepts but inherit a major limitation each time only one feature is used to partition the data until the data is divided into uniform segments. This limitation results in problems like replication, repetition, and fragmentation. Constructive induction has been an effective means to overcome some of the problems. The underlying idea is to construct compound features that increase the representation power so as to enhance the learning algorithm's capability in partitioning data. Unfortunately, many constructive operators are often manually designed and choosing which one to apply poses a serious problem itself. We propose an automatic way of constructing compound features. The method can be applied to both continuous and discrete data and thus all the three problems can be eliminated or alleviated. Our empirical results indicate the effectiveness of the proposed method.

The hidden layer of backpropagation neural networks (NNs) holds the key to the networks' success in solving pattern classification problems. The units in the hidden layer encapsulate the network's internal representations of the outside world described by the input data. this paper, the hidden representations of trained networks are investigated by means simple greedy clustering algorithm. This clustering algorithm is applied to networks have been trained to solve well-known problems: the monks problems, the 5-bit problem and the contiguity problem. The results from applying the algorithm to problems with known concepts provide us with a better understanding of NN learning. These also explain why NNs achieve higher predictive accuracy than that of decision-tree methods. The results of this study can be readily applied to rule extraction from Production rules are extracted for the parity and the monks problems, as well as benchmark data set: Pima Indian diabetes diagnosis. The extracted rules from the Indian diabetes data set compare favorably with rules extracted from ARTMAP NNs terms of predictive accuracy and simplicity.

This article proposes the use of a penalty function for pruning feedforward neural network by weight elimination. The penalty function proposed consists of two terms. The first term is to discourage the use of unnecessary connections, and the second term is to prevent the weights of the connections from taking excessively large values. Simple criteria for eliminating weights from the network are also given. The effectiveness of this penalty function is tested on three well-known problems: the contiguity problem, the parity problems, and the monks problems. The resulting pruned networks obtained for many of these problems have fewer connections than previously reported in the literature.

An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network but, more important, to detect the relevant inputs. The algorithm generates rules from the pruned network by considering only a small number of activation values at the hidden units. If the number of inputs connected to a hidden unit is sufficiently small, then rules that describe how each of its activation values is obtained can be readily generated. Otherwise the hidden unit will be split and treated as output units, with each output unit corresponding to an activation value. A hidden layer is inserted and a new subnetwork is formed, trained, and pruned. This process is repeated until every hidden unit in the network has a relatively small number of input units connected to it. Examples on how the proposed algorithm works are shown using real-world data arising from molecular biology and signal processing. Our results show that for these complex problems, the algorithm can extract reasonably compact rule sets that have high predictive accuracy rates.

Discretization can turn numeric attributes into discrete ones. Feature selection can eliminate some irrelevant and/or redundant attributes. Chi2 is a simple and general algorithm that uses the /spl chi//sup 2/ statistic to discretize numeric attributes repeatedly until some inconsistencies are found in the data. It achieves feature selection via discretization. It can handle mixed attributes, work with multiclass data, and remove irrelevant and redundant attributes.

Feature selection is an integral part of most learning algorithms. Due to the existence of irrelevant and redundant attributes, by selecting only the relevant attributes of the data, higher predictive accuracy can be expected from a machine learning method. In this paper, we propose the use of a three-layer feedforward neural network to select those input attributes that are most useful for discriminating classes in a given set of input patterns. A network pruning algorithm is the foundation of the proposed algorithm. By adding a penalty term to the error function of the network, redundant network connections can be distinguished from those relevant ones by their small weights when the network training process has been completed. A simple criterion to remove an attribute based on the accuracy rate of the network is developed. The network is retrained after removal of an attribute, and the selection process is repeated until no attribute meets the criterion for removal. Our experimental results suggest that the proposed method works very well on a wide variety of classification problems.

Classification is one of the data mining problems receiving great attention recently in the database community. The paper presents an approach to discover symbolic classification rules using neural networks. Neural networks have not been thought suited for data mining because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. With the proposed approach, concise symbolic rules with high accuracy can be extracted from a neural network. The network is first trained to achieve the required accuracy rate. Redundant connections of the network are then removed by a network pruning algorithm. The activation values of the hidden units in the network are analyzed, and classification rules are generated using the result of this analysis. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of standard data mining test problems.

Discretization can turn numeric attributes into discrete ones. Feature selection can eliminate some irrelevant attributes. This paper describes Chi2 a simple and general algorithm that uses the /spl chi//sup 2/ statistic to discretize numeric attributes repeatedly until some inconsistencies are found in the data, and achieves feature selection via discretization. The empirical results demonstrate that Chi/sup 2/ is effective in feature selection and discretization of numeric and ordinal attributes.

A new method for constructing a feedforward neural network is proposed. The method starts with a single hidden unit and more units are added to the hidden layer one at a time until a network that completely recognizes all its input patterns is constructed. The novel idea about this method is that the network is trained to maximize a certain likelihood function and not to minimize the more widely used mean squared error function. We show that when a new hidden unit is added to the network, this likelihood function is guaranteed to increase and this increase ensures the finite termination of the method. We also provide a wide range of numerical results. The method was tested on the n -bit parity problems and the spiral problem. It was able to construct networks having less than n hidden units that solve the n -bit parity problems for n = 4, 5, 6, 7 and 8. The method was also tested on some real-world data and the networks it constructed were shown to be able to predict patterns not in the training set with more than 95% accuracy.

This paper describes an algorithm for constructing a single hidden layer feedforward neural network. A distinguishing feature of this algorithm is that it uses the quasi-Newton method to minimize the sequence of error functions associated with the growing network. Experimental results indicate that the algorithm is very efficient and robust. The algorithm was tested on two test problems. The first was the n-bit parity problem and the second was the breast cancer diagnosis problem from the University of Wisconsin Hospitals. For the n-bit parity problem, the algorithm was able to construct neural network having less than n hidden units that solved the problem for n=4,...,7. For the cancer diagnosis problem, the neural networks constructed by the algorithm had small number of hidden units and high accuracy rates on both the training data and the testing data.