Multi-label classification paradigm has had a growing interest because of the emergence of a large number of classification problems where each of the instances of the data can be associated with several output labels simultaneously. Several ensemble methods were proposed to solve the multilabel classification problem. However, most of them simply create diversity in the ensemble by following a random procedure and give the same importance to all members. In this paper, we propose a Grammar-Guided Genetic Programming algorithm to build ensembles of multi-label classifiers. Given a pool of multilabel classifiers, each of them modeling dependencies among a subset of k labels, they are combined into a tree-shaped ensemble. At each node of the tree, predictions of its children nodes are combined, while each leaf represents a classifier from the pool. We propose two configurations for the method: using a fixed value of k for all classifiers in the pool, or using a variable value of k for each classifier, thus being able to capture relationships among groups of labels of different size in the ensemble. The experiments performed over sixteen multi-label dataset and using five evaluation metrics demonstrated that our method performs significantly better than the state-of-the-art ensembles of multilabel classifiers.

A new density-based clustering algorithm, RNN-DBSCAN, is presented which uses reverse nearest neighbor counts as an estimate of observation density. Clustering is performed using a DBSCAN-like approach based on k nearest neighbor graph traversals through dense observations. RNN-DBSCAN is preferable to the popular density-based clustering algorithm DBSCAN in two aspects. First, problem complexity is reduced to the use of a single parameter (choice of k nearest neighbors), and second, an improved ability for handling large variations in cluster density (heterogeneous density). The superiority of RNN-DBSCAN is demonstrated on several artificial and real-world datasets with respect to prior work on reverse nearest neighbor based clustering approaches (RECORD, IS-DBSCAN, and ISB-DBSCAN) along with DBSCAN and OPTICS. Each of these clustering approaches is described by a common graph-based interpretation wherein clusters of dense observations are defined as connected components, along with a discussion on their computational complexity. Heuristics for RNN-DBSCAN parameter selection are presented, and the effects of k on RNN-DBSCAN clusterings discussed. Additionally, with respect to scalability, an approximate version of RNN-DBSCAN is presented leveraging an existing approximate k nearest neighbor technique.

The paper introduces a multi-layer multi-column model of the cortex that uses four different neuron types and short-term plasticity dynamics. It was designed with details of neuronal connectivity available in the literature and meets these conditions: (1) biologically accurate laminar and columnar flows of activity, (2) normal function of low-threshold spiking and fast spiking neurons, and (3) ability to generate different stages of epileptiform activity. With these characteristics the model allows for modeling lesioned or malformed cortex, i.e. examine properties of developmentally malformed cortex in which the balance between inhibitory neuron subtypes is disturbed.

Multiple-instance learning (MIL) is a supervised learning technique that addresses the problem of classifying bags of instances instead of single instances. In this paper, we introduce a rule-based MIL algorithm, called mi-DS, and compare it with 21 existing MIL algorithms on 26 commonly used data sets. The results show that mi-DS performs on par with or better than several well-known algorithms and generates models characterized by balanced values of precision and recall. Importantly, the introduced method provides a framework that can be used for converting other rule-based algorithms into MIL algorithms.

Molecular dynamics method is used for computing equilibrium in many physical systems as well as a powerful solver used for minimization of criterion functions such as the stress function in multidimensional scaling. We have designed a new tool for visualization of highly-dimensional data in 3D based on the method. We have shown on several data sets that the new method is fast and accurate.

In general, simulation of brain function involves a serial process that includes conceptual definition and design, computational (or mathematical) modeling, testing the correlation between model output and the brain, and generation of biologically testable hypotheses. Processing simulators provide a specific simulation environment in which computational models can be custom built according to their conceptual model of interest, as well as evaluate the result of the simulation by using analytic tools in the simulator. In this paper we describe a new simulator, the Neuronal Multisensory Processing Simulator (NMPS) designed primarily to model the first requisite step of multisensory processing: multisensory convergence. The NMPS generated a network of spiking neurons and stimulated convergent inputs to those neurons. Analysis of both neurons and network revealed responses similar to biological multisensory processing, and provided insight into multisensory features currently inaccessible to observation or experimentation.

A majority of artificial spiking neural network models are used for modeling and/or understanding biological processes, while only in a few instances have they been used for solving engineering tasks. When spiking neural networks are used for an engineering task often the inputs are fed directly into the network and it is up to the network to evolve the connections from a randomly connected pool of neurons. This approach may generate a network capable of producing the desired output (mapping), however, the resulting network is typically undecipherable. In modern digital systems a microprocessor is used to perform a task. Specifically, an analog to digital converter is used to allow the microprocessor to accurately interpret analog signals coming from sensors monitoring the task. In this paper, a spiking neural network is used to measure the intensity of an input analog signal using a coincidence detection approach, where a pool of intensity detecting neurons is responsible for identifying specific intensities. At any instant in time the active neuron in the pool represents current input intensity of the signal. We first show the circuit for a single intensity case. Next, the entire intensity detection circuit is examined over a large range of intensities. Finally, the entire network is tested when the input signal is mixed with Gaussian noise. The resulting analog coincidence intensity detection network has a rapid and accurate response while being noise-resistant. The modularity of the design allows the network to be easily incorporated as a preprocessing system in other and/or larger simulations.

We present a new framework and method for solving Multiple Instance Learning (MIL) problems. As a variation on supervised learning, MIL addresses the problem of classifying a bag of instances. If at least one of the instances in a bag is positive the bag is labeled positive, otherwise it is negative. We use a divide and conquer strategy to identify true positive group of instances in the positive bags and use Bayesian statistics to minimize the false positive instances in the same bags. After testing on benchmark data we also use the method on a challenging task of predicting behavior from molecular profiles data. Comparison results show that our method performs on par or better than other MIL methods.

In this paper, we introduce a novel system for recognition of partially occluded and rotated images. The system is based on a hierarchical network of integrate-and-fire spiking neurons with random synaptic connections and a novel organization process. The network generates integrated output sequences that are used for image classification. The proposed network is shown to provide satisfactory predictive performance given that the number of the recognition neurons and synaptic connections are adjusted to the size of the input image. Comparison of synaptic plasticity activity rule (SAPR) and spike timing dependant plasticity rules, which are used to learn connections between the spiking neurons, indicates that the former gives better results and thus the SAPR rule is used. Test results show that the proposed network performs better than a recognition system based on support vector machines.

Literature-based discovery for hypothesis generation is a subarea of text mining that aims to discover novel or previously-unknown knowledge from two complementary but disjoint (CBD) sets of literatures. The discovery approach is based on Swanson's discovery models where indirect connections between two disjoint sets of literatures A and C could be found through a set of common terms B extracted from A and C. In this paper, we report an application of an inductive logic programming (ILP), specifically the WARMR algorithm, to the field of literature-based discovery. The application extends Swanson's closed discovery model to uncover potentially meaningful knowledge in forms of relational frequent patterns that may exist after the connections between the two sets of literatures are found. We conducted an experiment between two pairs of topics: Raynaud's disease and fish oils, and Down syndrome and cell polarity. The experimental results demonstrate that our method can be used to enhance a literature-based discovery approach by providing potentially meaningful knowledge in addition to the indirect connections.

BACKGROUND: Accurate peptide identification is important to high-throughput proteomics analyses that use mass spectrometry. Search programs compare fragmentation spectra (MS/MS) of peptides from complex digests with theoretically derived spectra from a database of protein sequences. Improved discrimination is achieved with theoretical spectra that are based on simulating gas phase chemistry of the peptides, but the limited understanding of those processes affects the accuracy of predictions from theoretical spectra. RESULTS: We employed a robust data mining strategy using new feature annotation functions of MAE software, which revealed under-prediction of the frequency of occurrence in fragmentation of the second peptide bond. We applied methods of exploratory data analysis to pre-process the information in the MS/MS spectra, including data normalization and attribute selection, to reduce the attributes to a smaller, less correlated set for machine learning studies. We then compared our rule building machine learning program, DataSqueezer, with commonly used association rules and decision tree algorithms. All used machine learning algorithms produced similar results that were consistent with expected properties for a second gas phase mechanism at the second peptide bond. CONCLUSION: The results provide compelling evidence that we have identified underlying chemical properties in the data that suggest the existence of an additional gas phase mechanism for the second peptide bond. Thus, the methods described in this study provide a valuable approach for analyses of this kind in the future.

BACKGROUND: Protein structure prediction methods provide accurate results when a homologous protein is predicted, while poorer predictions are obtained in the absence of homologous templates. However, some protein chains that share twilight-zone pairwise identity can form similar folds and thus determining structural similarity without the sequence similarity would be desirable for the structure prediction. The folding type of a protein or its domain is defined as the structural class. Current structural class prediction methods that predict the four structural classes defined in SCOP provide up to 63% accuracy for the datasets in which sequence identity of any pair of sequences belongs to the twilight-zone. We propose SCPRED method that improves prediction accuracy for sequences that share twilight-zone pairwise similarity with sequences used for the prediction. RESULTS: SCPRED uses a support vector machine classifier that takes several custom-designed features as its input to predict the structural classes. Based on extensive design that considers over 2300 index-, composition- and physicochemical properties-based features along with features based on the predicted secondary structure and content, the classifier's input includes 8 features based on information extracted from the secondary structure predicted with PSI-PRED and one feature computed from the sequence. Tests performed with datasets of 1673 protein chains, in which any pair of sequences shares twilight-zone similarity, show that SCPRED obtains 80.3% accuracy when predicting the four SCOP-defined structural classes, which is superior when compared with over a dozen recent competing methods that are based on support vector machine, logistic regression, and ensemble of classifiers predictors. CONCLUSION: The SCPRED can accurately find similar structures for sequences that share low identity with sequence used for the prediction. The high predictive accuracy achieved by SCPRED is attributed to the design of the features, which are capable of separating the structural classes in spite of their low dimensionality. We also demonstrate that the SCPRED's predictions can be successfully used as a post-processing filter to improve performance of modern fold classification methods.

This letter introduces a biologically inspired very simple spiking neuron model. The model retains only crucial aspects of biological neurons: a network of time-delayed weighted connections to other neurons, a threshold-based generation of action potentials, action potential frequency proportional to stimulus intensity, and interneuron communication that occurs with time-varying potentials that last longer than the associated action potentials. The key difference between this model and existing spiking neuron models is its great simplicity: it is basically a collection of linear and discontinuous functions with no differential equations to solve. The model's ability to operate in a complex network was tested by using it as a basis of a network implementing a hypothetical echolocation system. The system consists of an emitter and two receivers. The outputs of the receivers are connected to a network of spiking neurons (using the proposed model) to form a detection grid that acts as a map of object locations in space. The network uses differences in the arrival times of the signals to determine the azimuthal angle of the source and time of flight to calculate the distance. The activation patterns observed indicate that for a network of spiking neurons, which uses only time delays to determine source locations, the spatial discrimination varies with the number and relative spacing of objects. These results are similar to those observed in animals that use echolocation.

In this paper, we describe a new Synaptic Plasticity Activity Rule (SAPR) developed for use in networks of spiking neurons. Such networks can be used for simulations of physiological experiments as well as for other computations like image analysis. Most synaptic plasticity rules use artificially defined functions to modify synaptic connection strengths. In contrast, our rule makes use of the existing postsynaptic potential values to compute the value of adjustment. The network of spiking neurons we consider consists of excitatory and inhibitory neurons. Each neuron is implemented as an integrate-and-fire model that accurately mimics the behavior of biological neurons. To test performance of our new plasticity rule we designed a model of a biologically-inspired signal processing system, and used it for object detection in eye images of diabetic retinopathy patients, and lung images of cystic fibrosis patients. The results show that the network detects the edges of objects within an image, essentially segmenting it. Our ultimate goal, however, is not the development of an image segmentation tool that would be more efficient than nonbiological algorithms, but developing a physiologically correct neural network model that could be applied to a wide range of neurological experiments. We decided to validate the SAPR by using it in a network of spiking neurons for image segmentation because it is easy to visually assess the results. An important thing is that image segmentation is done in an entirely unsupervised way.

Business intelligence and bioinformatics applications increasingly require the mining of datasets consisting of millions of data points, or crafting real-time enterprise-level decision support systems for large corporations and drug companies. In all cases, there needs to be an underlying data mining system, and this mining system must be highly scalable. To this end, we describe a new rule learner called DataSqueezer. The learner belongs to the family of inductive supervised rule extraction algorithms. DataSqueezer is a simple, greedy, rule builder that generates a set of production rules from labeled input data. In spite of its relative simplicity, DataSqueezer is a very effective learner. The rules generated by the algorithm are compact, comprehensible, and have accuracy comparable to rules generated by other state-of-the-art rule extraction algorithms. The main advantages of DataSqueezer are very high efficiency, and missing data resistance. DataSqueezer exhibits log-linear asymptotic complexity with the number of training examples, and it is faster than other state-of-the-art rule learners. The learner is also robust to large quantities of missing data, as verified by extensive experimental comparison with the other learners. DataSqueezer is thus well suited to modern data mining and business intelligence tasks, which commonly involve huge datasets with a large fraction of missing data.

The paper describes development and application of a spiking neuron model. The neuron model mimics the observed behavior of neurons in that the firing frequency is proportional to the input signal. The network of such spiking neurons is used for processing of received echolocation data to map out an area in a 2D space in real-time.

The task of extracting knowledge from databases is quite often performed by machine learning algorithms. The majority of these algorithms can be applied only to data described by discrete numerical or nominal attributes (features). In the case of continuous attributes, there is a need for a discretization algorithm that transforms continuous attributes into discrete ones. We describe such an algorithm, called CAIM (class-attribute interdependence maximization), which is designed to work with supervised data. The goal of the CAIM algorithm is to maximize the class-attribute interdependence and to generate a (possibly) minimal number of discrete intervals. The algorithm does not require the user to predefine the number of intervals, as opposed to some other discretization algorithms. The tests performed using CAIM and six other state-of-the-art discretization algorithms show that discrete attributes generated by the CAIM algorithm almost always have the lowest number of intervals and the highest class-attribute interdependency. Two machine learning algorithms, the CLIP4 rule algorithm and the decision tree algorithm, are used to generate classification rules from data discretized by CAIM. For both the CLIP4 and decision tree algorithms, the accuracy of the generated rules is higher and the number of the rules is lower for data discretized using the CAIM algorithm when compared to data discretized using six other discretization algorithms. The highest classification accuracy was achieved for data sets discretized with the CAIM algorithm, as compared with the other six algorithms.

The main result of this paper is a constructive proof of a formula for the upper bound of the approximation error in Linfinity (supremum norm) of multidimensional functions by feedforward networks with one hidden layer of sigmoidal units and a linear output. This result is applied to formulate a new method of neural-network synthesis. The result can also be used to estimate complexity of the maximum-error network and/or to initialize that network weights. An example of the network synthesis is given.

Spatio-temporal coding that combines spatial constraints with temporal sequencing is of great interest to brain-like circuit modelers. In this paper we present some new ideas of how these types of circuits can self-organize. We introduce a temporal correlation rule based on the time difference between the firings of neurons.With the aid of this rule we show an analogy between a graph and a network of spiking neurons. The shortest path, clustering based on the nearest neighbor, and the minimal spanning tree algorithms are solved using the proposed approach.

Foreword. Preface. 1. Data Mining and Knowledge Discovery. 2. Rough Sets. 3. Fuzzy Sets. 4. Bayesian Methods. 5. Evolutionary Computing. 6. Machine Learning. 7. Neural Networks. 8. Clustering. 9. Preprocessing. Index.

Describes a semi-automated procedure for analyzing single photon emission computed tomography (SPECT) images of a human heart and classifying the images into one of several categories: normal, infarct, ischemia, infarct and ischemia, reverse re-distribution, artifact and equivocal. The procedure aids the physician in the interpretation of SPECT images and consists of two steps. The first step processes the reconstructed SPECT images. These images contain multiple slices of 64/spl times/64 pixels with 16 bits resolution. Scanned images are converted into numerical format by using boundary extraction, region of interest (ROI) selection, and segmentation techniques. A new algorithm was developed to extract a rectangular ROI from each image. The second step involves automatic classification of the processed images into one of the seven categories, listed above, using a knowledge-based system employing machine learning algorithms (C4.5 and CLIP3) and fuzzy logic modeling, to generate classification rules. The performance of the system is measured in terms of accuracy, the gold standard being interpretation of the images by experienced cardiologists. The accuracy of the system using the rules generated by the machine learning algorithms were 94% and 81%, respectively. Accuracy, after the fuzzy linguistic variables were specified from the rules generated by the C4.5 and CLIP3 algorithms, was 91% and 86%, respectively. Overall, the system performance closely approximated that of an experienced cardiologist.

The relation between the decision trees generated by a machine learning algorithm and the hidden layers of a neural network is described. A continuous ID3 algorithm is proposed that converts decision trees into hidden layers. The algorithm allows self-generation of a feedforward neural network architecture. In addition, it allows interpretation of the knowledge embedded in the generated connections and weights. A fast simulated annealing strategy, known as Cauchy training, is incorporated into the algorithm to escape from local minima. The performance of the algorithm is analyzed on spiral data.

An expert system for the diagnosis of stenoses in the three main coronary arteries (left anterior descending, right coronary artery and circumflex) is described. First, the knowledge base domain--201Tl scintigrams--is explained and the method of preprocessing the original heart images is given. Next, the method of dealing with the uncertainties present both in the cardiologist-specified rules and the data using the Dempster-Shafer theory of evidence is explained. Finally, the constructed expert system and the results are discussed and several graphical examples are shown.