Many real-world classification problems are represented by very sparse and high-dimensional data. The recent successes of a linear programming support vector machine (LPSVM) for feature selection motivated a deeper analysis of the method when applied to sparse, multivariate data. Due to the sparseness, the selection of a classification model is greatly influenced by the characteristics of that particular dataset. In this study, we investigate a feature selection strategy based on LPSVM as the initial feature filter, combined with state-of-art classification rules, and apply to five real-life datasets of the agnostic learning vs. prior knowledge challenge of IJCNN2007. Our goal is to better understand the robustness of LPSVM as a feature filter. Our analysis suggests that LPSVM can be a useful black box method for identification of the profile of the informative features in the data. If the data are complex and better separable by nonlinear methods, then feature pre-filtering by LPSVM enhances the data representation for other classifiers.

We investigate the relative efficacy of several classification models with and without feature selection. Simple classification rules are frequently preferable and superior to more complex models for microarray data that are typically undersampled. Improved classification accuracy is obtained with feature selection. We summarize some of the important questions considered in the literature that practitioners have to take into account when selecting a classifier for microarrays.

MOTIVATION: Two practical realities constrain the analysis of microarray data, mass spectra from proteomics, and biomedical infrared or magnetic resonance spectra. One is the 'curse of dimensionality': the number of features characterizing these data is in the thousands or tens of thousands. The other is the 'curse of dataset sparsity': the number of samples is limited. The consequences of these two curses are far-reaching when such data are used to classify the presence or absence of disease. RESULTS: Using very simple classifiers, we show for several publicly available microarray and proteomics datasets how these curses influence classification outcomes. In particular, even if the sample per feature ratio is increased to the recommended 5-10 by feature extraction/reduction methods, dataset sparsity can render any classification result statistically suspect. In addition, several 'optimal' feature sets are typically identifiable for sparse datasets, all producing perfect classification results, both for the training and independent validation sets. This non-uniqueness leads to interpretational difficulties and casts doubt on the biological relevance of any of these 'optimal' feature sets. We suggest an approach to assess the relative quality of apparently equally good classifiers.

Several methods of aggregating outcomes of classifiers are applied to artificial and real magnetic resonance MR spectra. Logistic regression, linear combination of classifiers, fuzzy integration, stacked generalization and some other fusion methods, as well as different ways of estimating necessary parameters, are considered. The results indicate that the fusion of classifiers improves the performance of the individual classifiers. On real MR spectra, which are characterized by the paucity of experimental data and low signal-tonoise ratio, the results vary. Some methods perform well on some data sets and poorly on others. Strategies are recommended to gain from classifier aggregation.

A questionnaire, designed to assess bleeding predispositions in tonsillectomy and/or adenoidectomy patients, was administered to 236 otherwise healthy children. For comparative purposes, 114 patients with bleeding disorders were also studied. An unsupervised non-metric clustering technique was used in an attempt to classify bleeders against non-bleeders based solely on the responses to the questionnaire. Non-metric techniques are essential for the classification process because of the large number of missing attribute values in the patient data set. As a benchmark, a supervised inductive machine learning strategy was also used to classify the patients. Performance results are compared and contrasted between the techniques across different subsets of the patient data. These techniques are also evaluated as a methodology for determining the relative significance of attributes vis-à-vis the reduction of the dimensionality of a large medical data set. In this investigation, the classification rate achieved using the non-metric technique (73%) was only marginally poorer than the rate using the supervised technique (76%). Moreover, these results were obtained with an accompanying 80% reduction in the number of attributes used to perform the analysis.