We describe a data complexity approach to kernel selection based on the behavior of polynomial and Gaussian kernels. Our resultsshow how the use of a Gaussian kernel produces a gram matrix with useful local information that has no equivalent counterpart inpolynomial kernels.By exploiting neighborhood information embedded by data complexity measures, we are able to carry out a form of meta-generalization.Our goal is to predict which data sets are more favorable to particular kernels (Gaussian or polynomial).The end result is a framework to improve the model selection process in Support Vector Machines.

Device scaling engineering is facing major challenges in producing reliable transistors for future electronic technologies. With shrinking device sizes, the total circuit sensitivity to both permanent and transient faults has increased significantly. Research for fault tolerant processors has primarily focused on the conventional processor architectures. Neural network computing has been employed to solve a wide range of problems. This paper presents a design and implementation of a physical neural network that is resilient to permanent hardware faults. To achieve scalability, it uses tiled neuron clusters where neuron outputs are efficiently forwarded to the target neurons using source based spanning tree routing. To achieve fault resilience in the face of increasing number of permanent hardware failures, the design pro-actively preserves neural network computing performance by selectively replicating performance critical neurons. Furthermore, the paper presents a spanning tree recovery solution that mitigates disruption to distribution of neuron outputs caused by failed neuron clusters. The proposed neuron cluster design is implemented in Verilog. We studied the fault resilience performance of the described design using a RBM neural network trained for classifying handwritten digit images. Results demonstrate that our approach can achieve improved fault resilience performance by replicating only 5% most important neurons.

We address a particular scenario within the area of domain adaptation, where a predictive model obtained from a source domain can be applied directly to a target domain. Both source and target domains share the same input or feature space, but we do not impose any restrictions on the marginal and class posterior distributions (both distributions can differ). Our main assumption is that the difference between the source and target domains can be traced to a systematic change caused by some modification in the sensing device, or environment surrounding the phenomenon under analysis, as for example when the training and testing samples correspond to stars (i.e., to light curves) that belong to different galaxies. We demonstrate how such systematic change can be reversed by shifting the target data towards the source data until both distributions are aligned. Our approach uses maximum likelihood to compute the right amount of displacement along each variable under analysis. We test our methodology on the classification of Cepheid variable stars according to their pulsation mode: fundamental and first-overtone. Experimental results with three galaxy datasets (Large Magellanic Cloud, Small Magellanic Cloud, and M33), show the effectiveness of our approach.

The presence of sub-classes within a data sample suggests a class decomposition approach to classification, where each subclass is treated as a new class. Class decomposition can be effected using multiple linear classifiers in an attempt to outperform a single global linear classifier; the goal is to gain in model complexity while keeping error variance low. We describe a study aimed at understanding the conditions behind the success or failure of class decomposition when combined with linear classifiers. We identify two relevant data properties as indicators of the suitability of class decomposition: 1) linear separability; and 2) class overlap. We use well-known data complexity measures to evaluate the presence of these properties in a data sample. Our methodology indicates when to avoid performing class decomposition based on such data properties. In addition we conduct a similar analysis at a more granular level for data samples marked as suitable for class decomposition. This extra analysis shows how to improve in efficiency during class decomposition. From an empirical standpoint, we test our technique on several real-world classification problems; results validate our methodology.

Counting craters in remotely sensed images is the only tool that provides relative dating of remote planetary surfaces. Surveying craters requires counting a large amount of small subkilometer craters, which calls for highly efficient automatic crater detection. In this article, we present an integrated framework on autodetection of subkilometer craters with boosting and transfer learning. The framework contains three key components. First, we utilize mathematical morphology to efficiently identify crater candidates , the regions of an image that can potentially contain craters. Only those regions occupying relatively small portions of the original image are the subjects of further processing. Second, we extract and select image texture features, in combination with supervised boosting ensemble learning algorithms, to accurately classify crater candidates into craters and noncraters. Third, we integrate transfer learning into boosting, to enhance detection performance in the regions where surface morphology differs from what is characterized by the training set. Our framework is evaluated on a large test image of 37,500 × 56,250 m 2 on Mars, which exhibits a heavily cratered Martian terrain characterized by nonuniform surface morphology. Empirical studies demonstrate that the proposed crater detection framework can achieve an F1 score above 0.85, a significant improvement over the other crater detection algorithms.

Identifying impact craters on planetary surfaces is one fundamental task in planetary science. In this paper, we present an embedded framework on auto-detection of craters, using feature selection and boosting strategies. The paradigm aims at building a universal and practical crater detector. This methodology addresses three issues that such a tool must possess: (i) it utilizes mathematical morphology to efficiently identify the regions of an image that can potentially contain craters; only those regions, defined as crater candidates, are the subjects of further processing; (ii) it selects Haar-like image texture features in combination with boosting ensemble supervised learning algorithms to accurately classify candidates into craters and non-craters; (iii) it uses transfer learning, at a minimum additional cost, to enable maintaining an accurate auto-detection of craters on new images, having morphology different from what has been captured by the original training set. All three aforementioned components of the detection methodology are discussed, and the entire framework is evaluated on a large test image of 37,500 x 56,250$ m2 on Mars, showing heavily cratered Martian terrain characterized by nonuniform surface morphology. Our study demonstrates that this methodology provides a robust and practical tool for planetary science, in terms of both detection accuracy and efficiency.

In this paper, we present a methodology for automatic geomorphic mapping of planetary surfaces that incorporates machine-learning techniques. Our application transforms remotely sensed topographic data gathered by orbiting satellites into semantically meaningful maps of landforms; such maps are valuable research tools for planetary science. As topographic data become increasingly available, the ability to derive geomorphic maps efficiently is becoming essential. In our proposed framework, the mapping is achieved by means of scene segmentation followed by supervised classification of segments. The two mapping steps use different sets of features derived from digital elevation models of planetary surfaces; selection of appropriate features is discussed. Using a particular set of terrain attributes relevant to annotating cratered terrain on Mars, we investigate the design choices for both segmentation and classification components. The segmentation assessment includesK-means-based agglomerative segmentation and watershed-based segmentation. The classification assessment includes three supervised learning algorithms: Naive Bayes, Bagging with decision trees, and support-vector machines (SVMs); segments are classified into the following landforms: crater floors, crater walls (concave and convex), ridges (concave and convex), and intercrater plains. The method is applied to six test sites on Mars. The analysis of the results shows that a combination ofK-means-based agglomerative segmentation and either SVM with a quadratic kernel or Bagging with C4.5 yields best maps. The presented framework can be adopted to generate geomorphic maps of sites on Earth.

Online learning, defined broadly as recording and delivering classroom experience with technology, has tremendous potential. However, success to date has been very limited in science and engineering. We believe this is because traditional video recording is cumbersome and not suitable for technical lectures and removing live classroom interaction is detrimental to learning. Employing Tablet PCs with slide presentation software has made it simple and convenient to develop and record high quality lectures. We employ such videos in a hybrid model of coursework. All lectures are made available as online videos, but limited classroom interaction is an important component; the classroom is used for review of lecture material, examinations, demonstrations, tutorials, and hands-on sessions. The hybrid framework is particularly suitable for students with logistical difficulties, e.g., because of work schedule. This paper is an evaluation of the hybrid learning approach as applied to upper level computer science coursework. We report our experience in teaching a suite of hybrid courses at the University of Houston and discuss the detailed feedback we received from the students who participated in the courses.

Adaptive clustering uses external feedback to improve cluster quality; past experience serves to speed up execution time. An adaptive clustering environment is proposed that uses Q-learning to learn the reward values of successive data clusterings. Adaptive clustering supports the reuse of clusterings by memorizing what worked well in the past. It has the capability of exploring multiple paths in parallel when searching for good clusters. In a case study, we apply adaptive clustering to instance-based learning relying on a distance function modification approach. A distance function adaptation scheme that uses external feedback is proposed and compared with other distance function learning approaches. Experimental results indicate that the use of adaptive clustering leads to significant improvements of instance-based learning techniques, such as k-nearest neighbor classifiers. Moreover, as a by-product a new instance-based learning technique is introduced that classifies examples by solely using cluster representatives; this technique shows high promise in our experimental evaluation.

The objective of a general-purpose content-based image retrieval system is to find images in a database that match an external measure of relevance. Since users follow different and inconsistent relevance measures, processing queries in a task-specific manner has shown to be an effective approach. Viewing specialized image retrieval algorithms as agents, we propose a general-purpose image retrieval system that uses a new multi-agent meta-learning framework. The framework adapts a distance function defined over both image distance weights and image queries to identify clusters of algorithms that produce similar solutions to similar problems. Experiments compare our approach with a traditional information retrieval algorithm; results show that our framework provides better average relevance scores

We propose to use an unsupervised automated classification of topographic features on Mars in order to speed up geomorphic and geologic mapping of the planet. We construct a digital topography model (DTM), a multilayer grid that stores various kinds of topographical information for every pixel in a site. The method uses a probabilistic clustering algorithm to assign topographically meaningful labels to all pixels in the DTM. The results are displayed as a thematic map of topography. Resultant topographical features are characterized and compared using statistics of their constituent pixels. We demonstrate the usage of our method by classifying and characterizing the topography of a landscape in the Tisia Valles region on Mars. We discuss extensions and further applications of our method.

The goal of dataset editing in instance-based learning is to remove objects from a training set in order to increase the accuracy of a classifier. For example, Wilson editing removes training examples that are misclassified by a nearest neighbor classifier so as to smooth the shape of the resulting decision boundaries. This paper revolves around the use of representative-based clustering algorithms for nearest neighbor dataset editing. We term this approach supervised clustering editing. The main idea is to replace a dataset by a set of cluster prototypes. A clustering approach called supervised clustering is introduced for this purpose. Our empirical evaluation using eight UCI datasets shows that both Wilson and supervised clustering editing improve accuracy on more than 50% of the datasets tested. However, supervised clustering editing achieves four times higher compression rates than Wilson editing.

We propose a preprocessing step to classification that applies a clustering algorithm to the training set to discover local patterns in the attribute or input space. We demonstrate how this knowledge can be exploited to enhance the predictive accuracy of simple classifiers. Our focus is mainly on classifiers characterized by high bias but low variance (e.g., linear classifiers); these classifiers experience difficulty in delineating class boundaries over the input space when a class distributes in complex ways. Decomposing classes into clusters makes the new class distribution easier to approximate and provides a viable way to reduce bias while limiting the growth in variance. Experimental results on real-world domains show an advantage in predictive accuracy when clustering is used as a preprocessing step to classification.

As the complexity of distributed computing systems increases, systems management tasks require significantly higher levels of automation; examples include diagnosis and prediction based on real-time streams of computer events, setting alarms, and performing continuous monitoring. The core of autonomic computing, a recently proposed initiative towards next-generation IT-systems capable of 'self-healing', is the ability to analyze data in real-time and to predict potential problems. The goal is to avoid catastrophic failures through prompt execution of remedial actions.This paper describes an attempt to build a proactive prediction and control system for large clusters. We collected event logs containing various system reliability, availability and serviceability (RAS) events, and system activity reports (SARs) from a 350-node cluster system for a period of one year. The 'raw' system health measurements contain a great deal of redundant event data, which is either repetitive in nature or misaligned with respect to time. We applied a filtering technique and modeled the data into a set of primary and derived variables. These variables used probabilistic networks for establishing event correlations through prediction algorithms. We also evaluated the role of time-series methods, rule-based classification algorithms and Bayesian network models in event prediction.Based on historical data, our results suggest that it is feasible to predict system performance parameters (SARs) with a high degree of accuracy using time-series models. Rule-based classification techniques can be used to extract machine-event signatures to predict critical events with up to 70% accuracy.

This article provides a characterization of bias for evaluation metrics in classification (e.g., Information Gain, Gini, χ2, etc.). Our characterization provides a uniform representation for all traditional evaluation metrics. Such representation leads naturally to a measure for the distance between the bias of two evaluation metrics. We give a practical value to our measure by observing the distance between the bias of two evaluation metrics and its correlation with differences in predictive accuracy when we compare two versions of the same learning algorithm that differ in the evaluation metric only. Experiments on real‐world domains show how the expectations on accuracy differences generated by the distance‐bias measure correlate with actual differences when the learning algorithm is simple (e.g., search for the best single feature or the best single rule). The correlation, however, weakens with more complex algorithms (e.g., learning decision trees). Our results show how interaction among learning components is a key factor to understand learning performance.

Temporal data mining aims at finding patterns in historical data. Our work proposes an approach to extract temporal patterns from data to predict the occurrence of target events, such as computer attacks on host networks, or fraudulent transactions in financial institutions. Our problem formulation exhibits two major challenges: 1) we assume events being characterized by categorical features and displaying uneven inter-arrival times; such an assumption falls outside the scope of classical time-series analysis, 2) we assume target events are highly infrequent; predictive techniques must deal with the class-imbalance problem. We propose an efficient algorithm that tackles the challenges above by transforming the event prediction problem into a search for all frequent eventsets preceding target events. The class imbalance problem is overcome by a search for patterns on the minority class exclusively; the discrimination power of patterns is then validated against other classes. Patterns are then combined into a rule-based model for prediction. Our experimental analysis indicates the types of event sequences where target events can be accurately predicted.