While increasingly complex algorithms are being developed for graph classification in highly-structured domains, such as image processing and climate forecasting, they often lead to over-fitting and inefficiency when applied to human interaction networks where the confluence of cooperation, conflict, and evolutionary pressures produces chaotic environments. We propose a graph transformation approach for efficient classification in chaotic human systems that is based on game theoretic, network theoretic, and chaos theoretic principles. Graph structural properties are compiled into time-series that are then transposed into the frequency domain to offer a dynamic view of the system for classification. We propose a set of benchmark data sets and show through experiments that the approach is efficient and appropriate for many dynamic networks in which agents both compete and cooperate, such as social media networks, stock markets, political campaigns, legislation, and geopolitical events.

We introduce a new inductive bias for learning in dynamic event-based human systems. This is intended to partially address the issue of deep learning in chaotic systems. Instead of fitting the data to polynomial expansions that are expressive enough to approximate the generative functions or of inducing a universal approximator to learn the patterns and inductive bias, we only assume that the relationship between the input features and output classes changes over time, and embed this assumption through a form of dynamic contrastive learning in pre-training, where pre-training labels contain information about the class labels and time periods. We do this by extending and integrating two separate forms of contrastive learning. We note that this approach is not equivalent to inserting an extra feature into the input data that contains time period, because the input data cannot contain the label. We illustrate the approach on a recently designed learning algorithm for event-based graph time-series classification, and demonstrate its value on real-world data.

In recent years, online Event Based Social Network (EBSN) platforms have become increasingly popular. One typical task of EBSN platforms is to help users make suitable and personalized plans for participating in different interesting social events. Existing techniques either ignore the minimum-participant requirement constraint for each event, which is crucially needed for some events to be held successfully, or assume that events would not change once announced. In this paper, we address the above inadequacies of existing EBSN techniques. We formally define the Global Event Planning with Constraints (GEPC) problem, and its incremental variant. Since these problems are NP-hard, and provide approximate solutions. Finally, we verify the effectiveness and efficiency of our proposed algorithms through extensive experiments over real and synthetic datasets.

Empirical evidence suggests that ensembles with adequate levels of pairwise diversity among a set of accurate member algorithms can significantly outperform any of the individual algorithms. As a result, several diversity measures have been developed for use in optimizing ensembles. We show, howeve r, that there is natural tension between the pairwise diversity of ensemble members and their individual accuracy. While efficient ensembles can be built with stronger forms of diversity, they also suffer in overall accuracy. On the other hand, ensembles built with weaker forms of diversity can be very accurate, but tend to be significantly more computationally expensive. We discuss these findings in light of the notion of diversity space.

In recent years, online Event Based Social Network (EBSN) platforms have become increasingly popular. One typical task of EBSN platforms is to help users make suitable and personalized plans for participating in different interesting social events. Existing techniques either ignore the minimum-participant requirement constraint for each event, which is crucially needed for some events to be held successfully, or assume that events would not change once announced. In this paper, we address the above inadequacies of existing EBSN techniques. We formally define the Global Event Planning with Constraints (GEPC) problem, and its incremental variant. We prove that both are NP-hard, and provide approximate solutions. Finally, we verify the effectiveness and efficiency of our proposed algorithms through extensive experiments over real and synthetic datasets.

One of the challenges of data mining is finding hyperparameters for a learning algorithm that will produce the best model for a given dataset. Hyperparameter optimization automates this process, but it can still take significant time. It has been found that hyperparameter optimization does not always result in induced models with significant improvement over default values, yet no systematic analysis of the role of hyperparameter optimization in machine learning has been conducted. We use metalearning to inform the decision of whether to optimize hyperparameters based on expected performance improvement and computational cost.

Among uncertain graph queries, reachability, i.e., the probability that one vertex is reachable from another, is likely the most fundamental one. Although this problem has been studied within the field of network reliability, solutions are implemented on a single computer and can only handle small graphs. However, as the size of graph applications continually increases, the corresponding graph data can no longer fit within a single computer's memory and must therefore be distributed across several machines. Furthermore, the computation of probabilistic reachability queries is #P-complete making it very expensive even on small graphs. In this paper, we develop an efficient distributed strategy, called DistR, to solve the problem of reachability query over large uncertain graphs. Specifically, we perform the task in two steps: distributed graph reduction and distributed consolidation. In the distributed graph reduction step, we find all of the maximal subgraphs of the original graph, whose reachability probabilities can be calculated in polynomial time, compute them and reduce the graph accordingly. After this step, only a small graph remains. In the distributed consolidation step, we transform the problem into a relational join process and provide an approximate answer to the #P-complete reachability query. Extensive experimental studies show that our distributed approach is efficient in terms of both computational and communication costs, and has high accuracy.

Traditional association rule mining algorithms often have difficulty handling questions that are implicitly related, producing rules that are very accurate but are so obvious as to be completely useless to researchers. This problem is compounded by t

We illustrate the danger of using default implementations of learning algorithms by showing that the implementation of RBF networks in the three most popular open source data mining software packages causes the algorithm to behave and perform like na

We examine the problem of identifying social circles, or sets of cohesive and mutually aware nodes surrounding an initial query set, in directed graphs where the complete graph is not known beforehand. This problem differs from local community mining, in that the query set defines the circle of interest. We explicitly handle edge direction, as in many cases relationships are not symmetric, and focus on the local context because many real-world graphs cannot be feasibly known. We outline several issues that are unique to this context, introduce a quality function to measure the value of including a particular node in an emerging social circle, and describe a greedy social circle discovery algorithm. We demonstrate the effectiveness of this approach on artificial benchmarks, large networks with topical community labels, and several real-world case studies.

Classification algorithm selection is an open research problem whose solution has tremendous value for practitioners. In recent years, metalearning has emerged as a viable approach. Unfortunately, the ratio of examples to classes is small at the meta

We argue the value of unsupervised metalearning and discuss the attendant necessity of suitable similarity, or distance, functions. We leverage the notion of diversity among learners used in ensemble learning to design a distance function for the clustering of learning algorithms. We revisit the mo st popular measures of diversity and show that only one of them, Classifier Output Difference (COD) is a metric. We then use COD to produce a clustering of 21 learning algorithms, and show how this clustering differs from a clustering based on accuracy, and how it can be used to highlight interesting, sometimes unexpected, similarities among learning algorithms.

Abstract We show how a quantitative context may be established for what is essentially qualitative in nature by topologically embedding a lexicon (here, WordNet) in a complete metric space. This novel transformation establishes a natural connection between the order relation in the lexicon (e.g., hyponymy) and the notion of distance in the metric space, giving rise to effective word-level and document-level lexical semantic distance measures. We provide a formal account of the topological transformation and demonstrate the value of our metrics on several experiments involving information retrieval and document clustering tasks.

The systematic collection and organization of routine medical care data, together with data mining techniques, offer a rich environment for valuable, large-scale observational studies. We report on one such study in the context of bariatric surgery,

Using decision trees that split on randomly selected attributes is one way to increase the diversity within an ensemble of decision trees. Another approach increases diversity by combining multiple tree algorithms. The random forest approach has become popular because it is simple and yields good results with common datasets. We present a technique that combines heterogeneous tree algorithms and contrast it with homogeneous forest algorithms. Our results indicate that random forests do poorly when faced with irrelevant attributes, while our heterogeneous technique handles them robustly. Further, we show that large ensembles of random trees are more susceptible to diminishing returns than our technique. We are able to obtain better results across a large number of common datasets with a significantly smaller ensemble.

We report on three distinct experiments that provide new valuable insights into learning algorithms and datasets. We first describe two effective meta-features that significantly impact the predictive accuracy of a broad range of learning algorithms. We then introduce a new efficient meta-feature that measures the degree of hardness (or difficulty) of datasets and show that it is highly linearly correlated with predictive accuracy. Finally, we use the notion of classifier output difference to cluster learning algorithms and show that learning algorithms from different model classes may demonstrate highly similar behaviors.

We revisit 26 meta-features typically used in the context of meta-learning for model selection. Using visual analysis and computational complexity considerations, we find 4 meta-features whose values are directly relevant to certain ranges of predictive accuracy for 7 learning algorithms on 135 UCI datasets. Discretization of these 4 meta-features based on thresholds derived from our analysis significantly boosts the accuracy of the meta-level classification task.

Most research in machine learning focuses on scenarios in which a learner faces a single learning task, independently of other learning tasks or prior knowledge. In reality, however, learning is not performed in isolation, starting from scratch with every new task. Instead, it is a lifelong activity during which a learner encounters many learning tasks, and usefully transfers to new tasks knowledge acquired from earlier related tasks. We propose a novel approach to transfer learning with decision trees. Our system learns a new task semi-incrementally from a partial decision tree model which captures knowledge from a previous task. Empirical results on several UCI data sets show that our approach is generally more effective and accurate than the base approach.

Many interesting real-world applications for temporal data mining are hindered by concept drift. One particular form of concept drift is characterized by changes to the underlying feature space. Seemingly little has been done in this area. This paper presents FAE, an incremental ensemble approach to mining data subject to such concept drift. Empirical results on large data streams demonstrate promise.

Most partitional clustering algorithms require the number of desired clusters to be set a priori. Not only is this somewhat counter-intuitive, it is also difficult except in the simplest of situations. By contrast, hierarchical clustering may create partitions with varying numbers of clusters. The actual final partition depends on a threshold placed on the similarity measure used. Given a cluster quality metric, one can efficiently discover an appropriate threshold through a form of semi-supervised learning. This paper shows one such solution for complete-link hierarchical agglomerative clustering using the F-measure and a small subset of labeled examples. Empirical evaluation demonstrates promise

This paper shows how the notion of value of cooperation, a measure of the percentage of a firm's profits due strictly to the cooperative effects among the goods it sells, can be used to analyze the relative economic advantage afforded by various organizational structures. The value of cooperation is computed from transactions data by solving a regression problem to fit the parameters of the consumer demand function, and then simulating the resulting profit-maximizing dynamic system under various organizational structures. A hierarchical agglomerative clustering algorithm can be applied to reveal the optimal organizational substructure.

In order to make machine learning algorithms more usable, our community must be able to design robust systems that offer support to practitioners. In the context of classification, this amounts to developing assistants, which deal with the increasing number of models and techniques, and give advice dynamically on such issues as model selection and method combination. This paper briefly reviews the potential of meta-learning in this context and reports on the early success of a Web-based data mining assistant.

Examples and concepts in traditional concept learning tasks are represented with the attribute-value language. While enabling efficient implementations, we argue that such propositional representation is inadequate when data is rich in structure. This paper describes STEPS, a strongly-typed evolutionary programming system designed to induce concepts from structured data. STEPS higher-order logic representation language enhances expressiveness, while the use of evolutionary computation dampens the effects of the corresponding explosion of the search space. Results on the PTE2 challenge, a major real-world knowledge discovery application from the molecular biology domain, demonstrate promise.

This paper presents a biologically plausible mechanism of backpropagation network output error to previous layers of processing in a particular multilayer neural network. This mechanism is used in a network that is designed to mimic familiarity discrimination as performed by the perirhinal cortex of the temporal lobe. In the algorithm, the error of the network during an initial classification period regulates the frequency of neuronal activity in a succeeding memorising period via an inhibitory circuit, such that the frequency in this memorising period is proportional to the error. Synaptic weight modifications are made according to activity-dependent Hebbian rules, such as may be used in the brain. The magnitude of the modification depends on the frequency of the activity. Hence, the magnitude of weight modification is proportional to the network error.

Olshausen & Field demonstrated that a learning algorithm that attempts to generate a sparse code for natural scenes develops a complete family of localised, oriented, bandpass receptive fields, similar to those of 'simple cells' in VI. This paper describes an algorithm which finds a sparse code for sequences of images that preserves information about the input. This algorithm when trained on natural video sequences develops bases representing the movement in particular directions with particular speeds, similar to the receptive fields of the movement-sensitive cells observed in cortical visual areas. Furthermore, to previous approaches to learning direction selectivity, the timing of neuronal activity encodes the phase of the movement, so the precise timing of spikes is crucially important to the information encoding.

This paper addresses the problem of sensor selection during an automatic task, for examples, a process of data fusion, a sensing task or the design of a perceptual system for a mobile robot. The authors propose an approach based on geometrical interaction between a sensor and an environment, this approach is enlarged with Gaussian approximation to take into account the measurement noise. The Bayes reasoning allows one to estimate the information given by the multi-sensor system for a given scene. Our model takes into account the number of use of each sensor. This characteristic interests us in two ways: it is possible to discard useless sensors, and it is possible to estimate the acquisition delay for a given multi-sensor system site. With this approach, we propose a quadratic criterion which is able to describe the distance between the desired information and an available information. The effectiveness of this procedure is illustrated with an example of application concerning the sensor placement problem.

Learning and reasoning are both aspects of what is considered to be intelligence. Their studies within AI have been separated historically, learning being the topic of machine learning and neural networks, and reasoning falling under classical (or symbolic) AI. However, learning and reasoning are in many ways interdependent. This paper discusses the nature of some of these interdependencies and proposes a general framework called FLARE, that combines inductive learning using prior knowledge together with reasoning in a propositional setting. Several examples that test the framework are presented, including classical induction, many important reasoning protocols and two simple expert systems.