Relational learning in networked data has been shown to be effective in a number of studies. Relational learners, composed of relational classifiers and collective inference methods, enable the inference of nodes in a network given the existence and strength of links to other nodes. These methods have been adapted to predict customer churn in telecommunication companies showing that incorporating them may give more accurate predictions. In this research, the performance of a variety of relational learners is compared by applying them to a number of CDR datasets originating from the telecommunication industry, with the goal to rank them as a whole and investigate the effects of relational classifiers and collective inference methods separately. Our results show that collective inference methods do not improve the performance of relational classifiers and the best performing relational classifier is the network-only link-based classifier, which builds a logistic model using link-based measures for the nodes in the network.

The field of process mining deals with the extraction of knowledge from event logs. One task within the area of process mining entails the discovery of process models to represent real-life behavior as observed in day-to-day business activities. A large number of such process discovery algorithms have been proposed during the course of the past decade, among which techniques to mine declarative process models (e.g. Declare and AGNEs Miner) as well as evolutionary based techniques (e.g. Genetic Miner and Process Tree Miner). In this paper, we present the initial results of a newly proposed evolutionary based process discovery algorithm which aims to discover declarative process models, hence combining these two classes (declarative and genetic) of discovery techniques. To do so, we herein use a language bias similar to the one found in AGNEs Miner to allow for the conversion from a set of declarative control-flow based constraints (determining the conditions which have to be satisfied to enable to execution of an activity) to a procedural process model, i.e. a Petri net, though this language bias can be extended to include data-based constraints as well.

Process mining encompasses the research area which is concerned with knowledge discovery from event logs. One common process mining task focuses on conformance checking, comparing discovered or designed process models with actual real-life behavior as captured in event logs in order to assess the “goodness” of the process model. This paper introduces a novel conformance checking method to measure how well a process model performs in terms of precision and generalization with respect to the actual executions of a process as recorded in an event log. Our approach differs from related work in the sense that we apply the concept of so-called weighted artificial negative events toward conformance checking, leading to more robust results, especially when dealing with less complete event logs that only contain a subset of all possible process execution behavior. In addition, our technique offers a novel way to estimate a process model's ability to generalize. Existing literature has focused mainly on the fitness (recall) and precision (appropriateness) of process models, whereas generalization has been much more difficult to estimate. The described algorithms are implemented in a number of ProM plugins, and a Petri net conformance checking tool was developed to inspect process model conformance in a visual manner.

Process mining encompasses the research area which is concerned with knowledge discovery from information system event logs. Within the process mining research area, two prominent tasks can be discerned. First of all, process discovery deals with the automatic construction of a process model out of an event log. Secondly, conformance checking focuses on the assessment of the quality of a discovered or designed process model in respect to the actual behavior as captured in event logs. Hereto, multiple techniques and metrics have been developed and described in the literature. However, the process mining domain still lacks a comprehensive framework for assessing the goodness of a process model from a quantitative perspective. In this study, we describe the architecture of an extensible framework within ProM, allowing for the consistent, comparative and repeatable calculation of conformance metrics. For the development and assessment of both process discovery as well as conformance techniques, such a framework is considered greatly valuable.

Process discovery is the learning task that entails the construction of process models from event logs of information systems. Typically, these event logs are large data sets that contain the process executions by registering what activity has taken place at a certain moment in time. By far the most arduous challenge for process discovery algorithms consists of tackling the problem of accurate and comprehensible knowledge discovery from highly flexible environments. Event logs from such flexible systems often contain a large variety of process executions which makes the application of process mining most interesting. However, simply applying existing process discovery techniques will often yield highly incomprehensible process models because of their inaccuracy and complexity. With respect to resolving this problem, trace clustering is one very interesting approach since it allows to split up an existing event log so as to facilitate the knowledge discovery process. In this paper, we propose a novel trace clustering technique that significantly differs from previous approaches. Above all, it starts from the observation that currently available techniques suffer from a large divergence between the clustering bias and the evaluation bias. By employing an active learning inspired approach, this bias divergence is solved. In an assessment using four complex, real-life event logs, it is shown that our technique significantly outperforms currently available trace clustering techniques.

Recent years have witnessed the ability to gather an enormous amount of data in a large number of domains. Also in the field of business process management, there exists an urgent need to beneficially use these data to retrieve actionable knowledge about the actual way of working in the context of a certain business process. The research field concerned is process mining, which can be defined as a whole family of analysis techniques for extracting knowledge from information system event logs. In this paper, we present a solution strategy to leverage traditional process discovery techniques in the flexible environment of incident management processes. In such environments, it is typically observed that single model discovery techniques are incapable of dealing with the large number of different types of execution traces. Accordingly, we propose a combination of trace clustering and text mining to enhance process discovery techniques with the purpose of retrieving more useful insights from process data.

Within process mining research, one of the most important fields of study is process discovery, which can be defined as the extraction of control-flow models from audit trails or information system event logs. The evaluation of discovered process models is an essential but difficult task for any process discovery analysis. With this paper, we propose a novel approach for evaluating discovered process models based on artificially generated negative events. This approach allows for the definition of a behavioral F-measure for discovered process models, which is the main contribution of this paper.

Various benchmarking studies have shown that artificial neural networks and support vector machines often have superior performance when compared to more traditional machine learning techniques. The main resistance against these newer techniques is based on their lack of interpretability: it is difficult for the human analyst to understand the reasoning behind these models' decisions. Various rule extraction (RE) techniques have been proposed to overcome this opacity restriction. These techniques are able to represent the behavior of the complex model with a set of easily understandable rules. However, most of the existing RE techniques can only be applied under limited circumstances, e.g., they assume that all inputs are categorical or can only be applied if the black-box model is a neural network. In this paper, we present Minerva, which is a new algorithm for RE. The main advantage of Minerva is its ability to extract a set of rules from any type of black-box model. Experiments show that the extracted models perform well in comparison with various other rule and decision tree learners.

Ant colony optimization (ACO) can be applied to the data mining field to extract rule-based classifiers. The aim of this paper is twofold. On the one hand, we provide an overview of previous ant-based approaches to the classification task and compare them with state-of-the-art classification techniques, such as C4.5, RIPPER, and support vector machines in a benchmark study. On the other hand, a new ant-based classification technique is proposed, named AntMiner+. The key differences between the proposed AntMiner+ and previous AntMiner versions are the usage of the better performing MAX-MIN ant system, a clearly defined and augmented environment for the ants to walk through, with the inclusion of the class variable to handle multiclass problems, and the ability to include interval rules in the rule list. Furthermore, the commonly encountered problem in ACO of setting system parameters is dealt with in an automated, dynamic manner. Our benchmarking experiments show an AntMiner+ accuracy that is superior to that obtained by the other AntMiner versions, and competitive or better than the results achieved by the compared classification techniques.

We address the problem of credit scoring as a classification and feature subset selection problem. Based on the current framework of sophisticated feature selection methods, we identify features that contain the most relevant information to distinguish good loan payers from bad loan payers. The feature selection methods are validated on several real-world datasets with different types of classifiers. We show the advantages following from using the subspace approach to classification. We discuss many practical issues related to the applicability of feature selection methods. We show and discuss some difficulties that used to be insufficiently emphasized in standard feature selection literature. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 985–999, 2005.

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.

Classification algorithms like linear discriminant analysis and logistic regression are popular linear techniques for modelling and predicting corporate distress. These techniques aim at finding an optimal linear combination of explanatory input variables, such as, e.g., solvency and liquidity ratios, in order to analyse, model and predict corporate default risk. Recently, performant kernel based nonlinear classification techniques, like support vector machines, least squares support vector machines and kernel fisher discriminant analysis, have been developed. Basically, these methods map the inputs first in a nonlinear way to a high dimensional kernel-induced feature space, in which a linear classifier is constructed in the second step. Practical expressions are obtained in the so-called dual space by application of Mercer's theorem. In this paper, we explain the relations between linear and nonlinear kernel based classification and illustrate their performance on predicting bankruptcy of mid-cap firms in Belgium and the Netherlands.

In this paper, we evaluate and contrast two types of fuzzy classifiers for credit scoring. The first classifier uses evolutionary optimization and boosting for learning fuzzy classification rules. The second classifier is a fuzzy neural network that employs a fuzzy variant of the classic backpropagation learning algorithm. The experiments are carried out on a real life credit scoring data set. It is shown that, for the case at hand, the boosted genetic fuzzy classifier performs better than both the neurofuzzy classifier and the well-known C4.5(rules) decision tree(rules) induction algorithm. However, the better performance of the genetic fuzzy classifier is offset by the fact that it infers approximate fuzzy rules which are less comprehensible for humans than the descriptive fuzzy rules inferred by the neurofuzzy classifier. © 2002 Wiley Periodicals, Inc.

We study the problem of repeat-purchase modeling in a direct marketing setting using Belgian data. More specifically, we investigate the detection and qualification of the most relevant explanatory variables for predicting purchase incidence. The analysis is based on a wrapped form of input selection using a sensitivity based pruning heuristic to guide a greedy, stepwise, and backward traversal of the input space. For this purpose, we make use of a powerful and promising least squares support vector machine (LS-SVM) classifier formulation. This study extends beyond the standard recency frequency monetary (RFM) modeling semantics in two ways: (1) by including alternative operationalizations of the RFM variables, and (2) by adding several other (non-RFM) predictors. Results indicate that elimination of redundant/irrelevant inputs allows significant reduction of model complexity. The empirical findings also highlight the importance of frequency and monetary variables, while the recency variable category seems to be of somewhat lesser importance to the case at hand. Results also point to the added value of including non-RFM variables for improving customer profiling. More specifically, customer/company interaction, measured using indicators of information requests and complaints, and merchandise returns provide additional predictive power to purchase incidence modeling for database marketing. © 2001 John Wiley & Sons, Inc.

We discuss the implementation of a wrapped neural network feature selection approach, introduced here as the weight cascaded retraining (WCR) algorithm. The paper provides an outline of the algorithm and elaborates on its formal underpinnings. Central to the whole feature pruning approach is the iteratively conceived guided function optimisation realised by passing the optimised weight vector from one iteration step to the next. This essentially gives rise to a cascaded form of neural network retraining. The theoretical exposition of the WCR algorithm is illuminated and benchmarked by means of the publicly available UCI case material. It is illustrated that WCR based neural network feature selection may be very effective in reducing model complexity for classification modelling via neural networks.

Recently, a modified version of support vector machines (SVMs), least-squares SVM (LS-SVM) classifiers, has been introduced, which is closely related to a form of ridge regression-type SVMs. In LS-SVMs, the classifier is obtained as the solution to a linear system instead of a quadratic programming problem. In this paper, UCI (University of California at Irvine) benchmark data sets are used to evaluate the performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. The hyperparameters of the LS-SVM problem formulation are tuned using a 10-fold cross-validation procedure and a grid search mechanism. When comparing the performance of a nonlinear (RBF or polynomial) LS-SVM classifier with that of a linear LS-SVM, additional insight can be gained into the degree of nonlinearity of the classification problem at hand. Using a statistical motivation, it is concluded that RBF LS-SVM classifiers consistently yield among the best results for each data set.

As of this writing, there exists a large variety of recently developed pattern classification methods coming from the domain of machine learning and artificial intelligence. In this paper, we study the performance of a recently developed and improved classifier that integrates fuzzy set theory in a neural network (NEFCLASS). The performance of NEFCLASS is compared to a well-known classification technique from machine learning (C4.5). Both C4.5 and NEFCLASS will be evaluated on a collection of benchmarking data sets. Further, to boost performance of NEFCLASS, we investigate the advantage of preprocessing the algorithm by means of an exploratory factor analysis. We compare the algorithms before and after applying an exploratory factor analysis on leading performance indicators, as there are the accuracy of the created classifier and the magnitude of the associated rule base. © 2000 John Wiley & Sons, Inc.

Recently, there has been a growing interest in the maintenance and efficiency of large knowledge based systems. Decomposition of knowledge based systems is recognized as an important research issue in this respect. We discuss the decomposition of knowledge bases that consist of decision tables. Several algorithms to decompose large decision tables into smaller components are proposed.

In this article different aspects of the decision table formalism are discussed. First, crisp decision tables are defined and their construction is described. Next, fuzzy extensions are made to crisp decision tables in order to deal with imprecision and uncertainty. As a result, with crisp decision tables as special cases, a form of fuzzy decision tables is defined which include fuzziness in the conditions as well as in the actions. Consequently, the concept of completeness is introduced in the context of fuzzy decision tables. Furthermore, fuzzy consultation of decision tables is discussed, which allows decision making with fuzziness based on the matching between fuzzy conditions and the concept of fuzzy logical implication. © 1996 John Wiley & Sons, Inc.

Rule bases are commonly acquired, by an expert and/or knowledge engineer, in a form which is well-suited for acquisition purposes. When the knowledge base is executed, however, a different structure may be required. Moreover, since human experts normally do not provide the knowledge in compact chunks, rule bases often suffer from redundancy. This may considerably harm their efficiency. In this paper, a procedure is examined to transform rules that are specified in the knowledge acquisition process into an efficient rule base by way of decision tables. This transformation algorithm allows the generation of a minimal rule representation of the knowledge, and verification and optimization of rule bases and other specifications (e.g. legal texts, procedural descriptions). The proposed procedures are fully supported by the PROLOGA tool.

This paper focuses on three different formalisms which play a major role in the development of knowledge based systems, decision trees, decision tables and rules, and how the formalisms appear in the main areas of knowledge acquisition, knowledge representation and knowledge implementation. It demonstrates that the decision tables, trees or rules in the distinct stages serve different purposes in the development of knowledge based systems and therefore will not necessarily remain unchanged. Restructuring and even optimizing representation formalisms is therefore necessary. Transitions and optimizations of the different formalisms are described in the context of their automation in the Prologa workbench.>

Knowledge based systems can be of great use to lawyers in many different areas. Within the framework of the INFOSOC project implemented at the Katholieke Universiteit Leuven, legal knowledge based systems were developed to achieve some of these purposes, using a methodology based on the decision table technique, and a decision table engineering workbench, PROLOGA.This paper describes the adopted methodology, which is still being refined, the tools used and the experiences in developing a concrete system: HANDIPAK, a knowledge based system with regard to financial benefits for the disabled in Belgium.Developing HANDIPAK showed that the decision table technique was not only very useful for testing the consistency of legal knowledge, but also for supporting the acquisition and the representation process of that knowledge.

The authors explain how automated decision table construction and interfacing can prevent a great deal of the current problems with knowledge based systems, viz, lack of adequate design methodologies, lack of validation and verification support, maintenance problems. A decision table engineering workbench, that addresses these issues of decision table modeling and interfacing is presented.