Obtaining an indication of confidence of predictions is desirable for many data mining applications. Such confidence levels, together with the predicted value, can inform on the certainty or extent of reliability that may be associated with the prediction. This can be useful, for example, where model outputs are used in making potentially costly decisions, and one may then focus on the higher confidence predictions, and in general across risk sensitive applications. The conformal prediction framework presents a novel approach for complementing predictions from machine learning algorithms with valid confidence measures. Confidence levels are obtained from the underlying algorithm, using a non-conformity measure which indicates how 'atypical' a given example set is. The non-conformity measure is key to determining the usefulness and efficiency of the approach. This paper considers inductive conformal prediction in the context of random tree ensembles like random forests, which have been noted to perform favorably across problems. Focusing on classification tasks, and considering realistic data contexts including class imbalance, we develop non-conformity measures for assessing the confidence of predicted class labels from random forests. We examine the performance of these measures on multiple datasets. Results demonstrate the usefulness and validity of the measures, their relative differences, and highlight the effectiveness of conformal prediction random forests for obtaining predictions with associated confidence.

This paper considers the discovery of trading decision models from high-frequency foreign exchange (FX) markets data using genetic programming (GP). It presents a domain-related structuring of the representation and incorporation of semantic restrictions for GP-based searching of trading decision models. A defined symmetry property provides a basis for the semantics of FX trading models. The symmetry properties of basic indicator types useful in formulating trading models are defined, together with semantic restrictions governing their use in trading model specification. The semantics for trading model specification have been defined with respect to regular arithmetic, comparison and logical operators. This study also explores the use of two fitness criteria for optimization, showing more robust performance with a risk-adjusted measure of returns.

Predictive models in direct marketing seek to identify individuals most likely to respond to promotional solicitations or other intervention programs.While standard modeling approaches embody single objectives, real-world decision problems often seek multiple performance measures.Decision-makers here desire solutions that simultaneously optimize on multiple objectives, or obtain an acceptable tradeoff amongst objectives.Multi-criteria problems often characterize a range of solutions, none of which dominate the others with respect to the multiple objectives -these specify the Pareto-frontier of nondominated solutions, each offering a different level of tradeoff.This paper proposes the use of evolutionary computation based procedures for obtaining a set of nondominated models with respect to multiple stated objectives.The targeting depth-of-file presents a crucial real-world criterion in direct marketing, and models here are tailored for specified file-depths.Decision-makers are thus able to obtain a set of models along the Pareto-frontier, for a specific file-depth.The choice of a model to implement can be thus based on observed tradeoffs in the different objectives, based on possibly subjective and problem specific judgements.Given distinct models tailored for different file-depths, the implementation decision can also consider performance tradeoffs at the different depths-offile.Empirical results from a real-world problem illustrate the benefits of the proposed approach.Both linear and nonlinear models obtained by genetic search are examined.

Financial markets data present a challenging opportunity for the learning of complex patterns not otherwise discernable, and machine learning techniques like genetic algorithms have been noted to be advantageous in this regard. Independent trials of the genetic algorithm are known to explore different parts of the search space and produce solutions which potentially capture different patterns in the data. Additionally, learning in domains prone to noisy data can generate solutions which obtain performance gains by fitting to what essentially is noise in the data. The article investigates possible strategies for combining the rules obtained from independent GA trials with the objective of noise filtering or enhanced pattern detection for improving the overall learning accuracy.

Data analysts in direct marketing seek models to identify the most promising individuals to mail to and thus maximize returns from solicitations. A variety of criterion can be used to assess model performance, including response to or revenue generated from earlier solicitations. Given budgetary limitations, typically a fraction of the total customer database is selected for mailing. This depth-of-file that is to be mailed to provides potentially useful information that should be considered in model determination. This article presents a genetic algorithm-based approach for obtaining models in explicit consideration of this mailing depth. Issues related to overfitting, common in application of machine learning techniques, are examined, and experiments are based on a real-life data set.

A complete generalization of the Vose genetic algorithm model from the binary to higher cardinality case is provided. Boolean AND and EXCLUSIVE-OR operators are replaced by multiplication and addition over rings of integers. Walsh matrices are generalized with finite Fourier transforms for higher cardinality usage. Comparison of results to the binary case are provided.