This paper introduces AURORA, an algorithm that builds binary classifiers to optimise the area under the ROC curve (the AUC). AURORA builds non-linear classifiers of the data by binarising the raw input features. Feature selection is performed using a fast heuristic routine, and gradient descent is used to optimise the coefficients of the selected features. Both feature selection and gradient descent are designed to be optimal for AUC. Non-differentiability of the objective function is overcome using a ramp-based approximation for the AUC. The use of this ramp-based approximation also allows the AUC to be calculated in near O(n) time, where n is the number of labelled observations available. The entire AURORA algorithm then also has computational complexity near O(n). AURORA is compared with several other classifiers, over eight binary classification tasks, and generally produces classifiers which are significantly more accurate. AURORA is also highly scalable with increasing number of training examples, and increasing number of input features.

This paper describes TIPPPS (Time Interleaved Product Purchase Prediction System), which analyses billing data of corporate customers in a large telecommunications company in order to predict high value upsell opportunities. The challenges presented by this prediction problem are significant. Firstly, the diversity of products used by corporate telecommunications customers is huge. This, coupled with low product take-up rates, makes this a problem of learning from a very high dimensional feature space with very few minority examples. Further, it is important to give priority specifically to the identification of those new customers who are of high value. These challenges are overcome by introducing a number of modifications to standard data pre-processing and machine learning algorithms, the most important of which are time-interleaving of data and value weighting. Time interleaving is the concatenation of examples from multiple time periods, thus increasing the number of training examples, and hence the number of minority examples. Value weighting assigns importance to minority examples in proportion to the dollar value of take-up, thus biasing the system to identify high value customers. These modifications create a novel algorithm that makes the prediction system practical and usable.Comparison with other techniques designed for similar problems shows that the expected average improvement in ranking accuracy achieved using these modifications is 3.7%. TIPPPS has been in operation for several months and has been successful in identifying many upsell opportunities that were not identified by using the previous manual system.

This paper introduces RankOpt, a linear binary classifier which optimises the area under the ROC curve (the AUC). Unlike standard binary classifiers, RankOpt adopts the AUC statistic as its objective function, and optimises it directly using gradient descent. The problems with using the AUC statistic as an objective function are that it is non-differentiable, and of complexity O(n2) in the number of data observations. RankOpt uses a differentiable approximation to the AUC which is accurate, and computationally efficient, being of complexity O(n.) This enables the gradient descent to be performed in reasonable time. The performance of RankOpt is compared with a number of other linear binary classifiers, over a number of different classification problems. In almost all cases it is found that the performance of RankOpt is significantly better than the other classifiers tested.

We investigate two seemingly incompatible approaches for improving document retrieval performance in the context of question answering: query expansion and query reduction. Queries are expanded by generating lexical paraphrases. Syntactic, semantic and corpus-based frequency information is used in this process. Queries are reduced by removing words that may detract from retrieval performance. Features that identify these words were obtained from decision graphs. These approaches were evaluated using a subset of queries from TREC8, 9 and 10. Our evaluation shows that each approach in isolation improves retrieval performance, and both approaches together yield substantial improvements. Specifically, query expansion followed by reduction improved the average number of correct documents retrieved by 21.7% and the average number of queries that can be answered by 15%.

In this paper, we present a new co-training strategy that makes use of unlabelled data. It trains two predictors in parallel, with each predictor labelling the unlabelled data for training the other predictor in the next round. Both predictors are support vector machines, one trained using data from the original feature space, the other trained with new features that are derived by clustering both the labelled and unlabelled data. Hence, unlike standard co-training methods, our method does not require a priori the existence of two redundant views either of which can be used for classification, nor is it dependent on the availability of two different supervised learning algorithms that complement each other.We evaluated our method with two classifiers and three text benchmarks: WebKB, Reuters newswire articles and 20 NewsGroups. Our evaluation shows that our co-training technique improves text classification accuracy especially when the number of labelled examples are very few.