The success of prediction models for business purposes should not be measured by their accuracy only. Their evaluation should also take into account the higher importance of precise prediction for "valuable" customers. We illustrate this idea through the example of churn modelling in telecommunications, where it is obviously much more important to identify potential churn among valuable customers. We discuss, both theoretically and empirically, the optimal use of "customer value" data in the model training, model evaluation and scoring stages. Our main conclusion is that a nontrivial approach of using "decayed" value-weights for training is usually preferable to the two obvious approaches of either using nondecayed customer values as weights or ignoring them.

We present and discuss the important business problem of estimating the effect of retention efforts on the Lifetime Value of a customer in the Telecommunications industry. We discuss the components of this problem, in particular customer value and length of service (or tenure) modeling, and present a novel segment-based approach, motivated by the segment-level view marketing analysts usually employ. We then describe how we build on this approach to estimate the effects of retention on Lifetime Value. Our solution has been successfully implemented in Amdocs' Business Insight (BI) platform, and we illustrate its usefulness in real-world scenarios.

We consider prediction-model evaluation in the context of marketing-campaign planning. In order to evaluate and compare models with specific campaign objectives in mind, we need to concentrate our attention on the appropriate evaluation-criteria. These should portray the model's ability to score accurately and to identify the relevant target population. In this paper we discuss some applicable model-evaluation and selection criteria, their relevance for campaign planning, their robustness under changing population distributions, and their employment when constructing confidence intervals. We illustrate our results with a case study based on our experience from several projects.

Many fraud analysis systems have at their heart a rule-based engine for generating alerts about suspicious behaviors.The rules in the system are usually based on expert knowledge.Automatic rule discovery aims at using past examples of fraudulent and legitimate usage to find new patterns and rules to help distinguish between the two.Some aspects of the problem of finding rules suitable for fraud analysis make this problem unique.Among them are the following: the need to find rules combining both the properties of the customer (e.g., credit rating) and properties of the specific "behavior" which indicates fraud (e.g., number of international calls in one day); and the need for a new definition of accuracy: We need to find rules which do not necessarily classify correctly each individual "usage sample" as either fraudulent or not, but ensure the identification, with a minimum of wasted cost and effort, of most of the fraud "cases" (i.e., defrauded customers).These aspects require a special-purpose rule discovery system.We present as an example a two-stage system based on adaptation of the C4.5 rule generator, with an additional rule selection mechanism.Our experimental results indicate that this route is very promising.