There is a useful counterpart of conformal prediction for e-values, called conformal e-prediction . Conformal prediction can serve as basis for testing the assumption of exchangeability, leading to conformal testing . Similarly, conformal e-prediction can also serve as basis for testing exchangeability. The resulting conformal e-testing looks very different from but inherits some strengths of conformal testing; it even has some advantages over conformal testing. In this paper we discuss systematically both strengths and limitations of conformal e-testing.

Conformal predictive systems are a recent modification of conformal predictors that output, in regression problems, probability distributions for labels of test observations rather than set predictions. The extra information provided by conformal predictive systems may be useful, e.g., in decision making problems. Conformal predictive systems inherit the relative computational inefficiency of conformal predictors. In this paper we discuss two computationally efficient versions of conformal predictive systems, which we call split conformal predictive systems and cross-conformal predictive systems. The main advantage of split conformal predictive systems is their guaranteed validity, whereas for cross-conformal predictive systems validity only holds empirically and in the absence of excessive randomization. The main advantage of cross-conformal predictive systems is their greater predictive efficiency.

Conformal approach to anomaly detection was recently developed as a reliable framework of classifying examples into normal and abnormal groups based on a training data set containing only normal examples. Its validity property is that a normal example, generated by the same distribution as the examples from the training set, is classified as anomaly with probability bounded from above by a pre-selected significance level. Parallel processing of big data may require a split of the training set into several sources. We also assume that the collection of data for two or more sources might be done in parallel and the data distribution may differ for these sources. The contribution of this work to conformal anomaly detection is studying the ways of keeping conformal validity when the training set is obtained from heterogeneous (differently distributed) sources.

We construct prediction intervals for the linear regression model with IID errors with a known distribution, not necessarily Gaussian. The coverage probability of our prediction intervals is equal to the nominal confidence level not only unconditionally but also conditionally given a natural sigma-algebra of invariant events. This implies, in particular, the perfect calibration of our prediction intervals in the on-line mode of prediction.

In this work we apply a new technique called conformal prediction to the Functional Clustering of Gene Expression Profiles in Human Cancers Challenge. The method not only allows us to make predictions but also include measures of accuracy and reliability of the prediction. These measures are provably valid under i. i. d. assumption. Using this approach it becomes possible to control the number of errors by selecting a suitable confidence level. This paper describes the application of the method to gene expression for various types of cancer.

Ovarian cancer is characterized by vague, non-specific symptoms, advanced stage at diagnosis and poor overall survival. A nested case control study was undertaken on stored serial serum samples from women who developed ovarian cancer and healthy controls (matched for serum processing and storage conditions as well as attributes such as age) in a pilot randomized controlled trial of ovarian cancer screening. The unique feature of this study is that the women were screened for up to 7 years. The serum samples underwent prefractionation using a reversed-phase batch extraction protocol prior to MALDI-TOF MS data acquisition. Our exploratory analysis shows that combining a single MS peak with CA125 allows statistically significant discrimination at the 5% level between cases and controls up to 12 months in advance of the original diagnosis of ovarian cancer. Such combinations work much better than a single peak or CA125 alone. This paper demonstrates that mass spectra from the low molecular weight serum proteome carry information useful for early detection of ovarian cancer. The next step is to identify the specific biomarkers that make early detection possible.

In the problem of probability forecasting the learner’s goal is to output, given a training set and a new object, a suitable probability measure on the possible values of the new object’s label. An on-line algorithm for probability forecasting is said to be well-calibrated if the probabilities it outputs agree with the observed frequencies. We give a natural non- asymptotic formalization of the notion of well-calibratedness, which we then study under the assumption of randomness (the object/label pairs are independent and identically distributed). It turns out that, although no probability forecasting algorithm is automatically well-calibrated in our sense, there exists a wide class of algorithms for “multiprobability forecasting” (such algorithms are allowed to output a set, ideally very narrow, of probability measures) which satisfy this property; we call the algorithms in this class “Venn probability machines”. Our experimental results demonstrate that a 1-Nearest Neighbor Venn probability machine performs reasonably well on a standard benchmark data set, and one of our theoretical results asserts that a simple Venn probability machine asymptotically approaches the true conditional probabilities regardless, and without knowledge, of the true probability measure generating the examples.