Multi-class classification methods that produce sets of probabilistic classifiers, such as ensemble learning methods, are able to model aleatoric and epistemic uncertainty. Aleatoric uncertainty is then typically quantified via the Bayes error, and epistemic uncertainty via the size of the set. In this paper, we extend the notion of calibration, which is commonly used to evaluate the validity of the aleatoric uncertainty representation of a single probabilistic classifier, to assess the validity of an epistemic uncertainty representation obtained by sets of probabilistic classifiers. Broadly speaking, we call a set of probabilistic classifiers calibrated if one can find a calibrated convex combination of these classifiers. To evaluate this notion of calibration, we propose a novel nonparametric calibration test that generalizes an existing test for single probabilistic classifiers to the case of sets of probabilistic classifiers. Making use of this test, we empirically show that ensembles of deep neural networks are often not well calibrated.

Modelling real-world time series can be challenging in the absence of sufficient data. Limited data in healthcare, can arise for several reasons, namely when the number of subjects is insufficient or the observed time series is irregularly sampled at a very low sampling frequency. This is especially true when attempting to develop personalised models, as there are typically few data points available for training from an individual subject. Furthermore, the need for early prediction (as is often the case in healthcare applications) amplifies the problem of limited availability of data. This article proposes a novel personalised technique that can be learned in the absence of sufficient data for early prediction in time series. Our novelty lies in the development of a subset selection approach to select time series that share temporal similarities with the time series of interest, commonly known as the test time series. Then, a Gaussian processes-based model is learned using the existing test data and the chosen subset to produce personalised predictions for the test subject. We will conduct experiments with univariate and multivariate data from real-world healthcare applications to show that our strategy outperforms the state-of-the-art by around 20%.

Early gestational weight gain prediction can help expecting women overcome several associated risks. However, training the model requires access to centrally stored privacy sensitive weight and other meta-data. In this demo, we present a privacy preserving federated learning approach where we train a global weight gain prediction model by aggregating client models trained locally on their personal data. We showcase a software data-exploration tool that exhibits local model generation, sharing and updating across users and server for proposed collaborative learning. Our proposed model predicts the final weight category with 61.3% accuracy on day 140, with a 8.8% compromise on the centralized training accuracy.

Novelty detection or one-class classification starts from a model describing some type of `normal behaviour' and aims to classify deviations from this model as being either novelties or anomalies. In this paper the problem of novelty detection for point patterns $S=\{\mathbf{x}_1,\ldots ,\mathbf{x}_k\}\subset \mathbb{R}^d$ is treated where examples of anomalies are very sparse, or even absent. The latter complicates the tuning of hyperparameters in models commonly used for novelty detection, such as one-class support vector machines and hidden Markov models. To this end, the use of extreme value statistics is introduced to estimate explicitly a model for the abnormal class by means of extrapolation from a statistical model $X$ for the normal class. We show how multiple types of information obtained from any available extreme instances of $S$ can be combined to reduce the high false-alarm rate that is typically encountered when classes are strongly imbalanced, as often occurs in the one-class setting (whereby `abnormal' data are often scarce). The approach is illustrated using simulated data and then a real-life application is used as an exemplar, whereby accelerometry data from epileptic seizures are analysed - these are known to be extreme and rare with respect to normal accelerometer data.

Anomaly detection starts from a model of normal behavior and classifies departures from this model as anomalies. This paper introduces a statistical non-parametric approach for anomaly detection that is based on a multivariate extension of the Poisson point process model for univariate extremes. The method is demonstrated on both a synthetic and a real-world data set, the latter being an unbalanced data set of acceleration data collected from movements of 7 pediatric patients suffering from epilepsy that is previously studied in [1]. The positive predictive values could be improved with an increase up to 12.9% (and a mean of 7%) while the sensitivity scores stayed unaltered. The proposed method was also shown to outperform an one-class SVM classifier. Because the Poisson point process model of extremes is able to combine information on the number of excesses over a fixed threshold with that on the excess values, a powerful model to detect anomalies is obtained that can be of high value in many applications.

Nocturnal home monitoring of epileptic children is often not feasible due to the cumbersome manner of seizure monitoring with the standard method of video/EEG-monitoring. We propose a method for hypermotor seizure detection based on accelerometers attached to the extremities. From the acceleration signals, multiple temporal, frequency, and wavelet-based features are extracted. After determining the features with the highest discriminative power, we classify movement events in epileptic and nonepileptic movements. This classification is only based on a nonparametric estimate of the probability density function of normal movements. Such approach allows us to build patient-specific models to classify movement data without the need for seizure data that are rarely available. If, in the test phase, the probability of a data point (event) is lower than a threshold, this event is considered to be an epileptic seizure; otherwise, it is considered as a normal nocturnal movement event. The mean performance over seven patients gives a sensitivity of 95.24% and a positive predictive value of 60.04%. However, there is a noticeable interpatient difference.