Modern technology integrates Cyber-Physical Systems (CPS), merging computational and physical processes. Ensuring CPS dependability is vital in averting adverse effects on critical applications due to unforeseen behaviour. To fortify CPS resilience, a novel technique for dynamic behavioural prediction and fault injection is introduced. It predicts dynamic CPS behaviour through system modelling under diverse operational scenarios, employing a fault model with diverse fault classes. Unlike the single model tenet, this approach engages multiple expert models to simulate both faultless and faulty behaviours. By adopting this approach, we can inject specialised faults and scale the analysis of the faults together or separately. Injecting faults assesses system reactions and reveals vulnerabilities. Tested on a water tank system, the approach proves effective in behaviour prediction and proactive fault handling, enhancing CPS design for robust, secure, and fault-tolerant systems.

Dimensionality reduction and clustering are often used as preliminary steps for many complex machine learning tasks. The presence of noise and outliers can deteriorate the performance of such preprocessing and therefore impair the subsequent analysis tremendously. In manifold learning, several studies indicate solutions for removing background noise or noise close to the structure when the density is substantially higher than that exhibited by the noise. However, in many applications, including astronomical datasets, the density varies alongside manifolds that are buried in a noisy background. We propose a novel method to extract manifolds in the presence of noise based on the idea of Ant colony optimization. In contrast to the existing random walk solutions, our technique captures points that are locally aligned with major directions of the manifold. Moreover, we empirically show that the biologically inspired formulation of ant pheromone reinforces this behavior enabling it to recover multiple manifolds embedded in extremely noisy data clouds. The algorithm performance in comparison to state-of-the-art approaches for noise reduction in manifold detection and clustering is demonstrated, on several synthetic and real datasets, including an N-body simulation of a cosmological volume.

In a streaming environment, the characteristics of the data themselves and their relationship with the labels are likely to experience changes as time goes on. Most drift detection methods for supervised data streams are performance-based, that is, they detect changes only after the classication accuracy deteriorates. This may not be sufcient in many application areas where the reason behind a drift is also important. Another category of drift detectors are data distribution-based detectors. Although they can detect some drifts within the input space, changes affecting only the labelling mechanism cannot be identied. Furthermore, little work is available on drift detection for high-dimensional supervised data streams. In this paper we propose an advanced Hierarchical Reduced-space Drift Detection Framework for Supervised Data Streams (HRDS) which captures drifts regardless of their effects on classication performance. This framework suggests monitoring both marginal and class-conditional distributions within a lower-dimensional space specically relevant to the assigned classication task. Experimental comparisons have demonstrated that the proposed HRDS not only achieves high-quality performance on high-dimensional data streams, but also outperforms its competitors in terms of detection recall, precision and F-measure across a wide range of different concept drift types including subtle drifts.

Cyber Physical Systems (CPS) pose a pressing need to ensure they are sufficiently reliable and continue to be dependable. It is, therefore, essential to test these systems to uncover any potential anomalies, which if not detected can lead to failure and/or cause loss or injury. Adequate or complete coverage of behaviours can be difficult to accomplish in CPS. We advocate a less expensive and easy-to-evaluate representation of the system via surrogate modelling. In this paper, we present a novel predictive fault modelling framework leveraging surrogate-based Digital Twin for probing for likely faults that can support software analysts and testers of CPS in their testing plans. The approach abstracts the CPS and uses a variant of Recurrent Neural Network known as Long Short-Term Memory (LSTM) surrogate model for forecasting. The forecasting can help in predicting multiple behaviours of the system components and the likely faults of systems under test; observations will consequently feed into the testing plans. Both direct and iterative (i.e. one-time and multiple-time varying steps) forecasting are supported as part of the framework. We evaluate our surrogate-based Digital Twins predictive modelling approach on two CPSs namely: water distribution system and air pollution detection system. The results show that our approach performed decently in predicting multiple time steps.

Noise is a challenge for process mining algorithms, but there is no standard definition of noise nor accepted way to quantify it. This means it is not possible to mine with confidence from event logs which may not record the underlying process correctly. We discuss one way of thinking about noise in process mining. We consider mining from a `noisy log' as learning a probability distribution over traces, representing the true process, from a log which is a sample from multiple distributions: the `true' process model and one or more `noise' models. We apply this using a probabilistic analysis of the Heuristics Miner algorithm, and demonstrate on a simple example. We show that for a given model it is possible to predict how much data is needed to mine the underlying model without the noise, and identify differences in the the robustness of Heuristics Miner to different types of noise.

We present novel, efficient, model based kernels for time series data rooted in the reservoir computation framework. The kernels are implemented by fitting reservoir models sharing the same fixed deterministically constructed state transition part to individual time series. The proposed kernels can naturally handle time series of different length without the need to specify a parametric model class for the time series. Compared with most time series kernels, our kernels are computationally efficient. We show how the model distances used in the kernel can be calculated analytically or efficiently estimated. The experimental results on synthetic and benchmark time series classification tasks confirm the efficiency of the proposed kernel in terms of both generalization accuracy and computational speed. This paper also investigates on-line reservoir kernel construction for extremely long time series.

An ensemble is a group of learners that work together as a committee to solve a problem. The existing ensemble learning algorithms often generate unnecessarily large ensembles, which consume extra computational resource and may degrade the generalization performance. Ensemble pruning algorithms aim to find a good subset of ensemble members to constitute a small ensemble, which saves the computational resource and performs as well as, or better than, the unpruned ensemble. This paper introduces a probabilistic ensemble pruning algorithm by choosing a set of ldquosparserdquo combination weights, most of which are zeros, to prune the ensemble. In order to obtain the set of sparse combination weights and satisfy the nonnegative constraint of the combination weights, a left-truncated, nonnegative, Gaussian prior is adopted over every combination weight. Expectation propagation (EP) algorithm is employed to approximate the posterior estimation of the weight vector. The leave-one-out (LOO) error can be obtained as a by-product in the training of EP without extra computation and is a good indication for the generalization error. Therefore, the LOO error is used together with the Bayesian evidence for model selection in this algorithm. An empirical study on several regression and classification benchmark data sets shows that our algorithm utilizes far less component learners but performs as well as, or better than, the unpruned ensemble. Our results are very competitive compared with other ensemble pruning algorithms.

Recently, we have developed the hierarchical generative topographic mapping (HGTM), an interactive method for visualization of large high-dimensional real-valued data sets. We propose a more general visualization system by extending HGTM in three ways, which allows the user to visualize a wider range of data sets and better support the model development process. 1) We integrate HGTM with noise models from the exponential family of distributions. The basic building block is the latent trait model (LTM). This enables us to visualize data of inherently discrete nature, e.g., collections of documents, in a hierarchical manner. 2) We give the user a choice of initializing the child plots of the current plot in either interactive, or automatic mode. In the interactive mode, the user selects "regions of interest", whereas in the automatic mode, an unsupervised minimum message length (MML)-inspired construction of a mixture of LTMs is employed. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. 3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualization plots, since they can highlight the boundaries between data clusters. We illustrate our approach on a toy example and evaluate it on three more complex real data sets.

There is a notable interest in extending probabilistic generative modeling principles to accommodate for more complex structured data types. In this paper we develop a generative probabilistic model for visualizing sets of discrete symbolic sequences. The model, a constrained mixture of discrete hidden Markov models, is a generalization of density-based visualization methods previously developed for static data sets. We illustrate our approach on sequences representing web-log data and chorals by J.S. Bach.