Current continual learning techniques are challenged by the harsh learning conditions that are frequently imposed by real-world environments, such as distributional shifts, feature evolution, label scarcity, imbalance, noise, and novel or recurring classes. Current research typically tackles these problems separately, which restricts its relevance to dynamic and uncertain situations. This survey offers a thorough analysis of continual learning in such challenging circumstances and presents self-adaptive learning as a broad conceptual framework to bring these initiatives together. Instead of suggesting a particular algorithm, we present self-adaptive learning as an approach where a learning system recognizes environmental or data deficiencies on its own and modifies its learning behavior accordingly. We identify commonalities, classify methodological developments, and delineate unresolved issues in the development of resilient, self-regulating continual learners for practical applications by arranging previous research around this adaptive perspective.

The serverless nature of Decentralized Federated Learning (DFL) requires allocating the aggregation role to specific participants in each federated round. Current DFL architectures ensure the trustworthiness of the aggregator node upon selection. However, most of these studies overlook the possibility that the aggregating node may turn rogue and act maliciously after being nominated. To address this problem, this paper proposes a DFL structure, calledTrustChain, that scores the aggregators before selection based on their past behavior and additionally audits them after the aggregation. To do this, the statistical independence between the client updates and the aggregated model is continuously monitored using the Hilbert-Schmidt Independence Criterion (HSIC). The proposed method relies on several principles, including blockchain, anomaly detection, and concept drift analysis. The designed structure is evaluated on several federated datasets and attack scenarios with different numbers of Byzantine nodes.

Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comprehensive review of FCL, focusing on key challenges such as heterogeneity, model stability, communication overhead, and privacy preservation. We explore various forms of heterogeneity and their impact on model performance. Solutions to non-IID data, resource-constrained platforms, and personalized learning are reviewed in an effort to show the complexities of handling heterogeneous data distributions. Next, we review techniques for ensuring model stability and avoiding catastrophic forgetting, which are critical in non-stationary environments. Privacy-preserving techniques are another aspect of FCL that have been reviewed in this work. This survey has integrated insights from federated learning and continual learning to present strategies for improving the efficacy and scalability of FCL systems, making it applicable to a wide range of real-world scenarios.

Intrusion detection systems are primarily designed to flag security breaches upon their occurrence. These systems operate under the assumption of single-label data, where each instance is assigned to a single category. However, when dealing with complex data, such as malware triage, the information provided by the IDS is limited. Consequently, additional analysis becomes necessary, leading to delays and incurring additional computational costs. Existing solutions to this problem typically merge these steps by considering a unified, but large, label set encompassing both intrusion and analytical labels, which adversely affects efficiency and performance. To address these challenges, this paper presents a novel framework for multi-label classification by employing an ensemble of sequential models that preserve the original label sets during training. Each model focuses on learning the distribution specifically related to its assigned set of labels, independent of the other label sets. To capture the relationship between different sets of labels, the parameters of each trained model initialize the subsequent model, ensuring that information from unrelated label sets does not interfere with the learning objective. Consequently, the proposed method enhances prediction performance without increasing computational complexity. To evaluate the effectiveness of our approach, we conduct experiments on a real-world dataset related to intrusion detection. The results clearly demonstrate the effectiveness of our proposed method in handling multi-label classification tasks.

Federated learning has been rapidly evolving and gaining popularity in recent years due to its privacy-preserving features, among other advantages. Nevertheless, the exchange of model updates and gradients in this architecture provides new attack surfaces for malicious users of the network which may jeopardize the model performance and user and data privacy. For this reason, one of the main motivations for decentralized federated learning is to eliminate server-related threats by removing the server from the network and compensating for it through technologies such as blockchain. However, this advantage comes at the cost of challenging the system with new privacy threats. Thus, performing a thorough security analysis in this new paradigm is necessary. This survey studies possible variations of threats and adversaries in decentralized federated learning and overviews the potential defense mechanisms. Trustability and verifiability of decentralized federated learning are also considered in this study.

Emerging data-driven technologies and big data analytics generate and deal with high-dimensional data. Transformation of such data into a low-dimensional feature space brings about numerous benefits, such as a more discriminant feature space, performance enhancement, less computational burden, and facilitating data visualization. This paper proposes a novel dimensionality reduction algorithm based on generative adversarial networks to tackle the issues related to high-dimensional data and common challenges in dimensionality reduction. To this aim, two constraints are defined to preserve the characteristics of the original data while rectifying the data distribution upon transformation. Formulating the transformation as sequential projections, the proposed Constrained Adversarial Dimensionality Reduction (CADR) method finds a set of sequential projection vectors that lead to a feature space in which between-class separability and within-class integrity are satisfied. This is while the transformed data perfectly comply with the pairwise affinity correlation in the original feature space. To evaluate the proposed method, nine advanced dimensionality reduction techniques are employed to enable a comparative study. The experiments are performed on several real-world benchmark datasets in terms of classification accuracy, F-measure, and G-mean. The obtained results show that the CADR could yield classification performance at a satisfactory level and outperforms the other competitors.

Many efforts have been dedicated to addressing data loss in various domains. While task-specific solutions may eliminate the respective issue in certain applications, finding a generic method for missing data estimation is rather complex. In this regard, this article proposes a novel missing data imputation algorithm, which has supreme generalization ability for a vast variety of applications. Making use of both complete and incomplete parts of data, the proposed algorithm reduces the effect of missing ratio, which makes it suitable for situations with very high missing ratios. In addition, this feature enables model construction on incomplete training sets, which is rarely addressed in the literature. Moreover, the nonparametric nature of this new algorithm brings about supreme flexibility against all variations of missing values and data distribution. We incorporate the advantages of denoising autoencoders and ladder architecture into a novel formulation based on deep neural networks. To evaluate the proposed algorithm, a comparative study is performed using a number of reputable imputation techniques. In this process, real-world benchmark datasets from different domains are selected. On top of that, a real cyber-physical system is also evaluated to study the generalization ability of the proposed algorithm for distinct applications. To do so, we conduct studies based on three missing data mechanisms, namely: 1) missing completely at random; 2) missing at random; and 3) missing not at random. The attained results indicate the superiority of the proposed method in these experiments.

There has been an increasing interest in the use of data-driven techniques for classifying cyber-attacks and physical faults in cyber-physical systems. In real-world applications, the number of cyber-attack and faulty samples is usually far less than normal samples. This causes the skewed class distribution in data collected from cyber-physical systems. Training an accurate predictive model under skewed class conditions is not an easy task. In this work, we introduce a new generative adversarial framework for learning from skewed class distributions. This novel Adversarial Class-Imbalance Learning (ACIL) scheme has a novel loss function that is used during the adversarial training session. ACIL tries to iteratively adjust weights of an auxiliary multilayer perceptron to learn the minority class (i.e., cyber-attacks and physical faults) distributions along with the majority class (i.e., normal) distribution. Moreover, we devise an inclusive data-driven scheme for classifying cyber-attacks and faults, which includes four experiments of a baseline, nine state-of-the-art class-imbalance learning methods, two different generative-adversarial network-based approaches, and ACIL. These techniques are verified and compared through several experimental cyber-physical power scenarios. The obtained results show the effectiveness of ACIL for classifying samples of cyber-attacks and faults with skewed class distributions.

This paper presents a novel data-driven framework to aid in system state estimation when the power system is under unobservable false data injection attacks. The proposed framework dynamically detects and classifies false data injection attacks. Then, it retrieves the control signal using the acquired information. This process is accomplished in three main modules, with novel designs, for detection, classification, and control signal retrieval. The detection module monitors historical changes of phasor measurements and captures any deviation pattern caused by an attack on a complex plane. This approach can help to reveal characteristics of the attacks including the direction, magnitude, and ratio of the injected false data. Using this information, the signal retrieval module can easily recover the original control signal and remove the injected false data. Further information regarding the attack type can be obtained through the classifier module. The proposed ensemble learner is compatible with harsh learning conditions including the lack of labeled data, concept drift, concept evolution, recurring classes, and independence to external updates. The proposed novel classifier can dynamically learn from data and classify attacks under all these harsh learning conditions. The introduced framework is evaluated w.r.t. real-world data captured from the Central New York Power System. The obtained results indicate the efficacy and stability of the proposed framework.

The performance of intrusion classification systems is often hampered by the presence of missing values in data collected from cyber-physical systems. Therefore, it is of paramount importance to robustly handle such missing scores, which in turn enhances the efficiency of intrusion classification task, and, consequently, the cybersecurity of cyber-physical systems. To this aim, this paper studies the efficacy of missing data imputation techniques for safeguarding intrusion classification systems against missing scores. To do this, a hybrid intrusion classification system is designed that comprises several advanced imputation techniques. To evaluate this intrusion classification framework, various incomplete scenarios have been simulated from data collected from a cyber-physical water system. In total, forty-four incomplete scenarios are considered throughout the experiments. The evaluation is conducted based on the classification accuracy and F-measure, as well as the root mean square error of the imputed data. The experimental results indicate the efficiency of the proposed intrusion classification system and find the best match missing data imputation technique for the sake of intrusion classification.

Security maintenance of Supervisory Control and Data Acquisition (SCADA) systems has been a point of interest during recent years. Numerous research works have been dedicated to the design of intrusion detection systems for securing SCADA communications. Nevertheless, these data-driven techniques are usually dependant on the quality of the monitored data. In this work, we propose a novel feature selection approach, called MSFS, to tackle undesirable quality of data caused by feature redundancy. In contrast to most feature selection techniques, the proposed method models each class in a different subspace, where it is optimally discriminated. This has been accomplished by resorting to ensemble learning, which enables the usage of multiple feature sets in the same feature space. The proposed method is then utilized to perform intrusion detection in smaller subspaces, which brings about efficiency and accuracy. Moreover, a comparative study is performed on a number of advanced feature selection algorithms. Furthermore, a dataset obtained from the SCADA system of a gas pipeline is employed to enable a realistic simulation. The results indicate the proposed approach extensively improves the detection performance in terms of classification accuracy and standard deviation.

Design of the data-driven diagnostic systems usually requires to have labeled data during the training session. This paper aims to design a hybrid data-driven framework for diagnosing faults, where the data labels are not available to a large extent. This hybrid framework has five steps for transforming raw vibration signals to informative sets of samples for decision making. It uses several state-of-the-art approaches for feature extraction and semi-supervised feature reduction. The decision-making step uses a number of state-of-the-art semi-supervised learners. This step also comprises a novel surface estimation approach that is developed for SSL. The proposed hybrid framework is applied for diagnosing bearing defects in induction motors and validated based on four scenarios, each of which is experimented with different amounts of labeled samples. The attained diagnostic accuracies show the efficiency of the proposed hybrid framework, including the novel semi-supervised learner in classifying bearing defects, regardless of the number of labeled samples.

Machine learning techniques are widely used for diagnosing faults to guarantee the safe and reliable operation of the systems. Among various techniques, semi-supervised learning can help in diagnosing faulty states and decision making in partially labeled data, where only a few number of labeled observations along with a large number of unlabeled observations are collected from the process. Thus, it is crucial to conduct a critical study on the use of semi-supervised techniques for both dimensionality reduction and fault classification. In this work, three state-of-the- art semi-supervised dimensionality reduction techniques are used to produce informative features for semi-supervised fault classifiers. This study aims to achieve the best pair of the semisupervised dimensionality reduction and classification techniques that can be integrated into the diagnostic scheme for decision making under partially labeled sets of observations.