This paper proposes ABIGX (Adversarial fault reconstruction-Based Integrated Gradient eXplanation), a unified framework for explainable fault detection and classification (FDC). ABIGX builds on the foundational principles of established fault diagnosis methods, including contribution plots (CP) and reconstruction-based contribution (RBC), while extending their applicability to general FDC models and improving fault explanation results. Central to ABIGX is the Adversarial Fault Reconstruction (AFR) method, which rethinks fault reconstruction from the perspective of adversarial attacks, introducing a novel fault index applicable to both fault detection and classification tasks. In fault detection, we theoretically bridge ABIGX with conventional fault diagnosis methods by proving that CP and RBC are the linear specifications of ABIGX. For fault classification, we address the challenge of fault class smearing, an inherent issue that can obscure accurate explanations. We demonstrate that ABIGX effectively mitigates this issue, outperforming current gradient-based explanation methods. The experiments evaluate the explanations of FDC by quantitative metrics and intuitive illustrations. The results validate the generality and accuracy of AFR, and show that ABIGX provides more comprehensive and precise explanations across various FDC models, offering a significant improvement over existing methods.

Deep neural networks (DNNs) have achieved satisfactory performance in multiple fields. However, recent studies have shown that DNNs can be easily fooled by adversarial examples. To mitigate the threats caused by adversarial attacks, a highly effective strategy is to design detectors to reject adversarial examples. This article proposes an unsupervised class- and classifier-free adversarial detection method. It only takes unlabeled clean data for training to discriminate illegal samples, and does not require any knowledge about the adversarial examples, sample classes, and the original classifier. More specifically, motivated by the idea that adversarial examples may differ significantly from benign data in terms of sample structural information, we develop an adversarial detector that can simultaneously capture the residual information and the variable-wise structural relationships of data. After that, we design an attribute called data identity (ID) that combines the extracted residual and structural information of data to identify adversarial examples. We validate the superiority of the proposed method through detecting adversarial attacks on CIFAR-10 and ImageNet datasets, and the experimental results demonstrate that the performance of our model is the best among various state-of-the-art adversarial detectors. Besides, we also conduct visualization experiments to illustrate the role of structural information in detecting adversarial examples.

The rapid advancement of deep learning and time-frequency analysis techniques have brought about a revolution in fault diagnosis for mechanical systems, offering flexible and efficient solutions. Nevertheless, data imbalance issues continue to pose significant obstacles in fault diagnosis modeling. In this research, we propose the use of a domain adaptation generative adversarial network (DAGAN) that capitalizes on transfer learning to extract valuable information from the majority-class data, while concurrently generating and augmenting minority-class data to expand the training dataset. DAGAN incorporates advanced techniques, including deep domain confusion and parameter forgetting, to enhance knowledge extraction and transfer during the transfer learning process, resulting in more realistic and comprehensive generation outcomes when dealing with small sample training. Furthermore, we have developed an imbalanced fault diagnosis method based on DAGAN, which further incorporates Continuous Wavelet Transform and Deep Residual Networks. Finally, the effectiveness and superiority of proposed method are validated on bearing and gearbox datasets. The experimental results demonstrate the outstanding performance of our method in effectively addressing imbalanced fault diagnosis. Note to Practitioners—In this paper, we present a practical approach to enhance the diagnosis of imbalanced faults. Our approach begins by utilizing Time-frequency images, which offer a comprehensive representation of the temporal and spectral characteristics of mechanical systems’ behavior. These images serve as input features and form the foundation for our fault diagnosis modeling. To address the limitations imposed by imbalanced data, we introduce DAGAN and incorporate advanced techniques such as deep domain confusion and parameter forgetting. These techniques facilitate the extraction and transfer of knowledge during the transfer learning process of DAGAN. Consequently, our approach generates more realistic and comprehensive outcomes, even when confronted with limited training samples. To validate the effectiveness and superiority of our proposed approach, we conducted extensive experiments on bearing and gearbox datasets. The results of these experiments demonstrate that our practical approach, which combines wavelet transform-based time-frequency analysis and the innovative DAGAN framework, offers a reliable and comprehensive solution for overcoming challenges associated with imbalanced fault data.

To combine the strengths of Gaussian and non-Gaussian latent variable models, a novel information fusion strategy has recently been proposed under the deep learning framework. Although promising results have been obtained, the critical structure learning problem remains unsolved, which seriously hinders the automation of data-driven modeling and analytics. In this article, the maximal information coefficient (MIC) method is introduced as a measurement of the association strength between two latent variables, which has no restriction in the type of data distribution. Through an assessment on the necessity of adding a new hidden layer into the deep model in each step, an evaluation index is defined for automatic determination of the required hidden layers during the model training process. For time-varying industrial production environments, reconfiguration or updating of the model structure is frequently required. In this case, automated data-driven modeling and structure learning can significantly improve the efficiency of data analytics. Based on the study results obtained from two real industrial examples, the proposed structure learning algorithm is feasible, and the automated data analytics scheme has significantly improved the online prediction performance in time-varying industrial processes.

In the past years, latent variable models have played an important role in various industrial AI systems, among which quality prediction is one of the most representative applications. Inspired by the idea of deep learning, those basic latent variable models have been extended to deep forms, based on which the quality prediction performance has been significantly improved. However, different latent variable models have their own strengths and weaknesses, a model works well under one scenario might not provide satisfactory performance under another. The motivation of this article is based on the viewpoint of information fusion and ensemble learning for heterogeneous latent variable models. Particularly, a collaborative deep learning and model fusion framework is formulated for the purpose of industrial quality prediction. In the first stage of the framework, collaborative layer-by-layer feature extractions are implemented among different latent variable models, through which different patterns of latent variables are identified in different layers of the deep model. Then, in the second stage, an ensemble regression modeling strategy is proposed to fuse the quality prediction results from different latent variable models, which is based on a well-designed data description method. Two real industrial examples are used for performance evaluation of the proposed method, based on which we can observe that information fusions in terms of both collaborative layer-by-layer feature extraction and heterogeneous model ensemble have positive effects in improving prediction accuracy and stability.

Recently, a deep residual form of the principal component analysis (PCA) model has been proposed as a feature engineering for industrial data analytics, which has obtained more satisfactory performances compared to the shallow feature engineering model. However, a critical issue remain unsolved is how to effectively determine the number of hidden layers in the deep model, which may significantly influence its performance. In this article, a novel hidden layer selection strategy is proposed to automate the training process of the deep residual PCA model. With a new definition of similarity factor based on cosine distance between two latent variables, the degree of pattern repetition can be well recognized and evaluated. In addition, a layer retained factor is further defined to assess the necessity of adding a new hidden layer to the deep model. As a result, the number of required hidden layers can be automatically determined, making the deep residual PCA model fully automated. Four industrial case studies are provided for performance evaluation, based on which both feasibility and effectiveness of the new strategy are confirmed.

Inspired by the idea of deep learning, several latent variable models have been successfully extended to the deep forms for industrial data analytics. Compared to traditional deep neural networks, deep latent variable models are more fitted to the requirement of data-driven modeling and applications in industrial production systems, due to the lightweight model structure and high-efficient data analytics process. The aim of this article is to develop a deep latent variable predictive modeling framework, which is based on a newly designed Bayesian soft attention mechanism. Instead of only using extracted features from the last hidden layer for predictive modeling, different attentions are focused across all hidden layers of the deep model. As a result, different layer-wise models make different contributions in predicting different data patterns, through which the ability of the deep latent variable model will be further explored. Two real industrial examples are provided for performance evaluation and comparative studies among different prediction models, based on which the superiority of the online Bayesian soft attention mechanism has been confirmed.

Probabilistic latent variable models (PLVMs), such as probabilistic principal component analysis (PPCA), are widely employed in process monitoring and fault detection of industrial processes. This article proposes a novel deep PPCA (DePPCA) model, which has the advantages of both probabilistic modeling and deep learning. The construction of DePPCA includes a greedy layer-wise pretraining phase and a unified end-to-end fine-tuning phase. The former establishes a hierarchical deep structure based on cascading multiple layers of the PPCA module to extract high-level features. The latter builds an end-to-end connection between the raw inputs and the final outputs to further improve the representation of the model to high-level features. After constructing the model structure of DePPCA, we first present the detailed training processes of the pretraining and fine-tuning stages, then clarify the theoretical merits of the proposed model from the perspective of variational inference. For process monitoring purposes, we develop two statistics based on the established DePPCA. The monitoring performance of these two statistics can remain superior even if the features extracted by DePPCA are significantly compressed to univariate. This makes the feature extraction process and online monitoring procedure of DePPCA quite fast. In other words, the proposed DePPCA can achieve accurate and efficient process monitoring by only extracting one feature for each sample. Finally, the effectiveness of DePPCA is evaluated on the Tennessee Eastman (TE) process and the multiphase flow (MPF) facility.

In the past years, the performance of latent variable models has been significantly improved by polishing their model structures, such as via kernel tricks, dynamic extensions, and even through deep learning techniques. It has become more difficult and cumbersome to explore further potentials from the aspect of model architecture. According to the recent perspective from data-centric artificial intelligence, improving the quality of the training dataset could be more effective in enhancing the performance of AI systems, compared to further polishing the model structure, which is already very complex. In this article, the reliability of the recently developed deep Gaussian and non-Gaussian (GnG) information fusion model is improved by designing an automatic data selection strategy and an iterative model updating scheme. By defining a new data quality monitoring index, those fault-free data samples are carefully selected from the raw dataset, which are then used for automatic updating of the monitoring framework. Based on the strict quality control of the training data samples, both monitoring performance and application reliability of the deep GnG model have been significantly improved. Besides, those expenses in terms of human labor and data labeling time can be largely saved at the same time.

Industrial intelligent systems (IIS) play a huge role in modern industry, and their intelligent models of IIS enable diagnosis of faults, key performance indicator (KPI) prediction, and other important industrial process analysis in a data-driven way. However, the performance of intelligent models is limited by the quantity and quality of local data in specific factories. At the same time, the privacy information and security concerns contained by industrial data lead to the problem of information silos in industry. This hinders data sharing and cross-factory collaborations. To address these issues, this article makes the following contributions. First, for the first time, we empower industrial federated big data analytics (IFBDA) of IIS with swarm learning, and propose a hyperledger fabric-based IFBDA blockchain (IFBDAchain) for multifactory information sharing and collaborative modeling. Second, in the IFBDAchain, we further consider potential dishonest behaviors among federated members, and design verification and privacy protection mechanisms to ensure trustworthiness of analytics. Third, we validate the IFBDAchain with two real industrial cases. The results demonstrate the effectiveness of the IFBDAchain in fault classification and KPI prediction tasks in industry. Compared to the average values of local learning, our method increases the classification accuracy by 27.6%, 69.4%, and 33.1% under Independent and identically distributed (IID), non-IID, and unbalanced conditions, respectively. Furthermore, the root-mean-square error of the KPI prediction decreases by 33.3%, 49%, and 45.7% for the IID, non-IID, and unbalanced conditions, respectively, indicating its significant potential as a generic backbone for industrial federated Big Data analytics.

Accurate quality variable inference by process variables is the core of industrial inferential sensor modeling, where recent advancements have seen deep learning (DL) models achieving remarkable success. However, integrating knowledge of unit operations is critical for improving inferential sensor performance, yet it has received little attention. The main challenge lies in the incompleteness and correctness of industrial knowledge due to its semi-empirical nature and inevitable engineering errors. Addressing this, this article introduces the gradient knowledge network based on the graph neural network’s message-passing mechanism within the variational Bayesian inference framework, which naturally copes with the abovementioned issues by fusing observational data. Initially, the prior knowledge about the process variables, which mirrors the graph in graph neural network, is parameterized as Dirichlet distribution based on the analysis of message-passing mechanism. However, the divergence computation and normalization constraints are challenging for model implementation. To navigate these challenges, the Bayesian inference problem is transformed into an optimization problem, subsequently recast as a simulation problem induced by the gradient field, ensuring compatibility with DL backends. Furthermore, a theoretical iteration equation is derived to maintain the normalization constraint. The architecture of the proposed model and its learning algorithm are then detailed. Finally, various experiments are conducted on two real industrial processes to demonstrate the model’s efficacy from the perspective of prediction accuracy, sensitivity analysis, and ablation study.

Data-driven soft sensing has become quite popular in recent years, which can provide real-time estimations of key variables in industrial processes. While the introduction of deep learning does improve the prediction performance, it is highly restricted to the number of labeled training data, as well as large computational burden and cumbersome parameter tuning procedures. How to break through the bottleneck of data-drive models in terms of limited labeled data and high computational complexity should be one of the main recent focuses in the field of industrial soft sensing. In this article, a deep co-training PLS (deep CT-PLS) model is proposed to extend the ordinary PLS model to the semi-supervised deep form. While the deep model can efficiently extract inherent natures of process data, the co-training strategy makes lots of unlabeled data useful through a two-view cross training and annotation process. In this case, the performance restriction of the deep PLS model can be greatly relieved, with the incorporation of additional unlabeled data, while at the same time the designed model structure keeps in a low computational complexity. Based on the case study on a real industrial production process, the deep CT-PLS model can significantly improve the soft sensing performance.

Feature attribution explains Artificial Intelligence (AI) at the instance level by providing importance scores of input features' contributions to model prediction. Integrated Gradients (IG) is a prominent path attribution method for deep neural networks, involving the integration of gradients along a path from the explained input (explicand) to a counterfactual instance (baseline). Current IG variants primarily focus on the gradient of explicand's output. However, our research indicates that the gradient of the counterfactual output significantly affects feature attribution as well. To achieve this, we proposeIterativeGradient pathIntegratedGradients (IG2), considering both gradients. IG2incorporates the counterfactual gradient iteratively into the integration path, generating a novel path (GradPath) and a novel baseline (GradCF). These two novel IG components effectively address the issues of attribution noise and arbitrary baseline choice in earlier IG methods. IG2, as a path method, satisfies many desirable axioms, which are theoretically justified in the paper. Experimental results on XAI benchmark, ImageNet, MNIST, TREC questions answering, wafer-map failure patterns, and CelebA face attributes validate that IG2delivers superior feature attributions compared to the state-of-the-art techniques.

Data-driven models are revealed to be vulnerable to adversarial examples, so improving the model’s adversarial robustness has attracted extensive research. However, in real-world industrial processes, the difficulty of collecting data for different fault types varies, leading to imbalanced data, which poses significant obstacles to adversarial robustness learning. Furthermore, it results in inaccurate boundary learning, poor robustness, and overfitting to the majority class in common empirical adversarial training methods. To overcome these problems, we propose a balanced adversarial learning strategy. On the one hand, the balanced generalization processing penalizes the imbalance in the generation of adversarial variant in inner training, and calibrates the confidence using the class-aware label smoothing in outer training, so as to learn more accurate boundaries with better game relations. On the other hand, it involves robust regularization processing, which adopts the smooth augmentation of adversarial examples in the input and the addition of data-related soft label regularization to learn good generalization with enriched information. Case studies on industrial benchmarks Tennessee Eastman process (TEP) and NEU surface defect database (SDD) demonstrate that our approach achieves better standard and robust performance.

Nonlinear dynamical probabilistic latent variable model (NDPLVM) and its variants, essential in industrial inferential sensors, face challenges in latent space inference and deep learning (DL) backend implementation. The first issue arises from the assumption that covariates directly infer the latent variable, potentially leading to inaccuracies. The second issue involves the discrepancy between the probabilistic distribution function form of NDPLVMs and data sample-based operation of DL backends. Addressing these, this study introduces the optimal control-NDPLVM (OC-NDPLVM), a model designed to enhance performance by analyzing NDPLVMs learning and tackling these issues. For the first problem, NDPLVMs' learning is reinterpreted as an optimization problem, solved by alternating direction method of multipliers, and selecting the inference network's input via studying optimal solution's structure. To address the second issue, OC-NDPLVM adapts mean and covariance equations for compatibility with DL backends. This model's effectiveness is validated through experiments on inferential sensor datasets.

In recent years, deep neural networks (DNNs) have been widely applied in fault classification tasks. Their adversarial security has received attention, but little consideration has been given to the robustness of adversarial attacks against imperfect DNNs. Owing to the data scarcity and quality deficiencies prevalent in industrial data, the performance of DNNs may be severely constrained. In addition, black-box attacks against industrial fault classification models have difficulty in obtaining sufficient and comprehensive data for constructing surrogate models with perfect decision boundaries. To address this gap, this article analyzes the outcomes of adversarial attacks on imperfect DNNs and categorizes their decision scenarios. Subsequently, building on this analysis, we propose a robust adversarial attack strategy that transforms traditional adversarial attacks into an iterative targeted attack (ITA). The ITA framework begins with an evaluation of DNNs, during which a classification confidence score (CCS) is designed. Using the CCS and the prediction probability of the data, the labels and sequences for targeted attacks are defined. The adversarial attacks are then carried out by iteratively selecting attack targets and using gradient optimization. Experimental results on both a benchmark dataset and an industrial case demonstrate the superiority of the proposed method.

In recent years, more and more open environment have led to confidential links and data exposure, which seriously threatens the security of industrial systems. Adversarial attacks can easily fool machine learning models by adding tiny perturbations to input data. Industrial fault detection and classification (FDC) system is an indispensable part of ensuring production safety, but it is also not immune to the impact of adversarial risks. Once it is under attack, the disastrous consequences that may be caused to the industrial system are unimaginable. Adversarial training is among the most effective defense methods to protect those data-driven intelligent systems. This article studies a novel reweighted adversarial training approach called adversarial weight prediction networks. By assigning more appropriate weights to different data samples, we can make better use of the limited model capacity of the industrial FDC system. Particularly, predicting weights through a synchronized network overcomes the limitations of insufficient information and nontransferability of existing statistical methods. Performance evaluation on three industrial cases containing structured and image data shows the superior generalization and stability of our proposed method.

Nowadays, data-driven models have been prevalent in predicting hard-to-measure key quality indicators of industrial processes in order to improve product quality and process safety. Such models are called soft sensors as they can serve the same role as physical sensors, but they do not require extra physical devices. Despite their success, soft sensors suffer from poor reliability. Reliability is the ability of soft sensors to give accurate predictions not only at the time they are trained, but also in the long term, despite potential drifts of the process. This is important when soft sensors are to be applied in critical industrial processes. In order to alleviate this problem, in this article, we propose a graph-constrained soft-sensor (GCSS) model that uses graph convolutions based on the a priori undirected graph of the process variables. Based on the modern control theory, we also propose an approach to extracting an undirected graph from process diagrams of the target process. This approach can identify relationships between process variables, which is easy to use and can be applied to a majority of industrial processes. The extracted graph structure serves as a constraint, and pushes the data-driven GCSS model into the direction of the true inner structure of the target process. With the aid of a priori graph knowledge, the GCSS model enjoys better generalizability and reliability. This has been validated in a simulation example and a real-world high-low transformer process. Compared to other soft sensors, the test performance of the GCSS model is improved by 6.5%. In the high-low transformer process, the GCSS model has the best test performance and the gap between training and test performance is reduced by 54%.

With the advent of the industrial Big Data era, accurate estimation of product quality and monitoring of working conditions from historical data have become crucial in the process industry. However, the majority of data-driven approaches predominantly rely on observational data, overlooking the valuable empirical knowledge derived from experience or underlying mechanisms. In order to leverage this knowledge, researchers employ various graph neural network-based methods which introduce connections among process variables for feature extraction. Nevertheless, it is imperative to recognize that process knowledge undergoes changes due to internal or external concept drift. To address this challenge, we propose a novel deep learning module called “variational inference over graph” to effectively harness shifting knowledge. Building upon the self-attention mechanism, we design a probabilistic self-attention mechanism for encoding and reconciling prior knowledge. Instead of directly encoding the prior knowledge through graph neural network edges, we incorporate it as regularization term within the variational inference framework that accounts for knowledge shift. Furthermore, we introduce reparameterization estimator to control the variance resulting from knowledge uncertainty. To showcase the capability of our proposed method, we conduct various experiments on quality prediction task in real industrial processes.

While the data-driven fault classification systems have achieved great success and been widely deployed, machine-learning-based models have recently been shown to be unsafe and vulnerable to tiny perturbations, i.e., adversarial attack. For the safety-critical industrial scenarios, the adversarial security (i.e., adversarial robustness) of the fault system should be taken into serious consideration. However, security and accuracy are intrinsically conflicting, which is a trade-off issue. In this article, we first study this new trade-off issue in the design of fault classification models and solve it from a brand new view, hyperparameter optimization (HPO). Meanwhile, to reduce the computational expense of HPO, we propose a new multiobjective (MO), multifidelity (MF) Bayesian optimization (BO) algorithm, MMTPE. The proposed algorithm is evaluated on safety-critical industrial datasets with the mainstream machine learning (ML) models. The results show that the following hold: 1) MMTPE is superior to other advanced optimization algorithms in both efficiency and performance and 2) fault classification models with optimized hyperparameters are competitive with advanced adversarially defensive methods. Moreover, insights into the model security are given, including the model intrinsic security properties and the correlations between hyperparameters and security.

In industrial processes, the sampling rates of process variables are discrepant because of the nature of instruments and measuring demands, which forms the challenging issue, that is, the multirate modeling in the data-driven soft sensor development. In this work, a multiresolution pyramid variational autoencoder (MR-PVAE) predictive model is proposed to solve this problem based on the deep feature extraction and feature pyramid augmentation. First, a multirate data filter is designed through a resolution searching strategy to turn the original process data into a multiresolution dataset. Then, the pyramid variational autoencoder (PVAE) is proposed to extract deep nonlinear features from the data with different resolutions. In PVAE, the augmented feature pyramid is constructed layer by layer to fuse extracted features from low resolution to the high. As a consequence, the extracted features with various resolutions are gathered to form the regression model, where the process information contained in data with discrepant sampling rates can be fully utilized. Due to the layer-by-layer enhanced features, the prediction accuracy of the soft sensing model are gradually improved. Meanwhile, an optimized training strategy is established to select the optimal feature pyramid for prediction. A numerical experiment and an industrial soft sensing case are given to validate the effectiveness and superiority of the proposed MR-PVAE model.

Due to the complex reaction mechanisms of industrial process units, causality and correlations exist between industrial process variables. Causal discovery algorithms have been utilized to discover the knowledge on variable relationships and guide process modeling and control optimization. However, most of them are limited by strict assumptions, such as linear relationships, additive noise, steady-state process, etc. Therefore, these methods cannot gain good performance for most practical industrial processes. To solve these problems, a novel weight comparison causal mining (WCCM) algorithm is proposed in this article for industrial causal graph discovery. It first trains a group of hidden layer neural networks with process data, then mines an undirected skeleton of the process variables according to the comparison of the network weights, and further determines the causal directions of the undirected edges in the skeleton to get a directed causal graph. The effectiveness of WCCM is verified on a benchmark and a practical industrial case from the urea synthesis process. The undirected and direct edges mined by WCCM show high consistency with the ground truths. Moreover, the causal discovery results of WCCM are utilized to guide the feature selection of soft sensor modeling, resulting in improved prediction accuracy and enhanced model interpretability.

Nonlinear and multimode characteristics commonly appear in modern industrial process data with increasing complexity and dynamics, which have brought challenges to soft sensor modeling. To solve these issues, in this article, a dynamic mixture variational autoencoder regression model is first proposed to handle the multimode industrial process modeling with dynamic features. Furthermore, to deal with the partially labeled process data with rare quality values and large-scale unlabeled samples, a semi-supervised mixture variational autoencoder regression model is proposed, where a corresponding semi-supervised data sequence division scheme is introduced to make full use of the information in both labeled and unlabeled data. Finally, to verify the feasibility and effectiveness of the proposed methods, the models are applied to a numerical case and a methanation furnace case. The results show that the proposed methods have superior soft sensing performance, compared with the state-of-the-art methods.

In modern industries, data-driven fault detection and classification (FDC) systems can efficiently maintain industrial security and stability, while the security of the data-driven FDC system itself is rarely or even never considered. The security problem named adversarial vulnerability is the intrinsic of data-driven machine learning models, which will give incorrect predictions under the maliciously perturbed input data. This paper presents a work on this new security topic of the data-driven FDC systems, by 1) summarizing and comparing various recent and typical adversarial attack and defense methods for fault classifiers; 2) proposing novel attack and defense techniques for unsupervised fault detectors; 3) constructing a novel industrial adversarial security benchmark on FDC systems in the Tennessee-Eastman process (TEP) dataset; 4) exploring and discussing which attack is most potentially threatening for FDC systems and which defense technique is most applicable to mitigate attacks. The results reveal unique security properties of FDC systems, mainly including 1) for fault classifiers, black-box attack is close to the attack strength of white-box FGSM and the universal transferable attack is not significantly stronger than random noise; 2) weak adversarial training is excellent with high adversarial accuracy improvement and negligible clean accuracy decrease; 3) fault detectors are intrinsically more robust, and can be well protected by strong adversarial training. More intriguing properties and profound insights are demonstrated in the paper. This pioneering work could guide researchers and practitioners in discovering and navigating the field of FDC system adversarial robustness, outlining the research directions and open problems.

The salient progress of deep learning is accompanied by nonnegligible deficiencies, such as: 1) interpretability problem; 2) requirement for large data amounts; 3) hard to design and tune parameters; and 4) heavy computation complexity. Despite the remarkable achievements of neural networks-based deep models in many fields, the practical applications of deep learning are still limited by these shortcomings. This article proposes a new concept called the lightweight deep model (LDM). LDM absorbs the useful ideas of deep learning and overcomes their shortcomings to a certain extent. We explore the idea of LDM from the perspective of partial least squares (PLS) by constructing a deep PLS (DPLS) model. The feasibility and merits of DPLS are proved theoretically, after that, DPLS is further generalized to a more common form (GDPLS) by adding a nonlinear mapping layer between two cascaded PLS layers in the model structure. The superiority of DPLS and GDPLS is demonstrated through four practical cases involving two regression problems and two classification tasks, in which our model not only achieves competitive performance compared with existing neural networks-based deep models but also is proven to be a more interpretable and efficient method, and we know exactly how it improves performance, how it gives correct results. Note that our proposed model can only be regarded as an alternative to fully connected neural networks at present and cannot completely replace the mature deep vision or language models.

Data-driven fault detection and classification (FDC) systems play an important role in ensuring the stability and security of modern industry. However, the security issue of the data-driven FDC itself poses new challenges, where the model prediction can be seriously damaged by maliciously manipulated imperceptible perturbations, known as the adversarial attack. Since the adversarial attacks may threaten the FDC models and even the whole safety-critical industrial systems, there is an urgent need for the guarantee of data-driven model security. In this article, a scheme is presented for formally and completely verifying the security properties by treating the target problem as a convex mathematical programming. The major contribution on methodology is the verification under multiple norms via multiobjective optimization with a novel Pareto front approximation algorithm. Moreover, this work studies security verification for both supervised (fault classification) and unsupervised (fault detection) models under all mainstream norms simultaneously. For four basic data-driven models on two industrial datasets, our exclusive verification scheme provides deep and novel security insight into FDC systems. Moreover, we compare with related works to validate the algorithm performances of verification and Pareto front approximation.

Adversarial attack can fabricate imperceptible fake samples to cheat a well-trained artificial intelligence (AI) model, and it has shown strong destructive power in many classification tasks. In real-world AI applications, there is another popular kind of machine learning paradigm—regression. The threats of adversarial attack may also exist in the regression scenario, however, the research on the adversarial vulnerability of the regression model has been basically neglected. This article first systematically explores the adversarial attack on regression problems. Starting from analyzing the difference between the attacking classification models and regression systems, we show the existing attack framework of classification problems is unsuitable for attacking regression systems. Then, we discuss the essence of regression tasks and design an appropriate attack objective for regression problems. After that, we propose two algorithms with different properties based on gradient optimization to achieve the attack objective. The proposed attack methods are evaluated on three real-world regression cases, and the results show that our attacks can successfully make the prediction deviate a lot from its original value by only exerting a tiny perturbation on the inputs. Finally, we conduct further experiments and analyses to discuss the effectiveness and characteristics of the proposed methods from various perspectives.

With the advent of the big data era and the timeliness requirements of data processing, a large amount of streaming industrial big data is continuously obtained in real time. Facing this kind of flowing and time-varying knowledge information form, incremental learning is necessary. Lifelong learning (LL), as a typical incremental learning method, can continuously retain and accumulate old knowledge while learning new knowledge, which is very suitable for streaming industrial big data scenarios. At the same time, Bayesian nonparametric (BNP) models can adjust the complexity of the model based on observation data. Motivated by ideas of BNP and LL, a lifelong Bayesian learning machines framework is proposed in this article, which includes model expansion and model optimization. In general, this framework not only learns new effective knowledge and accumulates knowledge through incremental variational Bayesian under model expansion but also uses optimization steps to avoid model degradation caused by unnecessary component information. As an example, Dirichlet processes Gaussian mixture regression (DPGMR) is utilized for process modeling under this framework. To evaluate the feasibility and efficiency of the developed method, a synthetic and a real industrial case are demonstrated.

For fault classification in industrial processes, both inaccurate and incomplete supervised information commonly exist in practice, which raise a big challenge to the research field. In this article, a weakly supervised form of the multilayer perceptron (MLP) model is proposed, with considerations of both inaccurate and incomplete labels in fault classification. First, a label probability transition matrix is used to describe the relationship between the inaccurate labels and unknown true labels of process data. This transition matrix is estimated through a Gaussian mixture model, and then used to correct the loss function of the MLP model. Based on the framework of the developed weakly supervised MLP (WS-MLP) model, the information of incomplete labels in the training data set is incorporated simultaneously with the inaccurate label information. The performance of the proposed model WS-MLP is evaluated through two industrial benchmark data sets, results of which indicate its effectiveness under different application cases.Note to Practitioners—Due to manual labeling and other reasons, it is difficult to ensure that the labels of industrial data samples are completely accurate. At present, a useful method is to use a label probability transition matrix to correct the model loss function, so that the model can learn the inaccurate labeled samples robustly. Based on the different distribution of feature representation in the model between accurate category samples and inaccurate category samples, we introduce the Gaussian mixture model to carry out the feature representation and estimate the label probability transition matrix. In addition, a large number of unlabeled samples are available in the industry. In this article, the inaccurate label data samples and incomplete label data samples are modeled using WS-MLP model. The performances of the estimated label probability transition matrix and WS-MLP model are validated by two industrial examples.

These days, data-driven soft sensors have been widely applied to estimate the difficult-to-measure quality variables in the industrial process. How to extract effective feature representations from complex process data is still the difficult and hot spot in the soft sensing application field. Deep learning (DL), which has made great progresses in many fields recently, has been used for process monitoring and quality prediction purposes for its outstanding nonlinear modeling and feature extraction abilities. In this work, deep stacked autoencoder (SAE) is introduced to construct a soft sensor model. Nevertheless, conventional SAE-based methods do not take information related to target values in the pretraining stage and just use the feature representations in the last hidden layer for final prediction. To this end, a novel gated stacked target-related autoencoder (GSTAE) is proposed for improving modeling performance in view of the above two issues. By adding prediction errors of target values into the loss function when executing a layerwise pretraining procedure, the target-related information is used to guide the feature learning process. Besides, gated neurons are utilized to control the information flow from different layers to the final output neuron that take full advantage of different levels of abstraction representations and quantify their contributions. Finally, the effectiveness and feasibility of the proposed approach are verified in two real industrial cases.

In industrial processes, data-driven soft sensors have played an important role for the effective process control, optimization, and monitoring. Deep learning technique has been widely used in soft sensor field in recent years for its excellent feature representation capability in spatial and temporal scales. However, the shortcomings for deep learning technique seriously hinder its application in industrial processes. For example, the knowledge cannot be added into the model, and the model prediction could not be well explained. To solve those problems, the graph mining, convolution, and explanation framework is proposed for knowledge automation in this article. Based on the equivalence analysis of the self-attention mechanism (SAM) and graph convolution (GC) operation, the spatial SAM is adopted for knowledge discovery from data directly. After that, the GC layer considering the relationship between process variables can utilize the knowledge for constructing soft sensor models. Besides, to explain which knowledge contributes to the final model prediction, the graph neural network explainer is designed for explaining the model output. Finally, the effectiveness and feasibility of the framework are evaluated on an industrial process, in which the knowledge discovered from the data is of great consistence with the prior knowledge, and the final explanation indicated that most of the knowledge is consistent with the prior knowledge contributed to the prediction.

Data-driven artificial intelligence (AI) models have been widely used in industrial systems helping big data analytic due to its convenience and flexibility. However, adversarial attacks have the ability to mislead AI models to make incorrect predictions just by adding specific perturbation to actual samples. With the high integration of industrial systems and information technology, the reliability and safety of AI models in industrial systems have been seriously threatened. In the article, fault diagnosis models and soft sensing models rely on AI technology are verified to be vulnerable facing adversarial attack. To this end, the concept of information fingerprint for industrial data is introduced to distinguish actual samples from adversarial samples with small perturbation. With fault diagnosis models and soft sensing models as the background, information fingerprint exaction networks based on deep learning is developed to extract the information fingerprint for further analysis. It utilizes supervised contrastive pretraining and unsupervised training to realize parameter learning for the structure of siamese neural network and autoencoder. Finally, the effectiveness and feasibility of the proposed information fingerprint for adversarial sample detection are verified in two industrial benchmark cases.

Neural-network-based soft sensors are widely employed in the industrial process. Such models have great significance to smart manufacturing. Considering the strict requirements of industrial production, it is vital to ensure the safety and robustness of these models in their actual deployment. However, recent research has shown that neural networks are quite vulnerable to adversarial attacks. By imposing tiny perturbation to the original sample, the fabricated adversarial sample can cheat these models to make wrong decisions. Such a phenomenon may bring serious trouble to the practical application of soft sensors. This article focuses on the adversarial attacks on industrial soft sensors. For the first time, we verify and analyze the effectiveness and deficiencies of the existing attack methods in the industrial soft sensor scenario. Based on solving these defects, this article proposes a novel perspective for attacking soft sensors. We analyze the optimization mechanism behind this new idea and then design two algorithms to perform attacks. The proposed methods more conform to the actual situation. Besides, compared with the existing approaches, the proposed methods have potentials to cause severer damages since their attacks are not only more concealed but also more likely to cheat the technicians to execute wrong operations. The research and analyses of the proposed methods lay a solid foundation for more thorough defenses against various attacks, which is quite necessary for making the deployed soft sensors more robust and secure.

Data-driven process monitoring based on latent variable models are widely employed in industry. This article proposes a novel monitoring framework for latent variable models using hierarchical feature extraction, Bayesian inference, and weighting strategy. We first establish a deep structure to implement hierarchical latent variables extraction, the extracted features are used to construct diverse monitoring statistics. Then, we utilize Bayesian inference and proper weighting strategy to fuse various useful information. In line with the different characteristics of principal component analysis (PCA) and independent component analysis (ICA), we construct a deep PCA-ICA model for process monitoring according to the proposed framework. The deep PCA-ICA model performs hierarchical feature extraction, which can simultaneously extract deep Gaussian information and deep non-Gaussian information. The features extracted by different layers are then transformed to posterior probabilities through Bayesian inference. After that, different posterior probabilities are combined through appropriate weighting strategy to build new probabilistic statistic, which can give more synthetic monitoring results. Moreover, the Bayesian inference and weighting strategy are further used to integrate the advantages of different models by transforming various probabilistic statistics into an overall monitoring index, which can comprehensively indicate the process status. The Tennessee Eastman process is used to validate the superiority of the proposed model over the existing methods. Besides, the extracted features are further analyzed to show the effectiveness and benefits of the deep hierarchical feature extraction structure.

Just-in-time learning (JITL) has become a widely used industrial process modeling tool. With the advent of the industrial big data era, rich data information has brought new opportunities to JITL. Specifically, the completeness of data samples in the era of big data provides an important premise and support for the JITL method, prompting us to rethink the application value of JITL in the context of industrial big data. At the same time, the huge amount of data causes certain difficulties in data searching, which is a key issue for JITL. In this article, a parallel computing strategy is adopted to divide the entire computational searching task into several subtasks, and assign the subtasks to parallel computing nodes to complete parallel searching. In this way, not only the full advantage of big data information is effectively utilized, but also the search capability and efficiency under big data are improved. In addition, in order to improve the real-time nature of JITL, a model library management (MLM) strategy is adopted, and the query similar samples are used to operate with existing similar models. And by selectively adding new data, the database management (DBM) strategy is also developed, which not only alleviates the problem of information redundancy, but also reduces the search pressure caused by the increasingly large database. Obviously, MLM and DBM are particularly important as JITL auxiliary tools under industrial big data. Combining parallel computing, a parallel JITL (P-JITL) framework is proposed. As an example, the variational Bayesian factor regression model is transformed into the parallel Bayesian-JITL method for big process data modeling, which is further extended to a nonlinear form. To evaluate the feasibility and efficiency of the developed methods, a real industrial case is demonstrated.

The arrival of the intelligent manufacturing and industrial internet era brings more and more opportunities and challenges to modern industry. Specifically, the revolution of the production mode of traditional manufacturing is undergoing thanks to the techniques including but not limited to digits, network, intelligence, and industrial automation fields. As the core link between intelligent manufacturing and industrial internet platform, industrial Big Data analytics has been paid more and more attention by academia and industry. The efficient mining of the high-value information covered under industrial Big Data and the utilization of the real-life industrial process are among the hottest topics at present. Meanwhile, with the advanced development of industrial automation toward knowledge automation, the learning paradigm of industrial Big Data analytics is also evolving accordingly. Therefore, starting from the perspective of industrial Big Data analytics and aiming at the corresponding industrial scenarios, this article actively explores the revolution of the learning paradigm under the background of industrial Big Data: 1) The evolution of the industry Big Data analytics paradigm is analyzed, that is, from isolated learning to lifelong learning, and their relationships are further summarized; 2) Mainstream directions of lifelong learning are listed, and their applications in industrial scenarios are discussed in detail; 3) Prospects and future directions are given.

Recently, data-driven industrial monitoring systems have been rapidly developed and significantly improved the performance on industrial monitoring tasks. However, the widely deployed data-driven models expose industrial data to more unsecured links, which significantly increase the safety risk of safe-critical industrial systems. The research about adversarial attacks has shown that the potential attackers can utilize tiny crafted perturbations on the input data to mislead the machine learning models’ output. In this article, a novel cross-domain data protection scheme named “Data Guardian” is proposed to authenticate and correct industrial data under potential attacks on data-driven monitoring systems. “Data Guardian” embeds designed redundancy information into the data least significant bits, based on$q$-ary low-density parity-check (LDPC) codes over the Galois field (finite field). The data are encoded at the secured industrial sites and then decoded before being input into the monitoring systems, in which the security risk is usually higher. The decoding capability of$q$-ary LDPC codes is improved by the data statistical characteristics, with a new proposed prior estimation method. In the experiments, “Data Guardian” is tested under the attacks to the fault diagnosis models on the Tennessee Eastman process and rolling element bearing from Case Western Reserve University. The results show that “Data Guardian” can efficiently reduce the success rate of adversarial attacks, especially when the attack variable ratio is small.

The problem of fault classification in industry has been studied extensively. Most classification algorithms are modeled on the premise of data balance. However, the difficulty of collecting industrial data in different modes is quite different. This inevitably leads to data imbalance, which will adversely affect the fault classification performance. This article proposes a novel data augmentation classifier (DAC) for imbalanced fault classification. Data augmentation based on generative adversarial networks (GANs) is an effective way to solve the problem of unbalanced classification. However, the randomness of the GAN generation process restricts the effect of data enhancement. DAC proposes a data selection strategy based on data filtering and data purification in model training to solve this problem. In addition, DAC combines supervised learning and data generation processes to obtain an end-to-end model. Meanwhile, multigenerator structure of DAC (MDAC) is proposed to solve the problem of incomplete learning of a single generator when data imbalances get complicated. The proposed DAC and MDAC are applied in two fault classification cases of the Tennessee Eastman (TE) benchmark process, results of which show superiority of DAC and MDAC compared to existing methods.Note to Practitioners—Data imbalances are common in fault classification and affect the effectiveness of modeling in industry. As a generative model, generative adversarial networks (GANs) provide new ideas for small-class data augmentation. However, the instability of its training process and the randomness of data generation affect the results of data augmentation. In this article, the GAN generation process is analyzed in detail. The results of the visualization indicate that no data generation was perfect at any one time. Based on the rules of GAN data generation, we propose a data selection strategy during training. High-quality data are selected for data augmentation through data filtering and data purification. Apart from this, we combine the training process of GAN and classification model for imbalanced data to reduce modeling time. Through industrial examples, we have evaluated the effectiveness of this method.

Soft sensors have been widely accepted for online estimating key quality-related variables in industrial processes. The Gaussian mixture models (GMM) is one of the most popular soft sensing methods for the non-Gaussian industrial processes. However, in industrial applications, the quantity of samples with known labels is usually quite limited because of the technical limitations or economical reasons. Traditional GMM-based soft sensor models solely depending on labeled samples may easily suffer from singular covariances, overfitting, and difficulties in model selection, which results in the performance deterioration. To tackle these issues, we propose a semisupervised Bayesian GMM (S2BGMM). In the S2BGMM, we first propose a semisupervised fully Bayesian model, which enables learning from both the labeled and unlabeled datasets for remedying the deficiency of infrequent labeled samples. Subsequently, a general framework of weighted variational inference is developed to train the S2BGMM, such that the rate of learning from unlabeled samples can be controlled by penalizing the unlabeled dataset. Case studies are carried out to evaluate the performance of the S2BGMM through a numerical example and two real-world industrial processes, which demonstrate the effectiveness and reliability of the proposed approach.

Soft sensor plays an increasingly important role in modern industrial processes for estimating key quality variables which are hard to measure. With the development of deep learning technologies, soft sensors based on the deep learning methods have drawn great attention. Aiming to predict key quality variables, a supervised weighted nonlinear dynamic system (WNDS) model aided by the maximal information coefficient (MIC) is proposed in this article. The variational autoencoder is employed into the system for extracting nonlinear dynamic features. The supervised WNDS model can simultaneously analyze the correlations between variables and the relationships between historical samples and present samples. Furthermore, the proposed method is extended to a semisupervised form, in order to handle the imbalanced numbers between routinely recorded process data and limited labeled quality data. The prediction performance is validated by an industrial case.

Soft-sensing techniques are of great significance in industrial processes for monitoring and prediction of key performance indicators. Due to the effectiveness of nonlinear feature extraction and strong expansibility, an autoencoder (AE) and its extensions have been widely developed for industrial applications. Nevertheless, an AE commonly uses the last hidden layer for regression modeling with the output, which seems to be a kind of information waste as the shallow layers are also abstractions of input data. Besides, when there are excessive unlabeled samples, AE-based models are less likely to make full use of them or even degrade the performance. To deal with these issues, a method called ensemble semi-supervised gated stacked AE (ES2GSAE) is proposed in this article. Gate units are used to develop connections between different layers and the output layer, which also help quantify the contribution of different hidden layers. Moreover, the idea of ensemble learning is combined with semi-supervised learning, in which different unlabeled datasets are used for training different submodels to ensure their diversities. In this way, unlabeled samples can be utilized more efficiently and help enhance the model performance. The effectiveness and superiority are verified in a real industrial process by comparing the proposed method with other typical AE-based models.

Soft sensors are widely constructed in process industry to realize process monitoring, quality prediction, and many other important applications. With the development of hardware and software, industrial processes have embraced new characteristics, which lead to the poor performance of traditional soft sensor modeling methods. Deep learning, as a kind of data-driven approach, shows its great potential in many fields, as well as in soft sensing scenarios. After a period of development, especially in the last five years, many new issues have emerged that need to be investigated. Therefore, in this article, the necessity and significance of deep learning for soft sensor applications are demonstrated first by analyzing the merits of deep learning and the trends of industrial processes. Next, mainstream deep learning models, tricks, and frameworks/toolkits are summarized and discussed to help designers propel the developing progress of soft sensors. Then, existing works are reviewed and analyzed to discuss the demands and problems occurred in practical applications. Finally, outlook and conclusions are given.

In this article, a novel data-driven industrial quality predictor is proposed based on the cooperative deep dynamic feature extraction and variable time-delay (VTD) estimation. A semisupervised dynamic feature extracting (SSDFE) network is first proposed to extract nonlinear dynamic features to build a regression model for output quality prediction. Due to the inherent process structure and different positions of sampling instruments, time-delays commonly exist between process variables and quality variables, which may distort the original distribution and relationship in collected data. To recover the original process data pattern, the VTDs are regarded as model parameters and cooperatively obtained in the training process of the SSDFE network through an integer differential evolution algorithm. With the estimated VTD values, the reconstructed dataset further helps improve the prediction performance of the proposed SSDFE network. Two case studies are presented to demonstrate the superiority of the proposed method with VTD estimation.

This article proposes a distributed parallel modeling and monitoring framework for plant-wide processes with big data. The “distributed” contains two layers of meaning. One is the spatially distributed modeling and hierarchical monitoring for the plant-wide process with multiple operating units. The other represents the distributed parallel modeling for big process data with various features. Under the framework, the distributed parallel mixture probabilistic latent variable model is proposed based on the stochastic variational inference algorithm and the parameter server architecture to cope with the big process data. Then, the model is utilized to develop the plant-wide hierarchical and distributed process monitoring algorithms, where the multilevel monitoring indexes and fault contribution indexes are established based on the Bayesian fusion algorithm for process fault detection and diagnosis. The performance comparison and visualization for the industrial plant-wide process case has demonstrated the reliability and superiority of the proposed algorithm and framework.

The label scarcity problem widely exists in industrial processes. In particular, samples of some fault types are extremely rare; even worse, the samples of certain faults cannot be accessed, but they may appear in the actual process. These two kinds of challenges together can be termed as any-shot learning problem in industrial fault diagnosis. In this article, taking the advantages of generative adversarial network, a generative approach is proposed to tackle the any-shot learning problem, which generates the abundant samples for those rare and inaccessible faults, and trains a strong diagnosis model. To reach this, an attribute space is built to introduce the auxiliary information, which achieves the diagnosis of unseen faults and makes the generated samples more resembled to the real data. Besides, an auxiliary loss of triplet form is introduced as a joint training loss term, further improving the quality of augmented data and diagnosis accuracy. Finally, the performance of model is verified by the experiments of a hydraulic system and Tennessee-Eastman process, the results of which show that our method performs excellently for both zero-shot and few-shot fault diagnosis problems.

For large-scale industrial plants, quality-related process monitoring is challenging because of the complex features of multiunit, multimode, high-dimension data. Hence, a hierarchical quality monitoring (HQM) algorithm based on the distributed parallel semisupervised Gaussian mixture model (dp-S2GMM) is proposed in this article. In HQM, a large-scale process is first decomposed into a group of unit blocks according to the process structure. Subsequently, in each block, a quality regression model with multimode big process data is built using the dp-S2GMM, which is derived from a scalable stochastic variational inference semisupervised GMM (SVI-S2GMM). With the regression model, a hierarchical fault detection and diagnosis scheme in both quality-related and quality-unrelated subspaces is proposed from the variable level, block level to plant-wide level. Finally, an industrial case study on the Tennessee Eastman process demonstrates the feasibility and effectiveness of the proposed HQM algorithm.

Virtual sensors have established themselves as effective tools in process industries for online estimating variables that are crucial but difficult to measure. However, multimode industrial processes render developing high-accuracy virtual sensors quite challenging. The main difficulties lie in that, in multimode processes, the distributions of process data are strongly non-Gaussian and the mathematical relationships between the explanatory and primary variables are highly nonlinear. Even within one operating mode, the primary variables could depend on the explanatory variables in nonlinear ways. In order to address these issues, this article proposes a virtual sensing approach named Bayesian nonlinear Gaussian mixture regression (BNGMR) with the aid of single-hidden layer feedforward neural networks (SLFNs). In the BNGMR, a fully Bayesian model structure that absorbs the merits of SLFNs and the mixture models is designed. In addition, we develop a training algorithm for the BNGMR to realize predictive virtual sensor development based on variational inference. Extensive assessments of the performance of the BNGMR are carried out using both artificial example and real-world industrial processes. The experiments have demonstrated the predictive advantage of the BNGMR over several benchmark methods and also have provided practitioners with good illustrations.

Just-in-time learning (JITL), which can deal with both process nonlinearities and time-varying characteristics, has become a widely used tool for industrial soft sensing. High performance of JITL lies in selecting an accurate relevant sample set and developing a good base learner, which, however, still have some issues unresolved. In this article, a Bayesian JITL (BJTIL) is established to improve the performance of a JITL-based soft sensor in terms of relevant sample selection and base learner construction. The BJITL has dual implications. First, a semi-supervised relevant sample selection strategy with a mixture of Mahalanobis distances based on the fully Bayesian Dirichlet process mixture model is proposed, such that both labeled and unlabeled samples can be exploited, and complicated non-Gaussian distributions (such as those with multi-peaks or severe asymmetry) can be accounted for. Second, a weighted fully Bayesian Gaussian regression model with randomized mean and covariance is proposed as the base learner training algorithm so as to deal with the overfitting and numerical issues. Two real-world industrial processes are employed to evaluate the performance of the BJITL when it is applied to soft sensor development. The results demonstrate that the BJITL can achieve higher predictive accuracy in contrast with some state-of-the-art schemes for relevant sample selection and base learner construction. In addition, it is shown that the BJTIL can provide better predictive uncertainties.

Gaussian mixture model (GMM) has been widely used for soft sensor modeling of multimode industrial processes. However, it has been recognized that the performance of GMM deteriorates with the presence of outliers, which commonly exist in industrial datasets. In addition, samples with known labels in soft sensor applications are often rare because of expensive sampling equipment or time-consuming laboratory analysis. Shortage of labeled samples could lead GMM-based models to low prediction accuracy. To tackle such problems, a semisupervised robust soft sensor modeling method called “semisupervised Student's t mixture model (SsSMM)” is proposed. Like the GMM, the SsSMM employs finite mixture models to learn data distributions; nevertheless, with the virtual of the long tail property of Student's t distribution, the SsSMM possesses stronger robustness against outliers compared with the GMM. Moreover, the semisupervised model structure of SsSMM enables exploiting unlabeled samples of the SsSMM, such that the issues caused by insufficient labeled samples can be tackled. To identify model parameters of the SsSMM, we also develop an expectation-maximization-based training algorithm. Experimental results on numerical and industrial examples demonstrate that the proposed method is effective in: first, modeling multimode characteristics; second, exploiting unlabeled samples for performance improvement; third, dealing with distinct outliers (in synthetic dataset) and indistinctive outliers (in industrial dataset).

Due to the advantages of high prediction accuracy, least squares support vector machine (LSSVM) has been widely utilized for soft sensor developments in industrial processes. The hyper-parameters of LSSVM are often determined by minimizing the predicted error of validation set based on the intelligent optimization algorithm, which may lead to excessive optimization and model overfitting when validation set are selected improperly. Meanwhile, online parameters optimization is difficult to implement, which results in poor effect of local modeling. This paper proposes UMDA-LOS-LSSVM that is a LSSVM with parameters optimization in local objective set (LOS-LSSVM) by univariate marginal distribution algorithm (UMDA) based on the idea of local modeling. First, the local objective set is extracted in the candidate set based on the testing samples. Then, UMDA is utilized for minimize the predicted error of the objective set and provides the optimized parameters. Finally, training and testing of LSSVM are carried out based on the optimal parameters. In addition, this paper provides the distributed parallel form of the proposed method, which can be used for big data modeling and soft sensor development. The proposed method is applied in a CO2absorbing column unit to estimate the residual CO2content, which is implemented through an industrial big data distributed analytics platform. The results show a significant improvement of proposed method based soft sensor, compared to traditional methods.

Dynamic is one of the main bottlenecks in the industrial soft sensor application, due to the difficulties in representing and extracting dynamic data features. Meanwhile, an end-to-end deep network owns the ability to characterize sequence data information, but its fitting ability requires improvements in practical applications. In this article, an ensemble tree model with transferable and robust dynamic features extracted by a newly developed automatic dynamic feature extractor is proposed. First, the dynamic feature extractor with an encoding-decoding structure can provide effective dynamic features, which is equivalent to crossing and nonlinear mapping of sequences under the supervision of a decoder. Meanwhile, a new “regularization” method by smoothing dynamic features based on attention weights is proposed to denoise and alleviate the overfitting of the regressor after adding new features. Then, the extracted dynamic features can be transferred to the regressor with strong generalization ability, which takes into account the feature extraction of the deep network and the generalization of strong models. Finally, application results on a debutanizer distillation process show that the incorporation of robust dynamic features can significantly improve the soft sensing performance, compared to traditional methods. Moreover, the proposed model is further implemented through a cloud computing platform for industrial big data analytics.

Multivariate statistical process monitoring (MSPM) has been widely used in modern industries and most of traditional MSPM methods are developed using uniformly sampled measurements. However, process variables are often sampled with different rates in practical industries. On the other hand, most of the industries are dynamic processes in which the measurements are highly autocorrelated. Thus, it is difficult to build a dynamic process model with incomplete data sets in multirate processes. In this paper, a multirate linear Gaussian state-space model is exploited to deal with the above issues. Both the offline model training and online process monitoring schemes are developed in the present of incomplete multirate process data sets. The proposed method is validated through a numerical example and the Tennessee Eastman benchmark process.

Many fault detection and identification methods have been developed in recent years; each method works under its own assumption, which means a method that works well under one condition may not provide a satisfactory performance under another condition. In this paper, several data-based process monitoring methods are used in order to provide an effective monitoring scheme for processes under various conditions, and then the analytic hierarchy process approach is introduced for model prioritization. Compared with conventional ensemble systems, the proposed method is able to provide different priorities for different models in monitoring different process faults. Furthermore, a new fuzzy decision fusion system is designed for the purpose of online process monitoring. The effectiveness of the developed method is verified through using the Tennessee Eastman benchmark process.

Soft sensing of quality/key variables is critical to the control and optimization of industrial processes. One of the main drawbacks of data-driven soft sensors is to deal with the dynamic and nonlinear characteristics of process data. This paper proposes a deep learning structure and corresponding training algorithm for the purpose of soft sensor, which is called probabilistic sequential network. The proposed model merges unsupervised feature extraction and supervised dynamic modeling approaches to improve the prediction performance. It is mainly based on the Gaussian-Bernoulli restricted Boltzmann machine and the recurrent neural network structure. To avoid the overfitting problem in the training procedure of deep learning algorithms, the L2 regularization and dropout technique are adopted. The new method can not only deeply extract the nonlinear feature but also widely capture dynamic characteristic of process data. Effectiveness and superiority of the new method are validated through an actual CO2absorption column, compared to traditional methods.

An enhanced nonlinear Gaussian mixture regression (NLGMR) algorithm is proposed for quality prediction of a nonlinear multimode process. The traditional Gaussian mixture regression (GMR) model has been utilized for quality prediction with a linear model in each local mode, which will not suit for many cases that nonlinear relationships exist between input and output variables. Besides, large scales of process data that can be used for modeling are partially labeled on account of the low sampling rate of quality variables. Most of the unlabeled samples are discarded while building the GMR model, which leads to the loss of information and limits the improvement of prediction accuracy. To tackle these two problems, a locally weighted semisupervised factor analysis model is developed in each mode of GMR. The locally weighted model divides the nonlinear process into pieces of linear model and the semisupervised factor analysis model can effectively take advantage of the massive unlabeled data. Moreover, the variational inference (VI) algorithm is conducted on the GMR model to determine the amount of process modes automatically. The proposed method is first verified by a numerical example and then applied in a multimode primary reformer to predict the oxygen content, where prominent improvements are obtained, compared with traditional methods.

Generally, the measurements of modern industries are collected from different sources, which indicates that the traditional multivariate statistical process monitoring methods cannot be directly used in the multirate systems if one aims to utilize the complete multirate measurements. Hence, a set of multirate factor analysis models is developed for process modeling and fault detection purpose in the multirate processes. In the proposed model, the cross correlations are described and bounded by the common factors and the model parameters are calibrated using the expectation-maximum algorithms. Also, the proposed models are further discussed both from theoretical and geometric perspective. Finally, the proposed fault detection methods are tested by a simulated Tennessee-Eastman process and a real R2S anaerobic reactor unit in the wastewater treatment process.

Through calculating the similarity between the historical and the new query data samples, a probabilistic locally weighted prediction method based on supervised latent factor analysis (SLFA) model is proposed. In this method, the contributions of different historical samples are expressed through incorporating the similarity index into the noise variance of the process variables, which renders strong adaptability of the method for describing nonlinear relationships and abrupt changes of the process. Additionally, the proposed locally weighted method is extended to the semisupervised form, which is apparently more practical in real industrial processes, since the sampling rates of quality variables are much lower than those of ordinary process variables. Efficient expectation maximization algorithms are designed for parameter learning in both SLFA and semisupervised locally weighted LFA methods. Two real industrial processes are provided to evaluate the feasibility and the effectiveness of the newly developed soft sensors.

Independent component analysis (ICA) is widely used for modeling and monitoring non-Gaussian process. However, traditional ICA lacks probabilistic representation of process uncertainties. In this study, a probabilistic ICA (PICA) model is proposed for non-Gaussian process modeling and monitoring. The independent latent spaces are specified with Student’s${\rm t}$formulation to account for both Gaussian and non-Gaussian data characteristics while the additional noise term is further served as a complement for explaining underlying process uncertainties. The Student’s${\rm t}$distribution with adjustable tails is essentially an infinite mixture of Gaussians with various scaling variances. In order to monitor retained variations, the noise space is further extracted and analyzed with probabilistic principal component analysis (PPCA). Simulation results show that compared with the deterministic ICA-based method, the proposed two-stage probabilistic extraction method is more effective for monitoring non-Gaussian industrial processes.

Just-in-time learning (JITL) is a commonly used technique for industrial soft sensing of nonlinear processes. However, traditional JITL approaches mainly focus on equal sample sizes between process (input) variables and quality (output) variables, which may not be practical in industrial processes since quality variables are usually much harder to obtain than other process variables. In order to handle unequal length dataset with only a few labeled data, a novel semisupervised JITL framework is proposed for soft sensor modeling for nonlinear processes, which is based on semisupervised weighted probabilistic principal component regression (SWPPCR). In the new semisupervised JITL framework, traditional Mahalanobis distance and a new proposed scaled Mahalanobis distance are used for similarity measurement and weight assignment. By selecting the most relevant labeled and unlabeled samples and assigning them with the corresponding weights, a local SWPPCR can be built to estimate the output variables of the query sample. Case studies are carried out to evaluate the prediction performance of the proposed semisupervised JITL framework on a numerical example and an industrial process. The effectiveness and flexibility of the proposed method are demonstrated by the prediction results.

In order to deal with the modeling and monitoring issue of large-scale industrial processes with big data, a distributed and parallel designed principal component analysis approach is proposed. To handle the high-dimensional process variables, the large-scale process is first decomposed into distributed blocks with a priori process knowledge. Afterward, in order to solve the modeling issue with large-scale data chunks in each block, a distributed and parallel data processing strategy is proposed based on the framework of MapReduce and then principal components are further extracted for each distributed block. With all these steps, statistical modeling of large-scale processes with big data can be established. Finally, a systematic fault detection and isolation scheme is designed so that the whole large-scale process can be hierarchically monitored from the plant-wide level, unit block level, and variable level. The effectiveness of the proposed method is evaluated through the Tennessee Eastman benchmark process.

With the growing complexity of the modern industrial process, monitoring large-scale plant-wide processes has become quite popular. Unlike traditional processes, the measured data in the plant-wide process pose great challenges to information capture, data management, and storage. More importantly, it is difficult to efficiently interpret the information hidden within those data. In this paper, the road map of a distributed modeling framework for plant-wide process monitoring is introduced. Based on this framework, the whole plant-wide process is decomposed into different blocks, and statistical data models are constructed in those blocks. For online monitoring, the results obtained from different blocks are integrated through the decision fusion algorithm. A detailed case study is carried out for performance evaluation of the plant-wide monitoring method. Research challenges and perspectives are discussed and highlighted for future work.

For fault classification in industrial processes, the performance of the classification model highly depends on the size of labeled dataset. Unfortunately, labeling the fault types of data samples need expert experiences and prior knowledge of the process, which is costly and time consuming. As a result, semisupervised modeling with both labeled and unlabeled data have recently become an interest in industrial processes. In this paper, a kernel-driven semisupervised fisher discriminant analysis (FDA) model is proposed for nonlinear fault classification. Two discriminant analytical strategies are introduced for online fault assignment, namely k-nearest neighborhood and Bayesian inference. Detailed comparative studies are carried out through two industrial benchmark processes between the linear and kernel-driven semisupervised FDA models, in which the best fault classification performance is obtained by the kernel semisupervised model with Bayesian inference as its discriminant strategy.