Data-driven soft sensors have been widely applied to estimating important yet difficult-to-measure quality-relevant variables in industrial processes. The complex industrial data exhibits nonlinearity and dynamics due to changes in operating conditions. Soft sensors developed based on the assumption of identical and independent distributions often struggle to adapt to target domain data with significant distribution discrepancy, which poses a great challenge to traditional soft sensor approaches. Additionally, the dynamic features embedded in the practical industrial processes are of great importance for an accurate soft sensor. Existing soft sensor approaches pay little attention to the combined challenge of distribution discrepancy and dynamic feature transfer, referred to as the dynamic domain adaptation challenge. In this work, we propose a Self-modified Dynamic Domain Adaptation (SDDA) soft sensor approach to solve this problem. We develop a novel sequential optimization framework for dynamic domain adaptation, where the target samples are progressively incorporated and pseudo-labels are iteratively refined to preserve the underlying temporal dependency and enable efficient dynamic feature transfer. Also, we propose a feature alignment with the transfer component analysis (TCA) to avoid potential significant distribution discrepancy and guarantee a stable prediction. We demonstrate the superiority of the proposed method via two real-world industrial cases.

Due to operating condition drift, environmental changes, and system oscillations, industrial processes often exhibit nonstationary characteristics that involve both stable long-term trend and fluctuant short-term dynamics. In this article, a novel multiscale gated structure model (MGSM) is proposed for nonstationary process soft sensing, which includes long-term memory chain (stable and low frequency) and short-term dynamic chain (respond to fluctuations). The information decomposed from input data is introduced into the MGSM to learn long-term dependency relationships and dynamic behavior in the nonstationary process. In addition, a novel two-dimensional random missing function is designed to handle randomly missing data, which fully considers the data missing in variable-wise and time-wise dimensions. The proposed model is further constructed for the soft sensing of nonstationary processes with random missing data. Finally, application studies to the Tennessee Eastman process and a thermal power generating process show that the proposed method has significant advantages in the quality prediction of nonstationary process.

Temporal feature representation is critical for soft sensor modeling in industrial time sequences. Deep learning networks like long short-term memory are often used to model the temporal dynamics of data sequences. However, the data collected from industrial plants are usually sampled with irregular frequency, making it challenging for traditional methods to handle these temporally changeable relationships. Therefore, a Gaussian-based interval-aware transformer (GIA-Trans) with interval embedding is proposed in this article to model industrial data with irregular sampling frequency. In GIA-Trans, positional and temporal embedding layers are established to take positional distances and time intervals of samples into account. Then, Gaussian-based time-aware attention is proposed to tackle the changeable time intervals with adaptive weights. In this way, the temporal correlations between samples can be adaptively captured. The GIA-Trans is applied to an industrial hydrocracking process to predict the C5 and C6 content of light naphtha.

Accurate feature extraction and quality variable prediction are critical problems for time sequences in industrial processes. However, industrial samples often exhibit strong temporal correlations with each other that have different positional distances, making it challenging for conventional data-driven models like long short-term memory (LSTM) and Vanilla transformer to capture these underlying features. In this article, a difference metric attention with position distance-based weighting is proposed for transformer (DMA-trans) in industrial time series modeling. First, the DMA is established to calculate the difference of query-key vector pair in transformer to measure the spatial similarity. In this fashion, the difference can accurately represent the spatial similarity of vectors, compared with the original dot product directly on two vectors. Then, positional distance-based weights are designed to capture the sample relevance that has different positional distances. This may help to extract more potential features because the closer samples tend to have higher relevance while there may be weak correlations if two samples are far in positional distance. The effectiveness of the DMA-trans model is validated in industrial hydrocracking processes for C5 content of the light naphtha and the final boiling point of the jet fuel.

Inspired by the idea of deep learning, several latent variable models have been successfully extended to the deep forms for industrial data analytics. Compared to traditional deep neural networks, deep latent variable models are more fitted to the requirement of data-driven modeling and applications in industrial production systems, due to the lightweight model structure and high-efficient data analytics process. The aim of this article is to develop a deep latent variable predictive modeling framework, which is based on a newly designed Bayesian soft attention mechanism. Instead of only using extracted features from the last hidden layer for predictive modeling, different attentions are focused across all hidden layers of the deep model. As a result, different layer-wise models make different contributions in predicting different data patterns, through which the ability of the deep latent variable model will be further explored. Two real industrial examples are provided for performance evaluation and comparative studies among different prediction models, based on which the superiority of the online Bayesian soft attention mechanism has been confirmed.

The prediction of key quality variables plays an important role in industrial status identification and monitoring. Due to process disturbance and hard device limitation, data collection in modern industries often exhibits high noise and irregular data sampling. To solve the above problems, this article proposes a stacked supervised and reconstructed input denoising autoencoder integrated with internal attention long short-term memory (SSRDAE-IALSTM) network for soft sensing modeling. First, a stacked supervised and reconstructed input denoising autoencoder (SSRDAE) is designed. Compared with the original DAE, each supervised and reconstructed input DAE (SRDAE) can simultaneously reconstruct the process data and quality data at the output layer, aiming to reduce information loss and extract quality-related features. Second, the denoised features are fed into the interval attention LSTM (IALSTM) to adjust the influence of different historical samples on the current sample in irregular sampling data to capture long-term temporal features. Finally, performance validations are carried out on an industrial debutanizer column and a penicillin fermentation process. The experimental results show that the proposed model can enhance the learning ability of process features and obtain better prediction performance than other comparison methods.

The growth of data collection in industrial processes has led to a renewed emphasis on the development of data-driven soft sensors. A key step in building an accurate, reliable soft sensor is feature representation. Deep networks have shown great ability to learn hierarchical data features using unsupervised pretraining and supervised fine-tuning. For typical deep networks like stacked auto-encoder (SAE), the pretraining stage is unsupervised, in which some important information related to quality variables may be discarded. In this article, a new quality-driven regularization (QR) is proposed for deep networks to learn quality-related features from industrial process data. Specifically, a QR-based SAE (QR-SAE) is developed, which changes the loss function to control the weights of the different input variables. By choosing an appropriate inductive bias for the weight matrix, the model provides quality-relevant information for predictive modeling. Finally, the proposed QR-SAE is used to predict the quality of a real industrial hydrocracking process. Comparative experiments show that QR-SAE can extract quality-related features and achieve accurate prediction performance.

For industrial processes, it is significant to carry out the dynamic modeling of data series for quality prediction. However, there are often different sampling rates between the input and output sequences. For the most traditional data series models, they have to carefully select the labeled sample sequence to build the dynamic prediction model, while the massive unlabeled input sequences between labeled samples are directly discarded. Moreover, the interactions of the variables and samples are usually not fully considered for quality prediction at each labeled step. To handle these problems, a hierarchical self-attention network (HSAN) is designed for adaptive dynamic modeling. In HSAN, a dynamic data augmentation is first designed for each labeled step to include the unlabeled input sequences. Then, a self-attention layer of variable level is proposed to learn the variable interactions and short-interval temporal dependencies. After that, a self-attention layer of sample level is further developed to model the long-interval temporal dependencies. Finally, a long short-term memory network (LSTM) network is constructed to model the new sequence that contains abundant interactions for quality prediction. The experiment on an industrial hydrocracking process shows the effectiveness of HSAN.

Industrial intelligent systems (IIS) play a huge role in modern industry, and their intelligent models of IIS enable diagnosis of faults, key performance indicator (KPI) prediction, and other important industrial process analysis in a data-driven way. However, the performance of intelligent models is limited by the quantity and quality of local data in specific factories. At the same time, the privacy information and security concerns contained by industrial data lead to the problem of information silos in industry. This hinders data sharing and cross-factory collaborations. To address these issues, this article makes the following contributions. First, for the first time, we empower industrial federated big data analytics (IFBDA) of IIS with swarm learning, and propose a hyperledger fabric-based IFBDA blockchain (IFBDAchain) for multifactory information sharing and collaborative modeling. Second, in the IFBDAchain, we further consider potential dishonest behaviors among federated members, and design verification and privacy protection mechanisms to ensure trustworthiness of analytics. Third, we validate the IFBDAchain with two real industrial cases. The results demonstrate the effectiveness of the IFBDAchain in fault classification and KPI prediction tasks in industry. Compared to the average values of local learning, our method increases the classification accuracy by 27.6%, 69.4%, and 33.1% under Independent and identically distributed (IID), non-IID, and unbalanced conditions, respectively. Furthermore, the root-mean-square error of the KPI prediction decreases by 33.3%, 49%, and 45.7% for the IID, non-IID, and unbalanced conditions, respectively, indicating its significant potential as a generic backbone for industrial federated Big Data analytics.

Data-driven soft sensing has become quite popular in recent years, which can provide real-time estimations of key variables in industrial processes. While the introduction of deep learning does improve the prediction performance, it is highly restricted to the number of labeled training data, as well as large computational burden and cumbersome parameter tuning procedures. How to break through the bottleneck of data-drive models in terms of limited labeled data and high computational complexity should be one of the main recent focuses in the field of industrial soft sensing. In this article, a deep co-training PLS (deep CT-PLS) model is proposed to extend the ordinary PLS model to the semi-supervised deep form. While the deep model can efficiently extract inherent natures of process data, the co-training strategy makes lots of unlabeled data useful through a two-view cross training and annotation process. In this case, the performance restriction of the deep PLS model can be greatly relieved, with the incorporation of additional unlabeled data, while at the same time the designed model structure keeps in a low computational complexity. Based on the case study on a real industrial production process, the deep CT-PLS model can significantly improve the soft sensing performance.

Soft sensors have been increasingly applied for quality prediction in complex industrial processes, which often have different scales of topology and highly coupled spatiotemporal features. However, the existing soft sensing models usually face difficulties in extracting the multiscale local spatiotemporal features in multicoupled complex process data and harnessing them to their full potential to improve the prediction performance. Therefore, a multiscale attention-based CNN (MSACNN) is proposed in this article to alleviate such problems. In MSACNN, convolutional kernels of different sizes are first designed in parallel in the convolutional layers, which can generate feature maps containing local spatiotemporal features at different scales. Meanwhile, a channel-wise attention mechanism is designed on the feature maps in parallel to get their attention weights, representing the significance of the local spatiotemporal feature at different scales. The superiority of the proposed MSACNN over the other state-of-the-art methods is validated through the performance evaluation in two real industrial processes.

In recent years, more and more open environment have led to confidential links and data exposure, which seriously threatens the security of industrial systems. Adversarial attacks can easily fool machine learning models by adding tiny perturbations to input data. Industrial fault detection and classification (FDC) system is an indispensable part of ensuring production safety, but it is also not immune to the impact of adversarial risks. Once it is under attack, the disastrous consequences that may be caused to the industrial system are unimaginable. Adversarial training is among the most effective defense methods to protect those data-driven intelligent systems. This article studies a novel reweighted adversarial training approach called adversarial weight prediction networks. By assigning more appropriate weights to different data samples, we can make better use of the limited model capacity of the industrial FDC system. Particularly, predicting weights through a synchronized network overcomes the limitations of insufficient information and nontransferability of existing statistical methods. Performance evaluation on three industrial cases containing structured and image data shows the superior generalization and stability of our proposed method.

In complex process industries, multivariate time sequences are omnipresent, whose nonlinearities and dynamics present two major challenges for soft sensing of important quality variables. Consequently, due to the potent representational capabilities, nonlinear dynamic models like gated recurrent unit (GRU) and long short-term memory (LSTM) networks have been used for data sequence modeling. Though it is a common occurrence in many industrial plants, data series with heterogeneous sample intervals and missing values cannot be directly handled by these dynamic algorithms. To this end, attention-based interval-aided networks (AIA-Net) are proposed in this article to adaptively model the temporal information for heterogeneous sampling sequences with missing values in the processes industry. It includes two main mechanisms, which are named attention-based time-aware dynamic imputation and interval-aided time-aware network, respectively. The reduction rate is introduced by the attention-based time-aware dynamic imputation to apply the effects of time intervals and is used in the imputation of missing data. The interval-aided time-aware network includes time intervals in the model structure and uses a sampling interval gate to correct the temporal correlations in time series. The proposed AIA-Net is successfully applied to a real hydrocracking process to predict the C5 and C6 content in the light naphtha.

Fault detection and fault identification are consecutive steps of multivariate statistical process monitoring. In recent years, increasing attention has been paid to process dynamics. For dynamic process modeling, canonical variate analysis (CVA) extracts process dynamics effectively. However, process noises are not well analyzed in traditional CVA and corresponding fault identification methods are less studied. To solve these issues, a variational Bayesian CVA (VBCVA) model is proposed for dynamic process monitoring. Through a probabilistic perspective, the inevitable noises in realistic industrial processes can be captured in the new model. Moreover, the proposed model is further extended in the variational Bayesian framework to overcome the common problems in probabilistic methods. Besides, an improved fault identification approach based on fault relevance is introduced, which avoids the smearing effect caused by data reconstruction. Finally, the feasibility of the proposed process monitoring scheme is verified on the TE benchmark and a real wastewater treatment process.

Loop pairing is one of the major concerns when designing decentralized control systems for multivariable processes. Most existing pairing tools, such as the relative gain array (RGA) method, have shortcomings both in measuring interaction and in integrity issues. To evaluate the overall interaction among loops, we propose a statistics-based criterion via enumerating all possible combinations of loop statuses. Furthermore, we quantify the traditional concept of integrity to represent the extent of integrity of a decentralized control system. Thus, we propose that a pairing decision should be made by taking both factors into consideration. Two examples are provided to illustrate the effectiveness of the proposed criterion.