Compared to the standard variational autoencoder (VAE), the Stacked VAE (SVAE) is widely recognized for its ability to extract high-level representations from strongly nonlinear data for soft sensor modeling. However, conventional stacking approaches still suffer from two key limitations: target-correlated information is not explicitly incorporated during the pretraining phase, and predictions rely exclusively on the final-layer features during fine-tuning, leaving valuable intermediate representations underutilized. To address these limitations, this paper proposes a novel soft sensor modeling framework based on an attention-gated, stacked, information-separation, target-supervised VAE (AGSISTVAE). The proposed model adopts a multi-layer VAE architecture to facilitate the progressive extraction of meaningful latent features. During layer-wise pretraining, the latent space is explicitly partitioned into output-correlated and output-irrelevant subspaces. By introducing a prior encoder to derive data-driven prior distributions conditioned on both inputs and outputs, target-correlated information is concentrated within the correlated subspace, enabling more effective extraction of predictive nonlinear features. Furthermore, an attention mechanism is incorporated during the fine-tuning stage to adaptively regulate information flow and fully exploit multi-level representations across all network depths. The proposed model is comprehensively validated on a numerical benchmark and a real-world debutanizer column industrial case study. Experimental results demonstrate that AGSISTVAE achieves superior predictive performance relative to existing state-of-the-art stacked architectures, yielding a low root-mean-square error of 0.00610 and a coefficient of determination of 0.9990 on the industrial case study. Moreover, the explicit separation and selective propagation of latent variables offer enhanced physical interpretability regarding which features are retained or discarded during the compression process.

The intricate and dynamic characteristics of time series data in industrial processes necessitates sophisticated data-driven soft sensor methodologies for modeling. However, the performance of such models often heavily hinges data quality, leading to potential neglect of critical spatial and temporal correlations essential for comprehending model behavior. To address these challenges, this study proposes a novel approach integrating spatiotemporal-based graph convolutional networks with attention-based preprocessing for soft sensor modeling. This framework not only enhances model reliability by extracting high-quality time series data but also captures localized spatiotemporal correlations vital for comprehending the intricate interactions among variables in the soft sensor. Additionally, the integration of attention-based preprocessing enables adaptive updates to the soft sensor model, thereby ensuring accurate prediction of process quality. The effectiveness and applicability of the proposed method are demonstrated through experimentation on both simulated processes and real-world applications in the ethylene oxide industry. Importantly, this data-driven approach to constructing transfer entropy aligns with fundamental a priori knowledge, enhancing the interpretability of the model results.

Soft sensing technology plays a crucial role in the real-time monitoring and optimization of key industrial variables. Recently, Transformers have emerged as a promising tool for soft sensor development, owing to natural advantages in dealing with long-range temporal correlations and complex nonlinearities. However, the conventional Transformer-based soft sensors are developed in a global learning framework, suffering from performance degradation in processes with multiple working conditions. To address this limitation, a conditional Gaussian mixture model (CGMM)-guided Transformer is developed in this article. Specifically, a CGMM, differentiating the physical properties and distributional discrepancies of manipulated and process variables, is first designed for high-accuracy recognition of the working conditions. Then, a condition-adaptive Transformer is proposed to accommodate variations in working conditions by capturing localized spatial-temporal characteristics based on the CGMM. Experimental results on both a numerical example and an industrial case demonstrate the superiority of the CGMM-Transformer over baseline models.

Generative latent variable models (GLVMs) are prevalent in developing soft sensors for predicting industrial process quality indices owing to their exceptional capability in extracting features, reducing data dimensionality, and explaining data generation mechanisms. Dynamical GLVMs (DGLVMs) are able to effectively deal with the dynamical characteristics of industrial data changes over time, exhibiting prominence in time-series prediction and dynamical process modeling. However, the existing DGLVM-based soft sensors share an intrinsic imperfection that they focus on capturing the temporal correlations between variables while disregarding the causal relationships between variables. Based on industrial data, it is difficult to recover the causalities between variables from the correlations, resulting in compromised generalization performance of the existing DGLVM-based soft sensors. In light of this limitation, a novel causal knowledge-assisted semi-supervised DGLVM (CK-SsDGLVM) is proposed, and an ad hoc parameter learning method based on the expectation–maximization (EM) algorithm is developed to train the CK-SsDGLVM. The performance of the CK-SsDGLVM is comprehensively evaluated using an artificial numerical case and an actual industrial process. The experimental results demonstrate that the CK-SsDGLVM could achieve superior generalization accuracy and interpretability compared to benchmark models.

Hidden Markov models (HMMs) for time-series data analysis are attracting wide interests in industries due to their ability to model the extensively existing dynamics and non-Gaussianities. In this paper, with the focus on industrial soft sensor applications, a semi-supervised robust hidden Markov regression (SsRHMR) model is first proposed to improve the performance of the HMMs in two challenging industrial scenarios, i.e., the scarcity of labeled samples and outlying data, which may prevent the HMMs from learning well-suited parameters. Furthermore, a distributed learning algorithm for the SsRHMR (termed D-SsRHMR) is developed to overcome the limitations of the HMMs in modeling large-scale time-series data, namely computational complexity and inability of handling long-period missing values. Performance evaluations of both the SsRHMR and D-SsRHMR are presented using a synthetic case and an actual process, based on which the effectiveness and feasibility of the proposed models and learning algorithms in improving the prediction accuracy and in accelerating the training speed have been demonstrated. Note to Practitioners—Before applying the SsRHMR to industrial soft sensing, we advise to first select features based on the process mechanisms and expert knowledge. That is, to carefully select the secondary variables so as to reduce the dimensionality of the input space. This is because, in general the lower the dimensionality of the secondary variables, the more accurate the estimated distributions of the secondary variables and the more efficient the training process for the SsRHMR. In addition, the D-SsRHMR would benefit from equal-sized subsets, since the efficiency of the distributed learning algorithm depends on the most computationally demanding slave computer, such as the one processing the largest number of data. Therefore, practically it is preferable for the D-SsRHMR to partition the entire time-series dataset with as equal size as possible.

Deep learning methods have been rapidly developed in recent decades. In this article, they are extended to model spatial-temporal industrial processes. Instead of pure black-box data-driven modeling approaches, the proposed model encodes the domain knowledge and physical rules governing the spatiotemporal system, called a dual-hierarchical recurrent neural network (DH-RNN). Both spatial and temporal relationships are modeled by multiple RNNs with diverse structures, which need correct specifications of all the interactions between spatial and temporal variables with a priori knowledge of the real process. A more accurate prediction can be obtained with fewer parameters employed in the network. And the effectiveness of the proposed DH-RNN is verified via a real ethylene oxychlorination process.

Semiconductor fabrication is regarded as one of the most complicated production processes with a high-mixed and uncertain production context, and it suffers from tool failures. Prognostics can predict the time when a process tool cannot perform its intended function. Therefore, preventive maintenance can be planned in time with the minimum impact on the production lines. Because most of the tool faults cannot be owed to the failure of a single component, it is challenging to model the system directly through mechanism analysis. Therefore, the development of failure prognostics in semiconductor manufacturing is somewhat impeded. Data-driven approaches are appropriate in this case, especially when the domain knowledge of the system under study is not comprehensive enough. This article attempts to propose a novel data-driven prognostic method integrated with the auto-associative regression and the Gaussian process to address the issue. This proposed method can extract the failure factors, establish the prognostic model and present the reliability of the model. On the basis of the built prognostic model, the failure tendency is predicted and the maintenance schedule can be determined. The validity and feasibility of the proposed method are demonstrated through a numerical example and a practical semiconductor manufacturing process, respectively. The proposed prognostic scheme can guide the frequency plan of preventive maintenance in the fabrication process and improve the productivity in semiconductor manufacturing.

Process complexities are characterized by strong nonlinearities, dynamics, and uncertainties. Monitoring such a complex process requires a high-quality model describing the corresponding nonlinear dynamic behavior. The proposed model is constructed using deep neural networks (DNNs) to represent the state transition and observation generation, both of which constitute a stochastic nonlinear state-space model. A new bidirectional recurrent neural network (RNN), creating a connection of the hidden layer between a forward RNN and a backward RNN, is proposed to generate the filtering estimation and the smoothing estimation of process states which further generate observations with DNN-based process models. The smoothing estimator and the process model are first learned offline with all collected samples. Then the filtering estimator is fine-tuned by the learned smoother and process models to achieve real-time monitoring since the filter state is estimated based on the past and the current observations. Two indices are designed based on the learned model for monitoring the process anomaly. The proposed process monitoring model can deal with complex nonlinearities, process dynamics, and process uncertainties, all of which can be very challenging for the existing methods, such as kernel mapping and stacked auto-encoder. Two case studies validate that the effectiveness of the proposed method outperforms the other comparative methods by at least 10% when using the averaged fault detection rate in the industrial experimental data.

Batch process quality prediction is an important application in manufacturing and chemical industries. The complexity of batch processes is characterized by multiphase, nonlinearity, dynamics, and uneven durations so that modeling of these batch processes is rather difficult. Moreover, there are other challenges in the face of quality prediction. Specifically, the process trajectories over the whole running duration potentially make specific contributions to the final targets so that the prediction issue embraces tremendously high-dimensional inputs but very low-dimensional outputs. This means that the prediction suffers from a severe dimensional imbalance between inputs and outputs. Motivated by these difficulties, this paper proposes a new deep learning-based framework for complex feature representative and quality prediction. Long short-term memory (LSTM) is used to extract comprehensive quality-relevant hidden features from a long-time sequence in each phase, significantly reducing the predictor dimensions. And these features from different phases are further integrated and compressed by a stacked auto-encoder (SAE). A practical industrial example testifies to the efficacy of the proposed framework.

Most data-driven process monitoring approaches consider the fault detection as a binary classification issue: normal or abnormal. All deviations from the nominal operating condition can trigger the same alarms. They fail to distinguish different fluctuation patterns and locate the positions of anomalies, such as the normal deviations in operating conditions, sensors faults, actuator faults, and process faults. A new process monitoring strategy based on orthogonal decomposition (OD) is proposed for the concurrent detection and location of different deviation patterns. OD is performed to discriminate the dynamics of data driven by measured disturbances and unmeasured disturbances in the same control system. This way, the original variable space is decomposed into the deterministic subspace and stochastic subspace. A dynamic principal component analysis-based subspace identification technique is used to construct the monitoring indices in the deterministic and stochastic subspaces, respectively. Two case studies show the validity of the OD-based process monitoring approach. Note to Practitioners-Fault diagnosis based on process data models always focused on analyzing variables' contributions to the anomaly in the past practice. But it is frequently difficult to decide the root causes just using the variables' contributions because different faults may induce a similar variation of the same variable. This paper provides a new scheme to locate the faulty components, including the sensor faults, actuator faults, process faults, and disturbance variations. It is more pertinent to learn about the fault locations than variables' contributions. Moreover, by locating faults first and then figuring out variables' contributions to a specific location, more detailed and precise diagnosis conclusions can be drawn when being compared with the results of using the variables' contributions in a global system. This new method is purely data driven and it has no demand for complex process knowledge.

The prediction accuracy of the traditional kernel with data-driven regression methods strongly depends on the appropriate selection of the kernel function and aims at solving the nonlinearity of the input space. In this paper, a self-learning kernel regression model is proposed. A special kernel space from the measured data is learned and designed, so that combined with dimension reductions on the input variables, the regression behavior between the projected input variables and the output variable is found. The model is posed as a semidefinite programming problem with the objective function to find the maximum variance between the learned manifolds. The kernel is data dependent and can be generated online whenever a new data point is available. The effectiveness of the proposed algorithm is demonstrated through the case studies on a simple nonlinear system and a real semiconductor process.

With the growing complexity of the modern industrial process, monitoring large-scale plant-wide processes has become quite popular. Unlike traditional processes, the measured data in the plant-wide process pose great challenges to information capture, data management, and storage. More importantly, it is difficult to efficiently interpret the information hidden within those data. In this paper, the road map of a distributed modeling framework for plant-wide process monitoring is introduced. Based on this framework, the whole plant-wide process is decomposed into different blocks, and statistical data models are constructed in those blocks. For online monitoring, the results obtained from different blocks are integrated through the decision fusion algorithm. A detailed case study is carried out for performance evaluation of the plant-wide monitoring method. Research challenges and perspectives are discussed and highlighted for future work.