Soft sensors are essential for online prediction of key quality variables in industrial processes. However, practical process data are often affected by asynchronous sensor sampling, delayed laboratory analysis, and heterogeneous update frequencies, resulting in pronounced multi-rate characteristics. Most existing methods rely on interpolation, resampling, or regular discrete-time modeling, which makes it difficult to preserve actual observation times, observation staleness, and continuous dynamics under asynchronous sampling conditions. To address this issue, this paper proposes an asynchronous multi-rate continuous-time Transformer, termed AMCT-Former, for industrial soft sensing. The proposed method first organizes multivariate observations within a historical window into a chronologically ordered event stream and constructs a rectilinear control path to represent time progression, latest observations, and observation staleness. An NCDE-inspired continuous-time encoder is then employed to learn the continuous-time evolution of process states. Furthermore, a variable-wise Transformer is introduced to characterize dynamic cross-variable dependencies, while a target-aware temporal Transformer adaptively aggregates prediction-relevant historical information. In this way, AMCT-Former enables unified modeling of continuous-time dynamics, cross-variable dependencies, and target-related historical features without enforcing explicit time alignment. Case studies on two real-world industrial processes demonstrate the effectiveness and superiority of the proposed method for asynchronous multi-rate soft sensing.

Modern industrial processes are rapidly evolving toward intelligent operation, creating new challenges for fault diagnosis in complex systems. This article presents a knowledge-data integrated graph convolutional network (KDIGCN) that combines domain knowledge with data-driven strategies. The method partitions process variables into subgraphs based on physical mechanisms and uses temporal convolutional networks (TCNs) to construct causal adjacency matrices, effectively integrating prior knowledge with temporal dependencies. An abnormal feature enhancement mechanism improves sensitivity to fault indicators, while multiscale convolutional neural networks (MS-CNNs) enable spatiotemporal feature fusion across different time scales and subgraphs. Extensive experiments on the Tennessee Eastman (TE) benchmark demonstrate that KDIGCN achieves superior diagnostic accuracy and robustness compared to state-of-the-art methods, particularly for similar faults and unknown fault scenarios.

The rapid development of soft sensors has significantly enhanced industrial operations by promoting sustainability, safety, and efficiency. However, modern process industries involve highly dynamic systems with multi-mode and nonlinear data, posing challenges for conventional soft sensor models that assume uniform data distributions. Additionally, limited target mode data hinders effective training. To address these issues, we propose the Seasonal-Trend Variational Transformer Network (ST-VTN), a deep probabilistic model based on seasonal-trend decomposition. ST-VTN employs a Transformer-based encoder to extract seasonal and trend patterns from multi-mode data, enhancing latent Gaussian feature representation. A mode discriminator further improves mode-specific feature learning by distinguishing trend characteristics across modes. Built on a variational Bayesian framework, ST-VTN models the regression between latent features and quality variables. A two-stage pre-training and fine-tuning strategy enables effective use of source mode data, even with scarce target mode samples. Experiments on gas turbine and sulfur recovery datasets confirm ST-VTN’s superior performance for multi-mode regression tasks.

In various industrial processes, soft sensors have become important tools for predicting key quality variables. However, traditional soft sensor models only use data from the same sampling moments as the key quality variables, thus wasting information from other sampling moments. In light of this, a novel soft sensor model named triple-gated bidirectional variational pyramid network (G3-BiVPN) is proposed. Within G3-BiVPN, the multirate dataset is first segmented into multiple datasets each with a single sampling rate. For each dataset, a corresponding bidirectional variational autoencoder (BiVAE) is utilized for feature extraction. BiVAEs are used as backbone models to form a bidirectional pyramid structure. A triple gating mechanism consisting of attention gate (AG), temporal gate (TG), and spatial gate (SG) is integrated into BiVAEs to regulate information flow. Information can be flowed bidirectionally through different levels, with each level establishing a regression relationship with the key quality variables and selecting the optimal level as the final output. The core advantage of G3-BiVPN lies in its utilization of the multirate nature of the data. Finally, the efficiency of G3-BiVPN has been validated through two sets of real-world industrial process data with multiple sampling rates.Note to Practitioners—Due to sensor specification differences or practical needs, industrial process variables often exhibit a multiplicity in sampling rates. The core concept of G3-BiVPN is the bidirectional transmission and fusion of features at different rates. Initially, multirate features are extracted using BiVAEs and filtered through AGs to retain the relevant feature information for the output. Subsequently, these filtered features are stacked, with higher sampling rate features forming the bottom layers and lower sampling rate features occupying the top layers. In this architecture, feature information from different sampling rates can freely flow bidirectionally between the layers. During the downward transmission of information, upsampling is performed. To mitigate issues of information redundancy, further filtering is applied through TG after the upsampling. And, during the upward transmission, downsampling is employed. To prevent loss of detailed information during downsampling, SG is utilized before downsampling. This bidirectional propagation enhances the effective fusion of information across different layers, ensuring comprehensive understanding of the data at each level. Through this intricate feature interaction, G3-BiVPN can adapt to the complexity of multirate data, effectively utilizing available information from different sampling rates to accurately predict key quality variables.

In actual industrial scenarios, different operating modes and workloads can lead to multiple modes of working conditions, resulting in significantly diverse feature spaces. However, the heterogeneity and complexity among these modes pose a challenge to traditional data processing methods. Therefore, this paper proposes the cross-modality manifold adaptive Network (CMAN) to facilitate cross-modal information transmission for addressing multi-modal prediction issues. Specifically, CMAN divides the prediction process into two steps. Firstly, the manifold discriminative autoencoder (MDAE) is proposed to extract both local and global manifold geometric structures. The loss function of the designed MDAE in mode recognition is formulated to minimize the ratio between within-modal and between-modal features. In this way, the autoencoder not only learns data representations but also learns to differentiate between data from different classes. This lays the foundation for determining fusion strategies between modes in subsequent steps. Secondly, in the process of multimode prediction, to assist the model in learning and understanding the mutual influences and dependencies between different modes, CMAN shares features between modes through cross connections. It can adaptively preserve task specificity while also utilizing between-task correlations. The effectiveness of the proposed method is validated in the Tennessee Eastman (TE) case and an actual power plant case. Note to Practitioners—The use of soft sensors to monitor key variables of multimode processes is essential for optimizing and controlling chemical processes. However, it is difficult for conventional methods to accurately and comprehensively utilize within- and between-modal information of multimode processes to build robust and powerful soft sensors. In addition, it is difficult to obtain mode-indicating variables in real-world processes. To address these issues, CMAN is proposed in this paper. Firstly, the historical data of each mode in a multimode industrial process are collected, and the CMAN utilizes the manifold discrimination idea to build a mode recognition model. Then, when modeling the specific modes, CMAN utilizes cross-connections to migrate knowledge between modes, which not only considers the information of the modes themselves, but also makes the features between modes cross-transferred. The gating mechanism enables adaptive optimal combination between various types of features. Finally, two sets of cases show that the proposed method has excellent prediction performance.

In the process industry, the challenge of missing data significantly impairs the efficacy of data-driven process monitoring systems and soft sensor modeling, particularly due to issues, such as unbalanced sampling intervals and sensor malfunctions. Process data, inherently nonlinear and characterized by spatiotemporal coupling, are prone to distribution shifts, which traditional imputation techniques often fail to address comprehensively. To overcome these limitations, this article introduces a novel data imputation framework, termed multiscale spatiotemporal information embedding with asymmetrical Transformer (MSST-Former). This framework reconceptualizes the missing data problem by integrating both global and local perspectives on time series and input variables. The proposed approach initiates with a hybrid 1-D convolutional network module that effectively captures local spatiotemporal correlations and dependencies within the time-series data. This is followed by an encoder-decoder structure, incorporating an inverted Transformer (iTransformer) in conjunction with a Transformer block, to embed series representations with a focus on long-term multivariate correlations and overarching spatiotemporal dependencies. Finally, a multilayer residual network executes the data imputation by leveraging the features embedded at multiple scales. Comparative experiments with several baseline and state-of-the-art models on two real-world industrial datasets verify the superiority and robustness of the proposed MSST-Former.

Variational auto-encoders (VAEs) have been widely used in soft sensing due to their ability to provide a probabilistic description of the hidden space. However, VAEs are static models that do not consider process dynamics, which can limit the ability of VAEs to accurately model complex industrial processes. To tackle this problem, this paper proposes a model called adaptive multi-head self-attention based supervised VAE (AMSA-SVAE). In AMSA-SVAE, an adaptive multi-head self-attention mechanism (AMSA) is proposed based on the multi-head self-attention mechanism (MSA). AMSA can dynamically extract different attention information depending on specific tasks. By adjusting the attention weights based on the input sequence, AMSA allows for more accurate and efficient modeling of complex industrial processes. Then, AMSA is used as the encoder and decoder of SVAE for soft sensing. Furthermore, with the data generation capabilities of VAE, an adaptive multi-head self-attention based VAE (AMSA-VAE) framework is proposed to address the issue of missing data. The AMSA-VAE is used to dynamically fill in missing data, thereby extending the capabilities of AMSA-SVAE. Finally, the performance of AMSA-SVAE is verified by a set of real industrial data, and the ability of AMSA-VAE framework is demonstrated by simulating different degrees of data missing rates. By combining the dynamic modeling capabilities of AMSA-SVAE with the data generation capabilities of AMSA-VAE, the proposed approach provides a robust solution to the challenges of incomplete data in soft sensing.Note to Practitioners— Soft sensors are widely used to measure key parameters in industrial processes, but missing values in the data are common due to sensor failures or transmission signal interference. This poses a significant challenge for traditional soft sensors, which require complete data to accurately model. Meanwhile, the dynamic nature of industrial process data further complicates the modeling process. To solve these challenges, this paper proposes an AMSA-SVAE model for soft sensing and an AMSA-VAE framework for filling in the missing values in the data, thereby extending the capabilities of AMSA-SVAE to handle missing data. When facing a dataset with missing values, AMSA-VAE framework is first used to fill in the missing values before the filled complete data is fed into AMSA-SVAE for modeling. Finally, the proposed approaches are evaluated through two sets of experiments using a real industrial dataset, showing the excellent performance of AMSA-SVAE and AMSA-VAE framework in modeling dynamic industrial process data and addressing the missing data problem.

Nowadays, the deep learning technique has been widely applied in soft sensing benefiting from its strong ability of feature representation. However, classic dynamic soft sensing methods based on deep learning do not consider spatial-temporal and quality-relevant information simultaneously in the process data. Additionally, feature extraction methods encounter the challenge of information redundancy when dealing with large datasets. To address these challenges, we propose a novel hybrid self-attention mechanism with distributed GRUs (HSAM-dGRUs) within an encoder-decoder framework. The HSAM-dGRUs architecture contains a hybrid self-attention encoder, integrating multi-channel multi-head self-attention (MC-MSA) and quality-related self-attention (QR-SAM). The MC-MSA method effectively extracts local features through multi-channels to capture spatio-temporal characteristics in sequence data. Subsequently, the QR-SAM method introduces supervisory knowledge into the features and capture the quality-relevant information adaptively. The distributed GRUs decoder is designed to extract local dynamic hidden states from multi-channel features for accurate prediction. Two case studies on real industrial process datasets demonstrate the effectiveness and superiority of the proposed HSAM-dGRUs soft sensing approach.Note to Practitioners—In the process industry, monitoring quality variables is beneficial for tracking process status, saving energy, and reducing emissions. However, it is difficult to implement accurate measurement and real-time control for process quality in general. In this work, we propose a novel hybrid self-attention mechanism utilizing distributed gated recurrent units for soft sensing modeling of dynamic processes. The proposed method integrates an multi-channel multi-head self-attention mechanism and a quality self-attention mechanism into the encoder. The encoder is responsible for adaptively extracting spatio-temporal features and assigning weights based on quality-related information. Furthermore, a decoder based on distributed GRUs is employed to capture the dynamic information from the extracted features, enhancing the prediction task. First, the proposed methods can be trained offline based on historical dataset. Then, the well-trained model can be uploaded and use the new data for accurate online prediction. Preliminary experiments in this paper have demonstrated the feasibility of the proposed method, but it has not been tested in actual industrial production. In the future, we will work on developing a quality prediction system based on the proposed method so that it can be applied in actual industrial production.

Effective dimensionality reduction (DR) and classification in fault diagnosis remain a significant challenge, primarily due to the increasing scale of industrial processes and the non-linear and high-dimensional features of process data. To address this challenge, we present local tangent space alignment (LTSA) integrated with a multi-distance adaptive order morphological filter (MARMF) fault diagnosis method (LTSA-MARMF). In LTSA-MARMF, LTSA that preserves the local manifold structure using tangent space is first utilized for DR to provide the required feature space data for ARMF. Next, the cosine distance and dynamic time warping distance are introduced into the distance error of ARMF, considering the spatial similarity and dynamic features to improve classification accuracy. Finally, the distance-matching result of the pattern is applied to determine the type of fault. Through simulations, it is evident that LTSA-MARMF can achieve more satisfactory fault diagnosis accuracy than other related methods on the Tennessee-Eastman process (TEP) and the actual Grid-connected PV System (GPVS).Note to Practitioners—This paper is inspired by the difficult-to-handle high-dimensional and non-linear features of process data but is also applicable to high-dimensional and non-linear data from other industrial processes. The DR and classification are important aspects of fault diagnosis. In this paper, a novel pattern-matching method is utilized for fault diagnosis, which uses LTSA and the modified multi-distance ARMF for DR and classification, respectively. In terms of mathematics, the distance error of ARMF is analyzed. The combination of multi-distance is used to enhance the accuracy of fault diagnosis. The preliminary experiments show that LTSA-MARMF is feasible, but has not been tested in the plant. We will consider testing LTSA-MARMF in an actual plant in future research.

As the production demand and external environment change, the same production process may have multiple stable working conditions, i.e., multimode process. The traditional process monitoring methods cannot be directly applied to industrial data with multipeak distribution. In order to address the multimode process monitoring problem, a cross-modality shared-specific network (CMSS-Net) is proposed in this article. First, to address the problem of unavailability of mode indicator variable, CMSS-Net adds a loss term based on discriminative idea to the loss function, which improves the mode recognition ability by maximizing the interclass distance and minimizing the intraclass distance. The multimode process is then monitored. Since different modes originate from the same production process, there exists some common information among modes. CMSS-Net extracts the shared information by minimizing the difference in the distribution of features among modes. At the same time, the gating mechanism is used to fuse the shared information between modes and the unique features of the modes into a multivariate feature fusion. This improves the performance of the model due to the information being enriched across the modes, while increasing the transparency of the model's decision-making process. Finally, the proposed method is developed as soft sensors for Tennessee Eastman process and power plant gas turbine emission process. It is compared with some popular methods. The experimental results demonstrate the effectiveness and superiority of CMSS-Net when applied to the multimode process.

It has become a big challenge and a hot topic of research to capture the most relevant features from high-dimensional process data for enhancing fault diagnosis. To effectively extract discriminative features from high-dimensional data, a novel dimensionality reduction (DR) approach named neighborhood and locality projections with the farthest and nearest distance (FNDNLP) is first proposed for industrial process fault feature acquisition and diagnosis. By constructing intraclass weights and interclass weights, FNDNLP takes both the intraclass distance and the interclass distance into consideration in its objective function, improving the diagnostic ability of extracted features through maximizing the interclass distance, and minimizing the intraclass distance. In addition, bootstrap-based FNDNLP (BFNDNLP) is further proposed to handle the matrix decomposition problem in FNDNLP. To find the proper order through DR, the Akaike information criterion is adopted. Finally, the Naïve Bayes based classifier is utilized to achieve acceptable fault diagnosis. The simulation results from two complex industrial cases indicate that the proposed methodology can achieve higher diagnosis accuracy than other related methods. What is more, the DR features are further analyzed to show the effectiveness and benefits of the proposed BFNDNLP extraction approach.

Data-driven fault diagnosis has attracted attention with the recent trend of obtaining representative features from high-dimensional, strongly coupled, and nonlinear process data. This article presents a novel dimensionality reduction (DR) algorithm named double preserving integrated with neighborhood locality projections (DPNLP) for fault diagnosis. To further solve the singular matrix problem in DPNLP, the regularization-based DPNLP (RDPNLP) that introduces the regularization into DPNLP is finally presented. In RDPNLP, first, the double preserving weight that can both preserve neighborhood similarity and preserve local linear reconstruction is utilized to make the neighbors in the same class close to each other and the neighbors from different classes far apart. Additionally, regularization is applied to solve the singular matrix problem enhancing the ability of DR. Akaike information criterion is utilized to determine the order of DR when using RDPNLP. Through simulations on two compound multifault cases, it can demonstrate that the presented RDPNLP could achieve higher performance in fault diagnosis than other related methods.

In complex industrial processes, the technique of fault diagnosis has been playing an increasingly considerable role in ensuring the safety of life and property. Unfortunately, the process data of complex industrial processes have the features of high dimension. Feature extraction from high-dimensional data is promising to coping with the fault data with high dimension. Recently, one of manifold learning methods named discriminant locality preserving projection achieves excellent performance in feature extraction. However, the performance of discriminant locality preserving projection (DLPP) is subject to the problem of matrix decomposition in the denominator of the objection function caused by the small sample size (SSS) issue. To overcome this limitation, novel DLPP integrated with Monte Carlo sampling is proposed to enhance the performance of feature extraction through dimensionality reduction. In the proposed MC-DLPP, Monte Carlo sampling is first utilized to generate fault samples for each fault type. With the aid of the virtually generated fault samples, the rank of the matrix in the denominator of the objection function of DLPP increases, thus well addressing the SSS problem. The Softmax classifier is used for fault diagnosis. To test the performance of the improved DLPP-based fault diagnosis, case studies using the Tennessee Eastman process are carried out. Simulation results confirm the presented MC-DLPP achieves superior accuracy in fault diagnosis.

Data-driven fault diagnosis techniques have been widely used in industrial processes. However, facing a large amount of high-dimensional, nonlinear, and strongly coupled process data, traditional data-driven methods achieve low diagnostic accuracy due to ignoring the structural features inside data. To overcome this problem, this article proposes an improved locality preserving projections based on the heat-kernel and cosine weight matrix named Heat-Kernel and Cosine Weights Locality Preserving Projections (HC-LPP). In HC-LPP, a novel weight matrix construction strategy is employed, where a heat-kernel function is combined with a cosine function to optimize the weight matrix between data in terms of shortening distance and angle correlation, respectively. With the new weight matrix, the proposed HC-LPP considers both the distance and the correlation of samples (the shorter the distance is, the closer the neighbors are; the smaller the angle is, the more similar the neighbors are). The dimensionality reduction process of HC-LPP can well maintain the spatial geometric structure of data. Finally, the proposed HC-LPP integrated with the AdaBoost. M2 classifier is applied to the Tennessee Eastman process and the PROcess NeTwork Optimization process for fault diagnosis performance verification. Simulation results show, the proposed HC-LPP achieves better performance in diagnostic accuracy compared with other related methods.