Stochastic configuration network (SCN) has become a favorable soft sensor model due to its innovative random parameter construction approach under inequality constraint. However, it may suffer from the disadvantage of model overfitting with the increase of hidden nodes. To handle this issue, an improved Regularized SCN based on Spatio-Temporal nearest neighbors (ST-RSCN) is presented for better nonlinear soft sensor development. Different from regularized SCN only with L2 regularization, a dually-regularized SCN optimization framework is designed, where the L2 regularization term and manifold regularization (MR) term are applied to enforce the constraints from the perspectives of model parameters and data structure. Peculiarly, considering the data dynamic property, the traditional spatial nearest neighbor selection method is upgraded by integrating temporal searching strategy. Two spatio-temporal neighbor searching strategies are formed by designing different neighbor determination orders in spatial and temporal domains. The efficiency of the developed ST-RSCNs is finally demonstrated by two industrial cases, including a debutanizer column process and a continuous stirred tank reactor. The outcomes in both supervised and semi-supervised scenarios indicate that ST-RSCNs have better prediction performance compared with several models with respect to stability and generalization. Note to Practitioners—Data structure characteristic is of great importance for soft sensor model performance in practice. However, this is omitted in the existing SCN models. This paper designs a new enhanced regularized SCN for soft sensing. Specifically, a unified optimization objective is established involving both L2 regularization for constraining output weight magnitude and manifold regularization for mining the underlying geometric structure information. Further, two spatio-temporal nearest neighbor strategies are introduced to better fit the dynamic data structure. ST-RSCN related principles and proofs are described in detail. The simulation results of DCP and CSTR cases show the proposed ST-RSCNs significantly reduce the model prediction root mean squared errors (RMSEs) and are applicable for supervised and semi-supervised domains. Additionally, the key parameters’ selection methods are discussed, which contributes to a sound understanding of this paper.

Recent advancements in computational fluid dynamics(CFD) driven by the heterogeneous computing techniques and high-fidelity numerical methods have significantly increased the demand for efficient input/output(I/O) operations. In GPU-accelerated CFD, redundant data copies and excessive CPU overhead have become prominent challenges for efficient IO due to increasing complexities in data transfers between memory and storage. In this work, we propose a GPU native I/O framework(named as GPUDirectIO) for high-performance CFD by redesigning the Data Mapping Layer(DML) and data structures of the CFD General Notation System(CGNS), which is a widely used file format for complex CFD applications. This GPU-centric system enables direct data transfers between NVMe storage and GPU memory via GPU Direct Storage(GDS), effectively streamlining heterogeneous CFD I/O workflows. To further improve I/O throughput, we develop a CGNS-based distributed data management strategy that leverages an NVMe storage array to fully utilize the GPU bandwidth. We compare the performance of the proposed GPUDirectIO with existing CPU-mediated I/O approaches with different number of threads using CFD datasets where the maximum number of computa tional grid points reaches 1.6 billion. The results demonstrate the superiority of GPUDirectIO. Single-threaded GPUDirectIO achieves read and write rates 2.95× and 3.49× those of CPU mediated I/O, respectively, and reduces transmission latency by approximately 59%. When applied to distributed storage systems, multi-threaded GPUDirectIO shows read and write rates 3.23× and 4.68× those of CPU-mediated parallel I/O, respectively, with transfer latency reduced by about 39%. GPUDirectIO has also demonstrated excellent parallel efficiency and speedup ratios in both strong and weak scalability tests.

Causality in an industrial process provides insights into how various process variables interact and affect each other within the system. It reveals the underlying mechanisms of industrial processes, which ensures predictive reliability and facilitates physical interpretability. However, existing causality-based techniques have limitations, as they neglect the temporal factor in causal description, introduce spurious causal associations in causal discovery, and fail to consider the spatial-temporal synchronicity in causal utilization. To address these issues, this article proposes a spatial-temporal causality modeling approach. A novel spatial-temporal causal digraph (STCG) is proposed to describe causal dependencies among process variables, which considers both spatial and temporal factors encompassing causal relationships and time delays. The STCG identification procedure is formulated as a Markov decision process, and knowledge-data guided reinforcement learning is developed to acquire the optimal identification policy and avoid spurious causal associations. With the identified STCG, a graph attention gate recurrent unit (GAGRU) is constructed for spatial-temporal process modeling, which is able to capture the synchronous evolution of industrial data in spatial-temporal dimensions. Finally, the effectiveness of the proposed modeling approach is verified by applying to two real industrial cases, including soft sensing for a sulfur recovery unit and anomaly detection for an argon distillation system. The experimental results demonstrate that the STCG-based industrial process modeling outperforms classical and state-of-the-art comparison methods in terms of reliability and interpretability.

Sensor faults are non-negligible issues for soft sensor modeling. However, existing deep learning-based soft sensors are fragile and sensitive when considering sensor faults. To improve the robustness against sensor faults, this article proposes a deep subdomain learning adaptation network (DSLAN) to develop a sensor fault-tolerant soft sensor, which is capable of handling both sensor degradation and sensor failure simultaneously. Primarily, domain adaptation works for process data with sensor degradation in industrial processes. Being founded on the basic structure of deep domain adaptation, a novel subdomain learner is added to automatically learn the subdomain division, enabling DSLAN adaptable to multimode industrial processes. Notably, the subdomain structure of each sample follows a categorical distribution parameterized by output of the subdomain learner. Based on the designed subdomain learner, a new probabilistic local maximum mean discrepancy (PLMMD) is presented to measure the difference in distribution between source and target features. In addition, a generator for failure data imputation is integrated in the framework, making DSLAN handle sensor failure simultaneously. Finally, the Tennessee Eastman (TE) benchmark process and two real industrial processes are used to verify the effectiveness of the proposed method. With the fault tolerance ability, soft sensing technology will take a step toward practical applications.

Kernel partial least squares (KPLS) is a widely used soft sensor modeling method for nonlinear industrial processes. However, the traditional KPLS is considered as the shallow learning machine and may not capture the vital information hidden among data. In order to exploit the intrinsic data feature information, in this article, we propose a deep supervised learning framework based on KPLS, which is referred to as deep KPLS (DeKPLS). First, inspired by the deep learning mechanism, a hierarchical feature extraction framework based on KPLS is proposed, where the KPLS is served as the basic feature extraction module. Then, a layer-wise feedforward training strategy is designed for the determination of model architecture. Finally, two actual industrial processes are utilized to demonstrate the effectiveness of the proposed DeKPLS.

Convolutional neural network (CNN)-based recommender systems are playing an increasingly significant role in the vigorous development of Industrial Internet of Things, and have made great contributions to analyzing and mining a large amount of data to provide various services for terminal users. However, as the lack of explainability in deep learning, users often have low trust in the system due to their incomprehension of recommendation results. In addition, recommender systems have been facing a serious sparsity problem, and relying only on sparse rating data to learn user preferences and similarities may face malicious recommendation attacks. The abovementioned problems have been hindering the further improvement of recommendation performance. Therefore, in order to effectively alleviate the sparsity problem and meanwhile enhance the trustworthiness, an auxiliary review-based personalized attentional CNN (ARPCNN) is proposed in this article. By applying the proposed personalized word-level attention mechanism and personalized review-level attention mechanism in parallel CNNs, critical words and informative reviews are given high attention weights. Moreover, a user auxiliary network is proposed, which regards the reviews written by kindred spirits who have a trust relationship with the user as auxiliary reviews, and effectively extracts the user’s auxiliary review features, thereby achieving more accurate user modeling to improve the recommendation performance. Extensive experiments are conducted on four real-world datasets, and the results show that the performance of the proposed model is better than that of baselines, which verifies the effectiveness of ARPCNN.

Solving partial differential equations of complex physical systems is a computationally expensive task, especially in Computational Fluid Dynamics(CFD). This drives the application of deep learning methods in solving physical systems. There exist a few deep learning models that are very successful in predicting flow fields of complex physical models, yet most of these still exhibit large errors compared to simulation. Here we introduce AMGNET, a multi-scale graph neural network model based on Encoder-Process-Decoder structure for flow field prediction. Our model employs message passing of graph neural networks at different mesh graph scales. Our method has significantly lower prediction errors than the GCN baseline on several complex fluid prediction tasks, such as airfoil flow and cylinder flow. Our results show that multi-scale representation learning at the graph level is more effective in improving the prediction accuracy of flow field.

Sensor degradation seriously hinders the practical application of soft sensors. To reduce the negative effect of sensor degradation, in this article, we propose a robust domain adaptation mixture of Gaussian processes (DA-MGP) for online soft sensor modeling of multimode processes. Based on the decomposition of industrial data into a group of Gaussian domains, Gaussian domain discrepancy (GDD) is designed for domain adaptation and process mode recognition. After recognizing the process mode based on GDD, a Gaussian domain adaptation is presented to correct the drifted online input data by domain mapping, which can significantly improve the robustness of the soft sensor against sensor degradation. Furthermore, the domain mapping matrix is utilized as a transferred basis function for a local transferred Gaussian process component, which is used for robust soft sensor modeling. Additionally, an online block processing framework is adopted when the DA-MGP-based soft sensor is applied in online quality prediction. Finally, the TE benchmark process and a real industrial polypropylene process are employed to verify the effectiveness of the proposed method. In the designed five cases of sensor degradation, the DA-MGP-based soft sensor shows its strong robustness against sensor degradation.

Many industrial processes contain both linear and nonlinear parts, and kernel principal component analysis (KPCA), widely used in nonlinear process monitoring, may not offer the most effective means for dealing with these nonlinear processes. This paper proposes a new hybrid linear-nonlinear statistical modeling approach for nonlinear process monitoring by closely integrating linear principal component analysis (PCA) and nonlinear KPCA using a serial model structure, which we refer to as serial PCA (SPCA). Specifically, PCA is first applied to extract PCs as linear features, and to decompose the data into the PC subspace and residual subspace (RS). Then, KPCA is performed in the RS to extract the nonlinear PCs as nonlinear features. Two monitoring statistics are constructed for fault detection, based on both the linear and nonlinear features extracted by the proposed SPCA. To effectively perform fault identification after a fault is detected, an SPCA similarity factor method is built for fault recognition, which fuses both the linear and nonlinear features. Unlike PCA and KPCA, the proposed method takes into account both linear and nonlinear PCs simultaneously, and therefore, it can better exploit the underlying process's structure to enhance fault diagnosis performance. Two case studies involving a simulated nonlinear process and the benchmark Tennessee Eastman process demonstrate that the proposed SPCA approach is more effective than the existing state-of-the-art approach based on KPCA alone, in terms of nonlinear process fault detection and identification.

Principal component analysis (PCA) and kernel PCA (KPCA) are the state-of-art machine learning methods widely used in industrial process monitoring and fault detection field. However, these methods build shallow statistical models based on single layer of features and may not achieve the best monitoring performance. In order to sufficiently mine the intrinsic data features, a deep learning based nonlinear PCA method, referred to as deep PCA (DePCA), is proposed in this paper. Motivated by the idea of deep learning, a layer-wise statistical model structure is designed to extract multilayer data features, including both linear and nonlinear principal components. At each layer, two monitoring statistics are constructed to monitor the feature changes. For integrating the monitoring statistics of all feature layers, a Bayesian inference strategy is applied to convert the monitoring statistics into fault probabilities, which are weighted to form two probability-based comprehensive monitoring statistics for process fault detection. A case study using the benchmark Tennessee Eastman process demonstrates the superior performance of the proposed DePCA method over the traditional PCA and KPCA methods.

Summary The lattice Boltzmann method (LBM) is a widely used computational fluid dynamics method for flow problems with complex geometries and various boundary conditions. Large‐scale LBM simulations with increasing resolution and extending temporal range require massive high‐performance computing (HPC) resources, thus motivating us to port it onto modern many‐core heterogeneous supercomputers like Tianhe‐2. Although many‐core accelerators such as graphics processing unit and Intel MIC have a dramatic advantage of floating‐point performance and power efficiency over CPUs, they also pose a tough challenge to parallelize and optimize computational fluid dynamics codes on large‐scale heterogeneous system. In this paper, we parallelize and optimize the open source 3D multi‐phase LBM code openlbmflow on the Intel Xeon Phi (MIC) accelerated Tianhe‐2 supercomputer using a hybrid and heterogeneous MPI+OpenMP+Offload+single instruction, mulitple data (SIMD) programming model. With cache blocking and SIMD‐friendly data structure transformation, we dramatically improve the SIMD and cache efficiency for the single‐thread performance on both CPU and Phi, achieving a speedup of 7.9X and 8.8X, respectively, compared with the baseline code. To collaborate CPUs and Phi processors efficiently, we propose a load‐balance scheme to distribute workloads among intra‐node two CPUs and three Phi processors and use an asynchronous model to overlap the collaborative computation and communication as far as possible. The collaborative approach with two CPUs and three Phi processors improves the performance by around 3.2X compared with the CPU‐only approach. Scalability tests show that openlbmflow can achieve a parallel efficiency of about 60% on 2048 nodes, with about 400K cores in total. To the best of our knowledge, this is the largest scale CPU‐MIC collaborative LBM simulation for 3D multi‐phase flow problems. Copyright © 2015 John Wiley & Sons, Ltd.

HOSTA is an in-house high-order CFD software that can simulate complex flows with complex geometries. Large scale high-order CFD simulations using HOSTA require massive HPC resources, thus motivating us to port it onto modern GPU accelerated supercomputers like Tianhe-1A. To achieve a greater speedup and fully tap the potential of Tianhe-1A, we collaborate CPU and GPU for HOSTA instead of using a naive GPU-only approach. We present multiple novel techniques to balance the loads between the store-poor GPU and the store-rich CPU, and overlap the collaborative computation and communication as far as possible. Taking CPU and GPU load balance into account, we improve the maximum simulation problem size per Tianhe-1A node for HOSTA by 2.3X, meanwhile the collaborative approach can improve the performance by around 45% compared to the GPU-only approach. Scalability tests show that HOSTA can achieve a parallel efficiency of above 60% on 1024 Tianhe-1A nodes. With our method, we have successfully simulated China's large civil airplane configuration C919 containing 150M grid cells. To our best knowledge, this is the first paper that reports a CPUGPU collaborative high-order accurate aerodynamic simulation result with such a complex grid geometry.