Estimating individual treatment effects (ITE) from networked observational data is a fundamental but challenging task. Network connections not only introduce complex confounding bias but also generate spillover effects, making causal inference in such settings particularly difficult. To tackle these challenges, existing methods often attempt to learn confounders from all observed variables to predict potential outcomes. However, this approach overlooks heterogeneous latent factors that differentially influence treatment assignment and outcomes, which may prevent accurate identification of the confounders and reduce the effectiveness of bias correction. To overcome these limitations, we propose the Network Disentangled Identifiable Variational Autoencoder (NDiVAE), a structured framework that learns three distinct latent factors for each unit, including instrumental, confounding, and adjustment factors. NDiVAE further incorporates graph-based neighborhood information to learn aggregated factor representations, applies sample reweighting to mitigate confounding bias, and enforces causal regularization to ensure precise disentanglement of latent factors. We establish theoretical guarantees through the identifiability of these latent factors and derive a generalization error bound for ITE estimation. Extensive experiments on synthetic and semi-synthetic datasets demonstrate that NDiVAE consistently outperforms state-of-the-art methods in estimating treatment effects.

Exogenous variables are specially used in Structural Causal Models (SCM), which, however, have some characteristics that are still useful under the property of the Bayesian network. In this paper, we propose a novel causal discovery learning algorithm called Endogenous and Exogenous Markov Blankets Intersection (EEMBI), which combines the properties of Bayesian networks and SCM. Through intersecting the Markov blankets of exogenous variables and endogenous variables (the original variables), EEMBI can remove the irrelevant connections and find the true causal structure theoretically. Furthermore, we propose an extended version of EEMBI, named EEMBI-PC, which integrates the last step of the PC algorithm into EEMBI. This extension enhances the algorithm's performance by leveraging the strengths of both approaches. Plenty of experiments are provided to prove that EEMBI have state-of-the-art performance on continuous datasets, and EEMBI-PC outperforms other algorithms on discrete datasets.

In this paper, we develop the notion of the difference of evidence lower bounds (DELBO), based on which an efficient score algorithm is presented to implement feature selection on latent variables of VAE and its variants. Furthermore, we propose marginalization approximation algorithms to optimize VAE-related models by weighting the "more important" latent variables selected and accordingly increasing evidence lower bound. We discuss two kinds of different Gaussian posteriors, mean-field and full-covariance, for latent variables, and make the corresponding theoretical analyses to support the effectiveness of algorithms. Plenty of comparative experiments are carried out between our algorithms and the other 9 feature selection methods on 7 public datasets to address generative tasks. The results demonstrate the superior performance of our algorithms. Finally, we extend DELBO to its generalized version and apply the latter to tackling classification tasks of 5 new public datasets with satisfactory experimental results.

To address the challenges encountered by dictionary learning-based monitoring, this article presents a novel pattern discovery scheme for detection and isolation of incipient faults that involves structured sparse coding and sequential dictionary augmentations. Through learning a basic dictionary for normal pattern and augmenting the low-dimensional sparse dictionaries for analyzing different fault patterns, the process signals can be decomposed into fault-free and fault-related components. To guarantee the in-statistical-control status of the fault-free part and improve detection sensitivity, a$\ell _{2}$-penalty is imposed on the sum of coefficient vectors to ensure that the monitoring statistic related to the fault-free part will not exceed the control limit. In addition, two Frobenius norm penalties are imposed on the zero centered coefficient matrix and atom matrix to improve the robustness of signal decomposition. Instead of imposing$\ell _{1}$-sparsity constraint on the atoms, a hard sparsity constraint is used to correctly select fault-related feature variables, so that fault patterns can be better revealed. The informative dictionaries are then incorporated into the moving window-based monitoring strategy, yielding a fault detection and isolation scheme suitable for incipient faults. The superior performance of our proposed approach is validated by application studies involving a numerical example and two practical industrial processes.

Disentangled representation learning aims to learn low-dimensional representations where each dimension corresponds to an underlying generative factor. While the Variational Auto-Encoder (VAE) is widely used for this purpose, most existing methods assume independence among factors, a simplification that does not hold in many real-world scenarios where factors are often interdependent and exhibit causal relationships. To overcome this limitation, we propose the Disentangled Causal Variational Auto-Encoder (DCVAE), a novel supervised VAE framework that integrates causal flows into the representation learning process, enabling the learning of more meaningful and interpretable disentangled representations. We evaluate DCVAE on both synthetic and real-world datasets, demonstrating its superior ability in causal disentanglement and intervention experiments. Furthermore, DCVAE outperforms state-of-the-art methods in various downstream tasks, highlighting its potential for learning true causal structures among factors.

Click-through rate prediction aims to predict the ratio of clicks to impressions of a specific link, which is challenging due to (1) extremely high-dimensional categorical features; (2) both important original features and their interactions; and (3) reliance on different features and interactions in different time periods. To overcome these difficulties, we propose a new feature interaction detection method based on the idea of frequent itemset mining, named Online Random Intersection Chains (ORIC), which detects informative feature interactions with high interpretability. ORIC can be updated by controlling the importance of the historical and latest data with a tuning parameter, which saves computational burden and makes full use of historical information. Further, Streaming Integrated Model (SIM) is developed to feed the time-varying feature interactions into CTR prediction models. Empirical results on three benchmark datasets show that SIM achieves better performance than many CTR prediction models, as well as the efficiency, consistency, and interpretability of ORIC.

Predicting the probability that a user clicks a specific item is fundamental in online advertising and recommendation. Further, it is crucial to use the latest and historical data appropriately in online scenarios to train CTR models. Online Random Interaction Chains (ORIC) was proposed to detect informative and interpretable feature interactions without retraining on historical data in online scenario, and the Streaming Integrated Model (SIM) framework was designed to integrate these time-varying feature interactions into CTR prediction models. Unfortunately, ORIC exhibits latency when provides the feature interactions used to evaluate SIM, and ORIC is not applicable for numerical features. For these reasons, we propose ORIC-V2 that uses time series models to predict the confidence of candidate evaluating feature interactions and selects reasonable feature interactions, and combines numerical features with ORIC-V2 through a discretization model to obtain DORIC-V2. Feeding the feature interactions found by ORIC-V2 and DORIC-V2 into SIM obtains significant experimental results on three datasets, demonstrating the effectiveness and interpretability of ORIC-V2 and DORIC-V2.

Modern industrial data is commonly dynamic and contains outliers, which challenges the accurate isolation of faulty variables in abnormal situations. To deal with process dynamics, a robust stacked probabilistic latent variable model is proposed, which is formed by stacking a series of static probabilistic latent variable models. A fault indicator matrix with the Bernoulli-Gaussian prior is constructed to indicate which process variables are faulty. The Bernoulli-Gaussian prior neatly accommodates the stacked structure of the indicator matrix so that rows corresponding to normal variables will shrink to zero. The stacked probabilistic latent variable model is further extended to deal with outliers by introducing an outlier indicator vector with the Beta-Bernoulli prior. The location and magnitude of the outliers can be successfully identified. Based on the robust stacked model, a variational Bayesian inference algorithm is developed to estimate unknown parameters. By using the piecewise affine approximation, the proposed fault isolation method can be extended to deal with nonlinear processes. The effectiveness and superiority of the method are illustrated by application studies to a simulation case and an industrial boiler case.Note to Practitioners—Data-driven fault isolation methods are critical to helping find the accurate root causes of industrial faults. While designing the fault isolation procedures, traditional data-based methods seldom consider autocorrelation and outliers characteristics in the collected process data simultaneously. This paper proposes an accurate and robust fault isolation method based on the stacked probabilistic latent variable model. For the full implementation of the model, it is necessary to: 1) construct the stacked structure of the fault indicator matrix based on the Bernoulli-Gaussian prior; 2) establish the outlier indicator vector with the Beta-Bernoulli prior to determine the location and magnitude of the outliers; 3) optimize the robust stacked model through the variational Bayesian inference algorithm with appropriately selected priors; 4) extract the faulty information of the industrial data to further enhance the faulty isolation performance. The two case studies have shown satisfactory fault-locating accuracy with the proposed model.

This article investigates the efficacy of a regularized multitask learning (MTL) framework based on SVM (M-SVM) to answer whether MTL always provides reliable results and how MTL outperforms independent learning. We first find that the M-SVM is Bayes risk consistent in the limit of a large sample size. This implies that despite the task dissimilarities, the M-SVM always produces a reliable decision rule for each task in terms of the misclassification error when the data size is large enough. Furthermore, we find that the task-interaction vanishes as the data size goes to infinity, and the convergence rates of the M-SVM and its single-task counterpart have the same upper bound. The former suggests that the M-SVM cannot improve the limit classifier's performance; based on the latter, we conjecture that the optimal convergence rate is not improved when the task number is fixed. As a novel insight into MTL, our theoretical and experimental results achieved an excellent agreement that the benefit of the MTL methods lies in the improvement of the preconvergence-rate (PCR) factor (to be denoted in Section III) rather than the convergence rate. Moreover, this improvement of PCR factors is more significant when the data size is small. In addition, our experimental results of five other MTL methods demonstrate the generality of this new insight.

The lack of interpretability often makes black-box models challenging to be applied in many practical domains. For this reason, the current work, from the black-box model input port, proposes to incorporate data-mined knowledge into the black-box soft-margin SVM model to enhance accuracy and interpretability. The concept and incorporation mechanism of data-mined knowledge are successively developed, based on which a partially interpretable soft-margin SVM ( pTsm -SVM) optimization model is designed and then solved through reformulating the optimization problem as standard quadratic programming. An algorithm for mining linear positive (negative) class knowledge from general data sets is also proposed, which generates a linear two-dimensional discriminative rule with specificity (sensitivity) equal to 1 and the highest possible sensitivity (specificity) among all two-dimensional feature spaces. The knowledge-integrated pTsm -SVM works by achieving a good trade-off among the “large margin”, “high specificity”, and “high sensitivity”. Our experimental results on eight UCI datasets demonstrate the superiority of the proposed pTsm -SVM over the standard soft-margin SVM both in terms of accuracy and interpretability.

With the growing size of datasets, feature selection becomes increasingly important. Taking interactions of original features into consideration will lead to extremely high dimension, especially when the features are discrete and one-hot encoding is applied. This makes it more worthwhile mining useful features as well as their interactions. Association rule mining aims to extract interesting correlations between items, but it is difficult to use rules as a qualified classifier themselves. Drawing inspiration from association rule mining, we come up with a method that uses association rules to select features and their interactions, then modify the algorithm for several practical concerns. We analyze the computation complexity of the proposed algorithm to show its efficiency. And the results of a series of experiments verify the effectiveness of the algorithm.

This paper investigates the effectiveness of transfer learning based on information criteria. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than the Cp estimator by the mean squared error (MSE) metric {in the case of orthogonal predictors}, provided that i) the dissimilarity between the tasks from source domain and target domain is small, and ii) the procedure parameters (complexity penalties) are tuned according to certain explicit rules. Moreover, we show that our transfer learning framework can be extended to other feature selection criteria, such as the Bayesian information criterion. By analyzing the solution of the orthogonalized Cp, we identify an estimator that asymptotically approximates the solution of the Cp criterion in the case of non-orthogonal predictors. Similar results are obtained for the non-orthogonal TLCp. Finally, simulation studies and applications with real data demonstrate the usefulness of the TLCp scheme.

In this article, a data-based compact Volterra model is constructed to carry out the silicon prediction task. The main motivations are to, on the one hand, make better use of the strong memory ability of the Volterra series, which is suitable for reflecting the large inertia of the blast furnace process, on the other hand, overcome the difficulty that high complexity of the original Volterra models may result in serious overfitting. Six kinds of models, including single-input and two-input linear, second-order and third-order compact Volterra models, are successively designed and verified through a real blast furnace case. The reasonable agreement between the predicted values and the observed values indicates the compact Volterra model, especially the single-input third-order Volterra model, are powerful and competitive for describing the complex blast furnace system. The experimental results can serve as a guide for the blast furnace operators to judge the in-furnace thermal state change in time and further provide an indication on how to control the blast furnace in advance.

Black-box models are a kind of effective means to describe extremely complex systems, such as blast furnaces. However, an evident deficiency for them lies in the lack of comprehensibility and transparency. For this reason, the current work, starting with the black-box model input port, contributes to enhancing the transparency of the blast furnace soft-margin support vector machine (SVM) model. In this article, we first develop a novel algorithm to mine linear prior knowledge from data sets. Then, the mined priors are integrated into the black-box soft-margin SVM model in the form of inequality constraints to create a partly transparent soft-margin SVM (pTsm-SVM) optimization model. The pTsm-SVM model has the advantages of both black-box models and white-box models, i.e., high precision and some transparency. Finally, we exhibit the effectiveness of the pTsm-SVM model through two real blast furnace examples.

Black-box models play an important role in advancing the blast furnace modeling technologies for control purposes. To further enhance their practical applications, this paper is concerned with the transparency and comprehensibility of blast furnace black-box models. A fuzzy-based support vector machine (SVM) classification algorithm is proposed to perform the tasks of determining the controllable bound from the real data, of reducing feature from extensive candidate inputs, and of training the SVM model parameters. Based on these results, a fuzzy-based blast furnace SVM three-class classifier is constructed to serve for the classification problem according to the output lower than its controlled bound, within the controlled bound and higher than the controlled bound. Further, rule extraction is made to achieve the understandability of the constructed SVM classifier. Through two typical real blast furnace cases, the extracted rules can work well in classifying the hot metal silicon content into low, proper, and high range with high transparency, as well as encouraging agreements between the predicted values and the real ones. Moreover, there needs to be very little information on the blast furnace variables when implementing every rule in practice. The extracted rules provide a more explicit and direct indication for the blast furnace operators and, thus, may serve better for decision-making with blast furnace control.

It is our pleasure to present this Special Issue on "Data-Driven Approaches for Complex Industrial Systems" of the IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, which provides a forum for researchers and practitioners to report recent results on data-driven methods with applications to complex industrial systems, and to identify critical issues and challenges for future investigations in this field. Roughly, data-driven methods can be categorized into three sets, i.e., data-driven modeling, data-driven monitoring and fault diagnosis, and data-driven control and optimization (cf. Fig. 1). In this Special Issue, 13 papers are selected with novel contributions in data-driven modeling, data-driven monitoring and diagnosis, data-driven control and their industrial applications, respectively.

A review of black-box models for short-term time-discrete prediction of the silicon content of hot metal produced in blast furnaces is presented. The review is primarily focused on work presented in journal papers, but still includes some early conference papers (published before 1990) which have a clear contribution to the field. Linear and nonlinear models are treated separately, and within each group a rough subdivision according to the model type is made. Within each subsection the models are treated (almost) chronologically, presenting the principle behind the modeling approach, the signals used and the main findings in terms of accuracy and usefulness. Finally, in the final section the approaches are discussed and some potential lines of future research are proposed. In an Appendix , a list of commonly used input and output variables in the models is presented.

This paper constructs the framework of the reproducing kernel Hilbert space for multiple kernel learning, which provides clear insights into the reason that multiple kernel support vector machines (SVM) outperform single kernel SVM. These results can serve as a fundamental guide to account for the superiority of multiple kernel to single kernel learning. Subsequently, the constructed multiple kernel learning algorithms are applied to model a nonlinear blast furnace system only based on its input-output signals. The experimental results not only confirm the superiority of multiple kernel learning algorithms, but also indicate that multiple kernel SVM is a kind of highly competitive data-driven modeling method for the blast furnace system and can provide reliable indication for blast furnace operators to take control actions.

The multidimensional blast furnace system is one of the most complex industrial systems and, as such, there are still many unsolved theoretical and experimental difficulties, such as silicon prediction and blast furnace automation. For this reason, this paper is concerned with developing data-driven models based on the Volterra series for this complex system. Three kinds of different low-order Volterra filters are designed to predict the hot metal silicon content collected from a pint-sized blast furnace, in which a sliding window technique is used to update the filter kernels timely. The predictive results indicate that the linear Volterra predictor can describe the evolvement of the studied silicon sequence effectively with the high percentage of hitting the target, very low root mean square error and satisfactory confidence level about the reliability of the future prediction. These advantages and the low computational complexity reveal that the sliding-window linear Volterra filter is full of potential for multidimensional blast furnace system. Also, the lack of the constructed Volterra models is analyzed and the possible direction of future investigation is pointed out.