• Introduces DepAD for using variable dependencies for detecting anomalies. • Reframes unsupervised detection as supervised feature selection and prediction. • Evaluates DepAD against nine state-of-the-art methods on 32 datasets. • DepAD outperforms in detecting anomalies with better interpretability. • Demonstrates new insights and interpretations for detected anomalies. Anomaly detection is crucial for identifying unusual behaviors in data, which often provide valuable insights. This paper introduces Dependency-based Anomaly Detection (DepAD), a general and modular framework that leverages variable dependencies to uncover meaningful anomalies with improved interpretability. DepAD reframes unsupervised anomaly detection as a sequence of supervised feature selection and prediction tasks, enabling users to tailor detection methods to their data and domain. We systematically evaluate 125 DepAD algorithm variants across 32 real-world datasets, combining different off-the-shelf feature selection and prediction techniques. We compare DepAD with twelve state-of-the-art anomaly detection methods and demonstrate its consistent superior performance. Furthermore, we demonstrate that DepAD provides intuitive and informative interpretations of detected anomalies, highlighting its utility in practical applications.

Large-scale multi-agent systems face two core challenges: inefficient policy learning and the explosion of state dimensions. Existing methods often rely on manually designed task sequences to guide agents’ learning in stages, but these designs lack adaptability to agents’ learning abilities, making it difficult to ensure the rationality of task difficulty. Moreover, the representation capability of current network structures is limited, making it challenging to efficiently handle high-dimensional state information and complex interaction relationships. To address these issues, we propose a Progressive Multi-Agent Reinforcement Learning (PMARL) framework. PMARL introduces a task adapter that adaptively selects task difficulty based on agents’ learning abilities, eliminating reliance on manual experience. Additionally, a Dynamic Dimension Adaptive Network (DDAN) is designed, incorporating hypernetwork and self-attention mechanisms to achieve adaptive feature extraction of high-dimensional states and efficient representation of agent interaction relationships. Experimental results demonstrate that PMARL exhibits higher efficiency and better adaptability compared to existing methods when addressing large-scale multi-agent tasks.

Single domain generalization aims to develop a robust model trained on a source domain to generalize well on unseen target domains. Recent progress in single domain generalization has focused on expanding the scope of training data through style (e.g., backgrounds) augmentation. However, existing methods are difficult to generate data with large style shifts due to the lack of precise correlation measures between the generated and original data, and they struggle to effectively capture the consistency between the generated and original data when learning feature representations. In this article, we propose a novel Mutual Information-guided Style Augmentation (MISA) based single domain generalization method. Specifically, MISA incorporates a style diversity module, which uses the matrix-based Rényi’s \(\alpha\) -order entropy functionals to compute an approximate mutual information value between the augmented and original data, minimizing it to guide style generator learning. Moreover, MISA combines the merits of the random convolution and affine transformation to further improve the texture diversity of the augmented data. Additionally, MISA introduces a representation learning module, which minimizes the approximate mutual information value between the prediction logits of the original sample and its corresponding residual component to capture the consistency between the generated and original data for feature representation optimization. Using five real-world datasets, the extensive experiments have demonstrated the effectiveness of MISA, in comparison with state-of-the-art methods.

Cognitive diagnosis (CD) is an important task in the field of intelligent education, aiming to discover the proficiency of students on knowledge concepts with response logs. In applications, different users of the tutoring system demand for a diagnosis of knowledge concepts at different granularities. However, recent methods assume that the concepts are of the same granularity and use explicit correlations between same-granularity concepts to improve the diagnosis performance. If required for diagnosing multi-granularity concepts, these methods will face diminished performance or partial invalidation. To this end, we make the first attempt for multiple-granularity cognitive diagnosis, i.e., diagnosis on coarse- and fine-grained concepts simultaneously. Specifically, in a skillful way, the same-granularity correlations are captured and embedded into concept representations in view of concept semantics and cross-granularity correlations to model the proficiency influence between concepts implicitly. Then, the specific loss for single-granularity diagnosis and the general loss for the consistency of multi-granularity are designed to train the model jointly, achieving multiple-granularity diagnosis. Extensive experiments demonstrate that our method can achieve state-of-the-art accuracy on both coarse- and fine-grained concepts.

Federated feature selection (FFS) is a promising field for selecting informative features while preserving data privacy in federated learning (FL) settings. Existing FFS methods focus on capturing the correlations between features and labels. They struggle to achieve satisfactory performance in the face of data distribution heterogeneity among FL clients, and cannot address the out-of-distribution (OOD) problem that arises when a significant portion of clients do not actively participate in FL training. To address these limitations, we propose Federated Causally Invariant Feature Learning (FedCIFL), a novel approach for learning causally invariant features in a privacy-preserving manner. We design a sample reweighting strategy to eliminate spurious correlations introduced by selection bias and iteratively estimate the federated causal effect between each feature and the labels (with the remaining features initially treated as confounders). By iteratively refining the confounding feature set to identify the true confounders, FedCIFL mitigates the impact of limited local data on the accuracy of federated causal effect estimation. Theoretical analysis proves the correctness of FedCIFL under reasonable assumptions. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of FedCIFL against eight state-of-the-art baselines, beating the best-performing approach by 3.19%, 9.07% and 2.65% in terms of average test Accuracy, RMSE and F1 score, respectively. It is a first-of-its-kind FFS approach capable of handling Non-IID and OOD data simultaneously. The source code is available at https://github.com/Xianjie-Guo/FedCIFL.

Local causal discovery is crucial for revealing the causal relationships between specific variables from data. Existing local causal discovery algorithms are designed under the assumption of causal sufficiency, which states that there are no latent common causes for two or more of the observed variables in data. However, the assumption of causal sufficiency is often violated in practice. To address this issue, we first propose the local Maximal Ancestral Graph (MAG), referred to as LocalMAG, to describe the local causal relationships of the target variable in the MAG. Then, we propose a local causal discovery algorithm without the assumption of causal sufficiency, called LatentLCD, to learn the LocalMAG. Specifically, LatentLCD first uses the traditional parents and children discovery algorithm to identify the local causal skeleton that includes latent variables and verifies it theoretically. It then identifies bidirectional edges by determining whether both the target variable and its adjacent variables are colliders, thereby identifying latent variables in the local structure of the target variable. Extensive experiments on synthetic datasets have validated that the proposed LatentLCD algorithm significantly outperforms the state-of-the-art methods.

The explosion of data and the widespread adoption of AI techniques, especially the success of foundation models and generative AI, have transformed knowledge discovery and data mining (KDD), making them integral to real-world decision-making. For both traditional AI methods and generative AI, issues such as data noise, algorithmic bias, lack of interpretability, and privacy concerns can significantly impact the quality and reliability of extracted knowledge, thereby affecting downstream decision-making. This workshop aims to bring together researchers and practitioners from information and knowledge management, data mining, and intelligent systems to explore trustworthy KDD across diverse settings in the generative AI era. We welcome contributions on robust data preprocessing, explainable learning algorithms, bias detection and mitigation, secure and privacy-preserving mining, trustworthy knowledge graph construction, resource-efficient deployment, alignment of foundation models, and applications for social good. Special emphasis is placed on emerging challenges posed by large-scale, pre-trained models in dynamic, multi-source, and user-centric environments. By fostering dialogue between traditional KDD approaches and innovations in the foundation model era, TrustKDD seeks to advance trustworthy methodologies that align with CIKM's mission of developing reliable, scalable, and intelligent information and knowledge systems.

Entity alignment (EA) aims to find the same entities in different knowledge graphs. Existing EA methods assume the supervised aligned pairs without noise. In applications, noisy pairs lead to degradation of EA performance. To this end, a robust EA method based on knowledge distillation is proposed for noisy pairs. Firstly, the dual-teacher model with online distillation is designed, in which, noise discriminator is performed to improve the noise resistance of teacher models. Secondly, a student model is offline distilled from the dual-teacher model without using the noisy supervised pairs, further enhancing the robustness of student model. In addition, the entity structure is combined with entity representation for alignment inference to alleviate the bias of entity representation in noisy environment. Extensive experiments demonstrate the effectiveness of the proposed method.

Federated causal structure learning aims to infer causal relationships from data stored on individual clients, with privacy concerns. Most existing methods assume identical variable sets across clients and present federated strategies for aggregating local updates. However, in practice, clients often observe overlapping but non-identical variable sets, and non-overlapping variables may introduce spurious dependencies. Moreover, existing strategies typically reflect only the overall quality of local graphs, ignoring the varying importance of relationships within each graph. In this paper, we study federated causal structure learning with non-identical variable sets, aiming to design an effective strategy for aggregating “correct” and “good” (non-)causal relationships across distributed datasets. Specifically, we first develop theories for detecting spurious dependencies, examining whether the learned relationships are “correct” or not. Furthermore, we define stable relationships as those that are both “correct” and “good” across multiple graphs, and finally design a two-level priority selection strategy for aggregating local updates, obtaining a global causal graph over the integrated variables. Experimental results on synthetic, benchmark and real-world data demonstrate the effectiveness of our method.

Local causal discovery aims to identify and distinguish the direct causes and effects of a target variable from observational data. Due to the inherent incompleteness of local information, popular methods from global causal discovery often face new challenges in local causal discovery tasks, such as 1) erroneous symmetry constraint tests and the resulting cascading errors in constraint-based methods, and 2) confusion within score-based approaches caused by local spurious equivalence classes. To address the above issues, we propose a Hybrid Local Causal Discovery algorithm, called HLCD. Specifically, HLCD initially utilizes a constraint-based approach with the OR rule to obtain a candidate skeleton, which is subsequently refined using a score-based method to eliminate redundant structures. Furthermore, during the local causal orientation phase, HLCD distinguishes between V-structures and equivalence classes by comparing local structure scores between the two, thereby avoiding orientation interference caused by local equivalence class ambiguities. Comprehensive experiments on 14 benchmark Bayesian networks and two real datasets validate that the proposed algorithm outperforms the existing local causal discovery methods.

The widespread application of pre-trained language models (PLMs) in natural language processing (NLP) has led to increasing concerns about their explainability. Selective rationalization is a self-explanatory framework that selects human-intelligible input subsets as rationales for predictions. Recent studies have shown that applying existing rationalization frameworks to PLMs will result in severe degeneration and failure problems, producing sub-optimal or meaningless rationales. Such failures severely damage trust in rationalization methods and constrain the application of rationalization techniques on PLMs. In this paper, we find that the homogeneity of tokens in the sentences produced by PLMs is the primary contributor to these problems. To address these challenges, we propose a method named Pre-trained Language Model's Rationalization (PLMR), which splits PLMs into a generator and a predictor to deal with NLP tasks while providing interpretable rationales. The generator in PLMR also alleviates homogeneity by pruning irrelevant tokens, while the predictor uses full-text information to standardize predictions. Experiments conducted on two widely used datasets across multiple PLMs demonstrate the effectiveness of the proposed method PLMR in addressing the challenge of applying selective rationalization to PLMs. Codes: https://github.com/ylb777/PLMR.

Many confounding-resistant domain generalization methods for image classification have been developed based on causal interventions. However, their reliance on strong assumptions limits their effectiveness in handling unobserved confounders. Although recent work introduces instrumental variables (IVs) to overcome this limitation, the reliance on manually predefined instruments, particularly in the context of visual data, may result in severe bias or invalidity when IV conditions are violated. To address these issues, we propose a novel approach to automatically learning Visual Instrumental Variables for confounding-resistant Domain Generalization (VIV-DG). We observe that certain non-causal visual attributes in image data naturally satisfy the basic conditions required for valid IVs. Motivated by this insight, we propose the visual instrumental variable, a novel concept that extends classical IV theory to the visual domain. Furthermore, we develop an automatic visual instrumental variable learner that enforces IV conditions on learned representations, enabling the automatic learning of valid visual instrumental variables from image data. Ultimately, VIV-DG inherits the strengths of classical IVs to mitigate unobserved confounding and avoids the significant bias caused by violations of IV conditions in predefined IVs. Extensive experiments on multiple benchmarks verify that VIV-DG achieves superior generalization ability.

The dung beetle optimizer (DBO) is a metaheuristic algorithm with fast convergence and powerful search capabilities, which has shown excellent performance in solving various optimization problems. However, it suffers from the problems of easily falling into local optimal solutions and poor convergence accuracy when dealing with large-scale complex optimization problems. Therefore, we propose an adaptive DBO (ADBO) based on an elastic annealing mechanism to address these issues. First, the convergence factor is adjusted in a nonlinear decreasing manner to balance the requirements of global exploration and local exploitation, thus improving the convergence speed and search quality. Second, a greedy difference optimization strategy is introduced to increase population diversity, improve the global search capability, and avoid premature convergence. Finally, the elastic annealing mechanism is used to perturb the randomly selected individuals, helping the algorithm escape local optima and thereby improve solution quality and algorithm stability. The experimental results on the CEC 2017 and CEC 2022 benchmark function sets and MCNC benchmark circuits verify the effectiveness, superiority, and universality of ADBO.

Cognitive diagnosis (CD) aims to evaluate students’ cognitive states based on their interaction data, enabling downstream applications such as exercise recommendation and personalized learning guidance. However, existing methods often struggle with accuracy drops in cross-domain cognitive diagnosis (CDCD), a practical yet challenging task. While some efforts have explored exercise-aspect CDCD, such as cross-subject scenarios, they fail to address the broader dual-aspect nature of CDCD, encompassing both student- and exercise-aspect variations. This diversity creates significant challenges in developing a scenario-agnostic framework. To address these gaps, we propose PromptCD, a simple yet effective framework that leverages soft prompt transfer for cognitive diagnosis. PromptCD is designed to adapt seamlessly across diverse CDCD scenarios, introducing PromptCD-S for student-aspect CDCD and PromptCD-E for exercise-aspect CDCD. Extensive experiments on real-world datasets demonstrate the robustness and effectiveness of PromptCD, consistently achieving superior performance across various CDCD scenarios. Our work offers a unified and generalizable approach to CDCD, advancing both theoretical and practical understanding in this critical domain. The implementation of our framework is publicly available athttps://github.com/Publisher-PromptCD/PromptCD.

Cognitive diagnosis (CD) aims to discover students’ mastery of knowledge concepts through response logs and it is an important task in intelligence education. CD is generally performed with the advantage of students doing exercises, which is learned by linear interacting between student proficiency and exercise difficulty. However, existing methods represent the proficiency and difficulty in view of knowledge concepts independently, which is in coarse-granularity, leading to the indiscrimination for different combinations of students and exercises under the same concept. To this end, we propose a cognitive diagnosis method with nonlinear dependence between students and exercises (CDND), in which, more fine-grained information is captured to distinguish the different combinations of students and exercises. First, the nonlinear dependence is captured with a novel attention-like mechanism by interacting between students and exercises. Moreover, this nonlinear dependence is embedded as the interactive discrimination vector. Second, the interactive discrimination is used to adjust the advantage using a second-order interaction, which can strengthen the discrimination of the advantage under different combinations of students and exercises. Extensive experimental results on three real data sets validate our interactive discrimination has good compatibility with existing methods, and with it, our CDND achieves an obvious improvement in performance. Our code is available onhttps://github.com/joyce99/LinZhihao/tree/main/CDND-master.

Clustering is a fundamental technique widely used for exploring the inherent data structure. Many studies indicate that an appropriate feature representation can effectively improve clustering performance. However, the existing feature representation methods are based on correlation to select or extract features, which makes it hard to deal with spurious correlations. The spurious correlations mislead the correlation-based methods to consider features that have no causal relationship as being correlative, which limits the clustering performance and feature interpretability. To tackle this issue, inspired by causal learning, we propose a new joint optimization Clustering Method with Causal Feature Embedding (CM-CaFE) , which utilizes the causality of features to learn more discriminative representation for clustering. Specifically, to eliminate spurious correlations among features, we first employ any state-of-the-art Markov blanket learning method to learn an undirected causal graph. Next, we extract the maximal fully connected causal subgraphs from the learned undirected causal graph and propose an approach to merge them to generate the causal matrix. Based on the causal matrix, we present an objective function that consists of a clustering loss term and a causal matrix fitting term to learn a causal transformation matrix. The causal transformation matrix is utilized to map the original data into a new space for clustering. Finally, we comprehensively compare the proposed method with some state-of-the-art clustering approaches on several datasets to demonstrate the effectiveness and interpretability of the proposed method.

Conditional Independence (CI) tests play an essential role in causal discovery from observational data, enabling the measurement of independence between two nodes. However, traditional CI tests ignore the imbalanced occurrence probabilities of node values, which may affect the accuracy of determining independence between nodes. To address this problem, we first introduce a new concept of the Node-imbalance phenomenon to describe the imbalance of node values in the Bayesian network data and analyze the influence of the Node-imbalance phenomenon on the traditional CI tests, then we propose a Weight-Based Conditional Independence (WCI) test to improve the accuracy of CI tests in the presence of Node-imbalance. In the experiments, we verify that WCI effectively measures the dependency between nodes in the Node-imbalance phenomenon compared with the traditional independence tests, and the state-of-the-art causal discovery algorithms reduce the number of false causal orientations through WCI.

Feature weighting aims to assign different weights to features based on their importance in machine learning tasks. In clustering tasks, the existing methods learn feature importance based on the clustering results derived from the collaborative contribution of all features, which overlooks the independent effect of each feature. In fact, there are underlying causal relationships between features and the clustering results, and the features with high causal effects are always more crucial for clustering. Therefore, we propose an enhancedFeatureWeighting method viaCausalEffect forClustering, calculating the causal effect of each feature on the clustering results for obtaining the independent contribution of each feature. Specifically, we start by identifying the causal relationships among the features and utilizing the causal relationships to generate a reasonable treatment group. Next, we compare the changes in the data distribution between the treatment and control groups to determine the causal effect of each feature. Finally, the causal effects of features are used for enhancing the clustering-driven weight learning. Moreover, we present a theory of relative order consistency in causal effect. Experimental results demonstrate that utilizing causal effect in weight learning facilitates efficient convergence and achieves superior accuracy compared to state-of-the-art clustering algorithms.

Causal feature selection has recently received increasing attention in machine learning and data mining, especially in the era of Big Data. Existing causal feature selection algorithms select unique causal features of the single class label as the optimal feature subset. However, a single class label usually has multiple classes, and it is unreasonable to select the same causal features for different classes of a single class label. To address this problem, we employ the class-specific mutual information to evaluate the causal information carried by each class of the single class label, and theoretically analyze the unique relationship between each class and the causal features. Based on this, aLabel-awareCausalFeatureSelection algorithm (LaCFS) is proposed to identifies the causal features for each class of the class label. Specifically, LaCFS uses the pairwise comparisons of class-specific mutual information and the size of class-specific mutual information values from the perspective of each class, and follows a divide-and-conquer framework to find causal features. The correctness and application condition of LaCFS are theoretically proved, and extensive experiments are conducted to demonstrate the efficiency and superiority of LaCFS compared to the state-of-the-art approaches.

As a promising strategy to achieve generalizable graph learning tasks, graph invariant learning emphasizes identifying invariant subgraphs for stable predictions on biased unknown distribution by selecting the important edges/nodes based on their contributions to the predictive tasks (i.e., subgraph predictivity). However, the existing approaches solely relying on subgraph predictivity face a challenge: the learned invariant subgraph often contains numerous spurious nodes and shows poor connectivity, undermining the generalization power of Graph Neural Networks (GNNs). To tackle this issue, we propose a summary graph-induced Invariant Learning (SIL) model that innovatively adopts a summary graph to leverage both the subgraph connectivity and predictivity for learning strong connected and accurate invariant subgraphs. Specifically, SIL first learns a summary graph containing multiple strongly connected supernodes while maintaining structure consistency with the original graph. Second, the learned summary graph is disentangled into an invariant supernode and spurious counterparts to eliminate the interference of highly predictive edges and nodes. Finally, SIL identifies a potential invariant subgraph from the invariant supernode to accomplish generalization tasks. Additionally, we provide a theoretical analysis of the summary graph learning mechanism, guaranteeing that the learned summary graph is consistent with the original graph. Experimental results validate the effectiveness of the SIL model.

As an emerging research direction, federated causal structure learning (CSL) aims at learning causal relationships from decentralized data across multiple clients while preserving data privacy. Existing federated CSL algorithms suffer from scalability and accuracy issues, since they require computationally expensive CSL algorithms to be executed at each client. Furthermore, in real-world scenarios, the number of samples held by each client varies significantly, and existing methods still assign equal weights to the learned structural information from each client, which severely harms the learning accuracy of those methods. To address these two limitations, we propose FedCSL, a scalable and accurate method for federated CSL. Specifically, FedCSL consists of two novel strategies: (1) a federated local-to-global learning strategy that enables FedCSL to scale to high-dimensional data for tackling the scalability issue, and (2) a novel weighted aggregation strategy that does not rely on any complex encryption techniques while preserving data privacy for tackling the accuracy issue. Extensive experiments on benchmark datasets, high-dimensional synthetic datasets and a real-world dataset verify the efficacy of the proposed FedCSL method. The source code is available at https://github.com/Xianjie-Guo/FedCSL.

Selective rationalization can be regarded as a straightforward self-explaining approach for enhancing model explainability in natural language processing tasks. It aims to provide explanations that are more accessible and understandable to non-technical users by first selecting subsets of input texts as rationales and then predicting based on chosen subsets. However, existing methods that follow this select-then-predict framework may suffer from the rationalization degeneration problem, resulting in sub-optimal or unsatisfactory rationales that do not align with human judgments. This problem may further lead to rationalization failure, resulting in meaningless rationales that ultimately undermine people's trust in the rationalization model. To address these challenges, we propose a Guidance-based Rationalization method (G-RAT) that effectively improves robustness against failure situations and the quality of rationales by using a guidance module to regularize selections and distributions. Experimental results on two synthetic settings prove that our method is robust to the rationalization degeneration and failure problems, while the results on two real datasets show its effectiveness in providing rationales in line with human judgments. The source code is available at https://github.com/shuaibo919/g-rat.

An essential and challenging problem in causal inference is causal effect estimation from observational data. The problem becomes more difficult with the presence of unobserved confounding variables. The front-door adjustment is an approach for dealing with unobserved confounding variables. However, the restriction for the standard front-door adjustment is difficult to satisfy in practice. In this paper, we relax some of the restrictions by proposing the concept of conditional front-door (CFD) adjustment and develop the theorem that guarantees the causal effect identifiability of CFD adjustment. By leveraging the ability of deep generative models, we propose CFDiVAE to learn the representation of the CFD adjustment variable directly from data with the identifiable Variational AutoEncoder and formally prove the model identifiability. Extensive experiments on synthetic datasets validate the effectiveness of CFDiVAE and its superiority over existing methods. The experiments also show that the performance of CFDiVAE is less sensitive to the causal strength of unobserved confounding variables. We further apply CFDiVAE to a real-world dataset to demonstrate its potential application.

Causal chain graphs model the dependency structure between individuals when the assumption of individual independence in causal inference is violated. However, causal chain graphs are often unknown in practice and require learning from data. Existing learning algorithms have certain limitations. Specifically, learning local information requires multiple subset searches, building the skeleton requires additional conditional independence testing, and directing the edges requires obtaining local information from the skeleton again. To remedy these problems, we propose a novel algorithm for learning causal chain graph structure. The algorithm alternately learns the adjacencies and spouses of each variable as local information and doubly prunes them to obtain more accurate local information, which reduces subset searches, improves its accuracy, and facilitates subsequent learning. It then directly constructs the chain graphs skeleton using the learned adjacencies without conditional independence testing. Finally, it directs the edges of complexes using the learned adjacencies and spouses to learn chain graphs without reacquiring local information, further improving its efficiency. We conduct theoretical analysis to prove the correctness of our algorithm and compare it with the state-of-the-art algorithms on synthetic and real-world datasets. The experimental results demonstrate our algorithm is more reliable than its rivals.

Fair feature selection for classification decision tasks has recently garnered significant attention from researchers. However, existing fair feature selection algorithms fall short of providing a full explanation of the causal relationship between features and sensitive attributes, potentially impacting the accuracy of fair feature identification. To address this issue, we propose a fair causal feature selection algorithm, called FairCFS . Specifically, FairCFS constructs a localized causal graph that identifies the Markov blankets of class and sensitive variables, to block the transmission of sensitive information for selecting fair causal features. Extensive experiments on seven public real-world datasets validate that FairCFS has accuracy comparable to eight state-of-the-art feature selection algorithms while presenting more superior fairness.

Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (parents and children) of each variable in a dataset, then orient edges in the skeleton. However, existing MB or PC learning methods are often computationally expensive especially with a large-sized BN, resulting in inefficient local-to-global learning algorithms. To tackle the problem, in this article, we link feature selection with local BN structure learning and develop an efficient local-to-global learning approach using filtering feature selection. Specifically, we first analyze the rationale of the well-known Minimum-Redundancy and Maximum-Relevance (MRMR) feature selection approach for learning a PC set of a variable. Based on the analysis, we propose an efficient F2SL (feature selection-based structure learning) approach to local-to-global BN structure learning. The F2SL approach first employs the MRMR approach to learn the skeleton of a DAG, then orients edges in the skeleton. Employing independence tests or score functions for orienting edges, we instantiate the F2SL approach into two new algorithms, F2SL-c (using independence tests) and F2SL-s (using score functions). Compared to the state-of-the-art local-to-global BN learning algorithms, the experiments validated that the proposed algorithms in this article are more efficient and provide competitive structure learning quality than the compared algorithms.

Cross-lingual entity alignment (CLEA) aims to find equivalent entity pairs between knowledge graphs (KGs) in different languages. It is an important way to connect heterogeneous KGs and facilitate knowledge completion. Existing methods have found that incorporating relations into entities can effectively improve KG representation and benefit entity alignment, and these methods learn relation representation depending on entities, which cannot capture the diverse structures of relations. However, multiple relations in KG form diverse structures, such as adjacency structure and ring structure. This diversity of relation structures makes the relation representation challenging. Therefore, we propose to construct the weighted line graphs to model the diverse structures of relations and learn relation representation independently from entities. Especially, owing to the diversity of adjacency structures and ring structures, we propose to construct adjacency line graph and ring line graph, respectively, to model the structures of relations and to further improve entity representation. In addition, to alleviate the hubness problem in alignment, we introduce the optimal transport into alignment and compute the distance matrix in a different way. From a global perspective, we calculate the optimal 1-to-1 alignment bi-directionally to improve the alignment accuracy. Experimental results on two benchmark datasets show that our proposed method significantly outperforms state-of-the-art CLEA methods in both supervised and unsupervised manners.

Inferring causal structures from experimentation is a challenging task in many fields. Most causal structure learning algorithms with unknown interventions are proposed to discover causal relationships over an identical variable set. However, often due to privacy, ethical, financial, and practical concerns, the variable sets observed by multiple sources or domains are not entirely identical. While a few algorithms are proposed to handle the partially overlapping variable sets, they focus on the case of known intervention targets. Therefore, to be close to the real-world environment, we consider discovering causal relationships over overlapping variable sets under the unknown intervention setting and exploring a scenario where a problem is studied across multiple domains. Here, we propose an algorithm for discovering the causal relationships over the integrated set of variables from unknown interventions, mainly handling the entangled inconsistencies caused by the incomplete observation of variables and unknown intervention targets. Specifically, we first distinguish two types of inconsistencies and then deal with respectively them by presenting some lemmas. Finally, we construct a fusion rule to combine learned structures of multiple domains, obtaining the final structures over the integrated set of variables. Theoretical analysis and experimental results on synthetic, benchmark, and real-world datasets have verified the effectiveness of the proposed algorithm.

Causal structure learning (CSL) from observational data is a crucial objective in various machine learning applications. Recent advances in CSL have focused on local-to-global learning, which offers improved efficiency and accuracy. The local-to-global CSL algorithms first learn the local skeleton of each variable in a dataset, then construct the global skeleton by combining these local skeletons, and finally orient edges to infer causality. However, data quality issues such as noise and small samples often result in the presence of problematicasymmetric edgesduring global skeleton construction, hindering the creation of a high-quality global skeleton. To address this challenge, we propose a novel local-to-global CSL algorithm with a progressive enhancement strategy and make the following novel contributions: 1) To construct an accurate global skeleton, we design a novel strategy to iteratively correctasymmetric edgesand progressively improve the accuracy of the global skeleton. 2) Based on the learned accurate global skeleton, we design an integrated global skeleton orientation strategy to infer the correct directions of edges for obtaining an accurate and reliable causal structure. Extensive experiments demonstrate that our method achieves better performance than the existing CSL methods.

Instrumental variable (IV) is a powerful approach to inferring the causal effect of a treatment on an outcome of interest from observational data even when there exist latent confounders between the treatment and the outcome. However, existing IV methods require that an IV is selected and justified with domain knowledge. An invalid IV may lead to biased estimates. Hence, discovering a valid IV is critical to the applications of IV methods. In this article, we study and design a data-driven algorithm to discover valid IVs from data under mild assumptions. We develop the theory based on partial ancestral graphs (PAGs) to support the search for a set of candidate ancestral IVs (AIVs), and for each possible AIV, the identification of its conditioning set. Based on the theory, we propose a data-driven algorithm to discover a pair of IVs from data. The experiments on synthetic and real-world datasets show that the developed IV discovery algorithm estimates accurate estimates of causal effects in comparison with the state-of-the-art IV-based causal effect estimators.

Causal feature selection methods aim to identify a Markov boundary (MB) of a class variable, and almost all the existing causal feature selection algorithms use conditional independence (CI) tests to learn the MB. However, in real-world applications, due to data issues (e.g., noisy or small samples), CI tests can be unreliable; thus, causal feature selection algorithms relying on CI tests encounter two types of errors: false positives (i.e., selecting false MB features) and false negatives (i.e., discarding true MB features). Existing algorithms only tackle either false positives or false negatives, and they cannot deal with both types of errors at the same time, leading to unsatisfactory results. To address this issue, we propose a dual-correction-strategy-based MB learning (DCMB) algorithm to correct the two types of errors simultaneously. Specifically, DCMB selectively removes false positives from the MB features currently selected, while selectively retrieving false negatives from the features currently discarded. To automatically determine the optimal number of selected features for the selective removal and retrieval in the dual correction strategy, we design the simulated-annealing-based DCMB (SA-DCMB) algorithm. Using benchmark Bayesian network (BN) datasets, the experimental results demonstrate that DCMB achieves substantial improvements on the MB learning accuracy compared with the existing MB learning methods. Empirical studies in real-world datasets validate the effectiveness of SA-DCMB for classification against state-of-the-art causal and traditional feature selection algorithms.

Few-shot knowledge graph completion (FKGC), which aims to infer new triples for a relation using only a few reference triples of the relation, has attracted much attention in recent years. Most existing FKGC methods learn a transferable embedding space, where entity pairs belonging to the same relations are close to each other. In real-world knowledge graphs (KGs), however, some relations may involve multiple semantics, and their entity pairs are not always close due to having different meanings. Hence, the existing FKGC methods may yield suboptimal performance when handling multiple semantic relations in the few-shot scenario. To solve this problem, we propose a new method named adaptive prototype interaction network (APINet) for FKGC. Our model consists of two major components: 1) an interaction attention encoder (InterAE) to capture the underlying relational semantics of entity pairs by modeling the interactive information between head and tail entities and 2) an adaptive prototype net (APNet) to generate relation prototypes adaptive to different query triples by extracting query-relevant reference pairs and reducing the data inconsistency between support and query sets. Experimental results on two public datasets demonstrate that APINet outperforms several state-of-the-art FKGC methods. The ablation study demonstrates the rationality and effectiveness of each component of APINet.

In educational data mining, knowledge tracing (KT) aims to model learning performance based on student knowledge mastery. Deep-learning-based KT models perform remarkably better than traditional KT and have attracted considerable attention. However, most of them lack interpretability, making it challenging to explain why the model performed well in the prediction. In this paper, we propose an interpretable deep KT model, referred to as fuzzy deep knowledge tracing (FDKT) via fuzzy reasoning. Specifically, we formalize continuous scores into several fuzzy scores using the fuzzification module. Then, we input the fuzzy scores into the fuzzy reasoning module (FRM). FRM is designed to deduce the current cognitive ability, based on which the future performance was predicted. FDKT greatly enhanced the intrinsic interpretability of deep-learning-based KT through the interpretation of the deduction of student cognition. Furthermore, it broadened the application of KT to continuous scores. Improved performance with regard to both the advantages of FDKT was demonstrated through comparisons with the state-of-the-art models.

Few-shot knowledge graph completion (FKGC), which aims to infer missing facts about a relation from only a few reference triples, has recently attracted great attention. The core of solving the FKGC task is to learn a vector representation for each few-shot relation using the corresponding entity represen-tations. To this end, existing models generally enhance entity representations with their direct neighbors. However, a large number of entities have few direct neighbors. Hence, encoding only direct neighborhood is insufficient to obtain satisfactory en-tity representations. In addition, current models typically utilize static embeddings to represent entities, ignoring their diverse semantics, i.e., an entity may show distinct semantics within different few-shot relations. To address these issues, we propose a new FKGC framework, namely TransD-based Multi-hop Meta Learning (TDML). TDML consists of three main components: a multi-hop neighbor encoder to enhance entity representations by aggregating heterogeneous multi-hop neighbors, a transformer encoder to generate the relation meta representations, and a TransD-based relation representation updater that allows each entity to exhibit relation-specific semantics and tune the relation meta representations. Extensive experiments on two public datasets demonstrate that our model outperforms state-of-the-art FKGC methods.

Causal structure learning has been extensively studied and widely used in machine learning and various applications. To achieve an ideal performance, existing causal structure learning algorithms often need to centralize a large amount of data from multiple data sources. However, in the privacy-preserving setting, it is impossible to centralize data from all sources and put them together as a single dataset. To preserve data privacy, federated learning as a new learning paradigm has attached much attention in machine learning in recent years. In this paper, we study a privacy-aware causal structure learning problem in the federated setting and propose a novel federated PC (FedPC) algorithm with two new strategies for preserving data privacy without centralizing data. Specifically, we first propose a novel layer-wise aggregation strategy for a seamless adaptation of the PC algorithm into the federated learning paradigm for federated skeleton learning, then we design an effective strategy for learning consistent separation sets for federated edge orientation. The extensive experiments validate that FedPC is effective for causal structure learning in federated learning setting.

Real-world applications especially in the fields of social media have produced massive short text streams. Unlike traditional normal texts, these data present the characteristics of short length, high-volume, high-velocity and variable data distribution etc, which lead to the issues of data sparsity and concept drift. It is hence very challenging for existing short text classification algorithms. Therefore, we propose a flexible Long Short-Term Memory (LSTM) ensemble network based short text stream classification approach, which is implemented in a distributed mode while maintaining the high-accuracy advantage of deep learning models. More specifically, external resource based short text embedding using a pretrained embedding model and CNN is first proposed for the solution to the data sparsity of short texts. Second, to adapt to the high-volume and high-velocity short text streams, a flexible LSTM network is developed and implemented in a distributed mode for classifying short text data streams. Third, a concept drift factor is introduced for adapting to the concept drifts caused by the changing of data distributions. Finally, experiments conducted on three real short text data sets demonstrate that as compared with several state-of-the-art short text (stream) classification approaches, the proposed approach can classify short text streams effectively and efficiently while adapting to concept drifts.

Causal feature selection has recently attracted much more attention because it can improve the interpretability of predictive models. However, the existing causal feature selection framework needs to discover the PC (i.e., parents and children) of each variable in the PC of a target variable for spouses discovery, which is time-consuming on high-dimensional data. To tackle this issue, we propose a novelCausalFeatureSelection framework with efficient spouses discovery, called CFS. Specifically, by exploiting the dependency change property between a variable and its non-PC, the proposed framework only discovers the PC of the variables in some children of the target variable for spouses discovery. Furthermore, based on the proposed CFS framework and existing PC discovery algorithms, we propose four new causal feature selection algorithms. Using benchmark Bayesian networks and real-world datasets, we experimentally validated the efficiency and accuracy of the proposed algorithms compared with seven state-of-the-art causal feature selection algorithms.

Almost all existing causal feature selection methods are proposed without considering the problem of sample selection bias. However, in practice, as data-gathering process cannot be fully controlled, sample selection bias often occurs, leading to spurious correlations between features and the class variable, which seriously deteriorates the performance of those existing methods. In this article, we study the problem of causal feature selection under sample selection bias and propose a novel Progressive Causal Feature Selection (PCFS) algorithm which has three phases. First, PCFS learns the sample weights to balance the treated group and control group distributions corresponding to each feature for removing spurious correlations. Second, based on the sample weights, PCFS uses a weighted cross-entropy model to estimate the causal effect of each feature and removes some irrelevant features from the confounder set. Third, PCFS progressively repeats the first two phases to remove more irrelevant features and finally obtains a causal feature set. Using synthetic and real-world datasets, the experiments have validated the effectiveness of PCFS, in comparison with several state-of-the-art classical and causal feature selection methods.

Directed acyclic graph (DAG) learning plays a key role in causal discovery and many machine learning tasks. Learning a DAG from high-dimensional data always faces scalability problems. A local-to-global DAG learning approach can be scaled to high-dimensional data, however, existing local-to-global DAG learning algorithms employ either the AND-rule or the OR-rule for constructing a DAG skeleton. Simply using either rule, existing local-to-global methods may learn an inaccurate DAG skeleton, leading to unsatisfactory DAG learning performance. To tackle this problem, in this paper, we propose anAdaptiveDAGLearning (ADL) algorithm. The novel contribution of ADL is that it can simultaneously and adaptively use the AND-rule and the OR-rule to construct an accurate global DAG skeleton. We conduct extensive experiments on both benchmark and real-world datasets, and the experimental results show that ADL is significantly better than some existing local-to-global and global DAG learning algorithms.

Causal feature selection has received increasing attention in recent years. However, the state-of-the-art causal feature selection algorithms use the conditional independence tests, which require enumerating conditioning sets, leading to an exponential increase in computational complexity along with an increase in feature space. To address this problem, in this paper, we theoretically analyze the unique performance of causal features in mutual information, and propose a novelCausalFeatureSelection algorithm usingMutualInformation, called CFS-MI. Specifically, CFS-MI separately instantiates the pairwise comparison of mutual information in two stages to reduce computational complexity, and thus improves the efficiency on high-dimensional data. Extensive experiments on 5 benchmark Bayesian networks and 16 real-world datasets validate that CFS-MI has comparable accuracy compared to 7 state-of-the-art causal feature selection algorithms, while presenting more superior computational efficiency.

In this study, we investigate a challenging problem, namely, robust domain adaptation, where data from only a single well-labeled source domain are available in the training phase. To address this problem, assuming that the causal relationships between the features and the class variable are robust across domains, we propose a novel causal autoencoder (CAE), which integrates a deep autoencoder and a causal structure learning model to learn causal representations using data from a single source domain. Specifically, a deep autoencoder model is adopted to learn the low-dimensional representations, and a causal structure learning model is designed to separate the low-dimensional representations into two groups: causal representations and task-irrelevant representations. Using three real-world datasets, the experiments have validated the effectiveness of CAE, in comparison with eleven state-of-the-art methods.

Cross-lingual entity alignment, which is an important task in the field of graph mining, aims to find equivalent entity pairs from two knowledge graphs. Recent methods show that relation representation can be used to improve entity representation and entity alignment. However, relation representation is learned dependently on entity representation, and both representations are learned from node-centered knowledge graphs. This dependency of relation on entity results in poor relation representation and leads to limited enhancement of entity representation. Therefore, to address this challenge, we propose novel relation-aware line graph neural networks for cross-lingual entity alignment (RALG). More specifically, we first propose to learn relation representation with heterogeneous line graphs independently from entities. The constructed heterogeneous line graphs can capture the correlation of relations explicitly. Secondly, we design a new way of aggregation in the form of triples to strengthen the relevance between entities and their corresponding relations. Experiments conducted on real-world datasets show that independent learning of relation representation with line graphs can represent relations better, and our method achieves better performance than the state-of-the-art methods for entity alignment.

Causal effect estimation from observational data is a crucial but challenging task. Currently, only a limited number of data-driven causal effect estimation methods are available. These methods either provide only a bound estimation of causal effects of treatment on the outcome or generate a unique estimation of the causal effect but making strong assumptions on data and having low efficiency. In this article, we identify a problem setting with the Cause Or Spouse of the treatment Only (COSO) variable assumption and propose an approach to achieving a unique and unbiased estimation of causal effects from data with hidden variables. For the approach, we have developed the theorems to support the discovery of the proper covariate sets for confounding adjustment (adjustment sets). Based on the theorems, two algorithms are proposed for finding the proper adjustment sets from data with hidden variables to obtain unbiased and unique causal effect estimation. Experiments with synthetic datasets generated using five benchmark Bayesian networks and four real-world datasets have demonstrated the efficiency and effectiveness of the proposed algorithms, indicating the practicability of the identified problem setting and the potential of the proposed approach in real-world applications.

We consider the problem of distinguishing direct causes from direct effects of a target variable of interest from multiple manipulated datasets with unknown manipulated variables and nonidentical data distributions. Recent studies have shown that datasets attained from manipulated experiments (i.e., manipulated data) contain richer causal information than observational data for causal structure learning. Thus, in this article, we propose a new algorithm, which makes full use of the interventional properties of a causal model to discover the direct causes and direct effects of a target variable from multiple datasets with different manipulations. It is more suited to real-world cases and is also a challenge to be addressed in this article. First, we apply the backward framework to learn parents and children (PC) of a given target from multiple manipulated datasets. Second, we orient some edges connected to the target in advance through the assumption that the target variable is not manipulated and then orient the remaining undirected edges by finding invariant V-structures from multiple datasets. Third, we analyze the correctness of the proposed algorithm. To the best of our knowledge, the proposed algorithm is the first that can identify the local causal structure of a given target from multiple manipulated datasets with unknown manipulated variables. Experimental results on standard Bayesian networks validate the effectiveness of our algorithm.

Multilabel feature selection plays an essential role in high-dimensional multilabel learning tasks. Existing multilabel feature selection approaches mainly either explore the feature-label and feature-feature correlations or the label-label and feature-feature correlations. A few of them are able to deal with all three types of correlations simultaneously. To address this problem, in this article, we formulate multilabel feature selection as a local causal structure learning problem and propose a novel algorithm, M2LC. By learning the local causal structure of each class label, M2LC considers three types of feature relationships simultaneously and is scalable to high-dimensional datasets as well. To tackle false discoveries caused by the label-label correlations, M2LC consists of two novel error-correction subroutines to correct those false discoveries. Through local causal structure learning, M2LC learns the causal mechanism behind data, and thus, it can select causally informative features and visualize common features shared by class labels and specific features owned by an individual class label using the learned causal structures. Extensive experiments have been conducted to evaluate M2LC in comparison with the state-of-the-art multilabel feature selection algorithms.

We consider the problem of reducing the false discovery rate in multiple high-dimensional interventional datasets under unknown targets. Traditional algorithms merged directly multiple causal graphs learned, which ignores the contradictions of different datasets, leading to lots of inconsistent directions of edges. For reducing the contradictory information, we propose a new algorithm, which first learns an interventional Markov equivalence class (I-MEC) before merging multiple graphs. It utilizes the full power of the constraints available in interventional data and combines ideas from local learning, intervention, and search-and-score techniques in a principled and effective way in different intervention experiments. Specifically, local learning on multiple datasets is used to build a causal skeleton. Perfect intervention destroys some possible triangles, leading to the identification of more possible V-structures. And then a theoretically correct I-MEC is learned. Search and scoring techniques based on the learned I-MEC further identify the remaining unoriented edges. Both theoretical analysis and experiments on benchmark Bayesian networks with the number of variables from 20 to 724 validate that the effectiveness of our algorithm in reducing the false discovery rate in high-dimensional interventional data.

Learning a causal structure from observational data is crucial for data scientists. Recent advances in causal structure learning (CSL) have focused on local-to-global learning, since the local-to-global CSL can be scaled to high-dimensional data. The local-to-global CSL algorithms first learn the local skeletons, then construct the global skeleton, and finally orient edges. In practice, the performance of local-to-global CSL mainly depends on the accuracy of the global skeleton. However, in many real-world settings, owing to inevitable data quality issues (e.g. noise and small sample), existing local-to-global CSL methods often yield many asymmetric edges (e.g., given anasymmetric edge containing variables A and B, the learned skeleton of A contains B, but the learned skeleton of B does not contain A), which make it difficult to construct a high quality global skeleton. To tackle this problem, this paper proposes a Bootstrap sampling based Causal Structure Learning (BCSL) algorithm. The novel contribution of BCSL is that it proposes an integrated global skeleton learning strategy that can construct more accurate global skeletons. Specifically, this strategy first utilizes the Bootstrap method to generate multiple sub-datasets, then learns the local skeleton of variables on each asymmetric edge on those sub-datasets, and finally designs a novel scoring function to estimate the learning results on all sub-datasets for correcting the asymmetric edge. Extensive experiments on both benchmark and real datasets verify the effectiveness of the proposed method.

Few-shot knowledge graph completion (FKGC) aims to infer unknown fact triples of a relation using its few-shot reference entity pairs.Recent FKGC studies focus on learning semantic representations of entity pairs by separately encoding the neighborhoods of head and tail entities.Such practice, however, ignores the inter-entity interaction, resulting in low-discrimination representations for entity pairs, especially when these entity pairs are associated with 1-to-N, N-to-1, and N-to-N relations.To address this issue, this paper proposes a novel FKGC model, named Cross-Interaction Attention Network (CIAN) to investigate the inter-entity interaction between head and tail entities.Specifically, we first explore the interactions within entities by computing the attention between the task relation and each entity neighbor, and then model the interactions between head and tail entities by letting an entity to attend to the neighborhood of its paired entity.In this way, CIAN can figure out the relevant semantics between head and tail entities, thereby generating more discriminative representations for entity pairs.Extensive experiments on two public datasets show that CIAN outperforms several state-of-the-art methods.The source code is available at https://github.com/cjlyl/FKGC-CIAN.

Multi-label feature selection is an effective approach to alleviate the high dimensionality of multi-label learning tasks. Most of the existing multi-label feature selection methods are based on the assumption that the relevant labels of each training sample are precisely annotated. However, in real-world applications, this assumption is not always held, each instance may be labeled with a set of candidate labels that contains all the ground-truth labels adulterated with noisy labels, which is called partial multi-label problem. Previous multi-label feature selection methods can not select the optimal feature set in the presence of partial multi-label. Therefore, to tackle this problem we propose a novel partial multi-label feature selection method, called PMLFS. To be specific, ground-truth labels and noisy labels are firstly distinguished in terms of label correlations and the sparsity of noisy labels. Secondly, manifold regularization is incorporated to explore the local structure. Based on the above, we design an objective framework involving$l_{2,1}$-norm regularization to achieve partial multi-label feature selection. Finally, extensive experiments on synthetic and real-world partial multi-label data sets demonstrate that the proposed method outperforms the state-of-the-art multi-label feature selection methods.

Abstract A large number of covariates can have a negative impact on the quality of causal effect estimation since confounding adjustment becomes unreliable when the number of covariates is large relative to the number of samples. Propensity score is a common way to deal with a large covariate set, but the accuracy of propensity score estimation (normally done by logistic regression) is also challenged by the large number of covariates. In this paper, we prove that a large covariate set can be reduced to a lower dimensional representation which captures the complete information for adjustment in causal effect estimation. The theoretical result enables effective data-driven algorithms for causal effect estimation. Supported by the result, we develop an algorithm that employs a supervised kernel dimension reduction method to learn a lower dimensional representation from the original covariate space, and then utilises nearest neighbour matching in the reduced covariate space to impute the counterfactual outcomes to avoid the large sized covariate set problem. The proposed algorithm is evaluated on two semisynthetic and three real-world datasets and the results show the effectiveness of the proposed algorithm.

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms.

Domain adaptation aims to facilitate the learning task in an unlabeled target domain by leveraging the auxiliary knowledge in a well-labeled source domain from a different distribution. Almost existing autoencoder-based domain adaptation approaches focus on learning domain-invariant representations to reduce the distribution discrepancy between source and target domains. However, there is still a weakness existing in these approaches: the class-discriminative information of the two domains may be damaged while aligning the distributions of the source and target domains, which makes the samples with different classes close to each other, leading to performance degradation. To tackle this issue, we propose a novel dual-representation autoencoder (DRAE) to learn dual-domain-invariant representations for domain adaptation. Specifically, DRAE consists of three learning phases. First, DRAE learns global representations of all source and target data to maximize the interclass distance in each domain and minimize the marginal distribution and conditional distribution of both domains simultaneously. Second, DRAE extracts local representations of instances sharing the same label in both domains to maintain class-discriminative information in each class. Finally, DRAE constructs dual representations by aligning the global and local representations with different weights. Using three text and two image datasets and 12 state-of-the-art domain adaptation methods, the extensive experiments have demonstrated the effectiveness of DRAE.

Online education promotes the sharing of learning resources. Knowledge tracing (KT) is aimed at tracking the cognition function of students according to their performance on various exercises at different times and has attracted considerable attention. Existing KT models primarily use bisection representations for the performance and cognitive states of students, thus limiting the application scope of these models and the accuracy of the evaluation of student cognitive performance in learning processes. Therefore, fuzzy Bayesian KT models (namely, FBKT and T2FBKT) are proposed to address continuous score scenarios (e.g., subjective examinations) so that the applicability of KT models may be broadened. Moreover, fine-grained cognitive states can be discerned. In particular, referring to type-2 fuzzy theory, T2FBKT mitigates the model uncertainty of FBKT induced by uncertain parameters. Finally, extensive experiments demonstrate the effectiveness of the proposed fuzzy KT models.

Learning partial Bayesian network (BN) structure is an interesting and challenging problem. In this challenge, it is computationally expensive to use global BN structure learning algorithms, while only one part of a BN structure is interesting, local BN structure learning algorithms are not a favourable solution either due to the issue of false edge orientation. To address the problem, this article first presents a detailed analysis of the false edge orientation issue with local BN structure learning algorithms and then proposes PSL, an efficient and accurate P artial BN S tructure L earning (PSL) algorithm. Specifically, PSL divides V-structures in a Markov blanket (MB) into two types: Type-C V-structures and Type-NC V-structures, then it starts from the given node of interest and recursively finds both types of V-structures in the MB of the current node until all edges in the partial BN structure are oriented. To further improve the efficiency of PSL, the PSL-FS algorithm is designed by incorporating F eature S election (FS) into PSL. Extensive experiments with six benchmark BNs validate the efficiency and accuracy of the proposed algorithms.

Causal feature selection aims at learning the Markov blanket (MB) of a class variable for feature selection. The MB of a class variable implies the local causal structure among the class variable and its MB and all other features are probabilistically independent of the class variable conditioning on its MB, this enables causal feature selection to identify potential causal features for feature selection for building robust and physically meaningful prediction models. Missing data, ubiquitous in many real-world applications, remain an open research problem in causal feature selection due to its technical complexity. In this article, we discuss a novel multiple imputation MB (MimMB) framework for causal feature selection with missing data. MimMB integrates Data Imputation with MB Learning in a unified framework to enable the two key components to engage with each other. MB Learning enables Data Imputation in a potentially causal feature space for achieving accurate data imputation, while accurate Data Imputation helps MB Learning identify a reliable MB of the class variable in turn. Then, we further design an enhanced kNN estimator for imputing missing values and instantiate the MimMB. In our comprehensively experimental evaluation, our new approach can effectively learn the MB of a given variable in a Bayesian network and outperforms other rival algorithms using synthetic and real-world datasets.

One mainstream method in cross-lingual word embeddings is to learn a linear mapping between two monolingual embedding spaces using a training dictionary. Successful linear mappings require isomorphic embedding spaces. However, monolingual embedding spaces are not perfectly isomorphic, and therefore, a linear mapping cannot align them accurately. In this study, we assume that two embedding spaces are composed of near-isomorphic translation pairs (NearITP) and non-isomorphic translation pairs. Owing to the nature of similar substructures, NearITP can make linear mapping work well. Motivated by this, we design a screening strategy to identify NearITP effectively. Based on this strategy, we find that the proportion of NearITP in the commonly used training dictionary is relatively low, leading to sub-optimal results. To address this problem, we propose a general framework that can be combined with any of the mapping methods, which further boosts subsequent mapping. Experimental results demonstrate that our framework is an improvement over existing mapping-based methods, and outperforms state-of-the-art models on two public data sets. Moreover, we show that our framework can be successfully generalized to contextual word embeddings such as multilingual BERT (mBERT), and further enhances the cross-lingual properties of mBERT.

In this article, we aim to develop a unified view of causal and non-causal feature selection methods. The unified view will fill in the gap in the research of the relation between the two types of methods. Based on the Bayesian network framework and information theory, we first show that causal and non-causal feature selection methods share the same objective. That is to find the Markov blanket of a class attribute, the theoretically optimal feature set for classification. We then examine the assumptions made by causal and non-causal feature selection methods when searching for the optimal feature set, and unify the assumptions by mapping them to the restrictions on the structure of the Bayesian network model of the studied problem. We further analyze in detail how the structural assumptions lead to the different levels of approximations employed by the methods in their search, which then result in the approximations in the feature sets found by the methods with respect to the optimal feature set. With the unified view, we can interpret the output of non-causal methods from a causal perspective and derive the error bounds of both types of methods. Finally, we present practical understanding of the relation between causal and non-causal methods using extensive experiments with synthetic data and various types of real-world data.

Multi-label feature selection has received considerable attentions during the past decade. However, existing algorithms do not attempt to uncover the underlying causal mechanism, and individually solve different types of variable relationships, ignoring the mutual effects between them. Furthermore, these algorithms lack of interpretability, which can only select features for all labels, but cannot explain the correlation between a selected feature and a certain label. To address these problems, in this paper, we theoretically study the causal relationships in multi-label data, and propose a novel Markov blanket based multi-label causal feature selection (MB-MCF) algorithm. MB-MCF mines the causal mechanism of labels and features first, to obtain a complete representation of information about labels. Based on the causal relationships, MB-MCF then selects predictive features and simultaneously distinguishes common features shared by multiple labels and label-specific features owned by single labels. Experiments on real-world data sets validate that MB-MCF could automatically determine the number of selected features and simultaneously achieve the best performance compared with state-of-the-art methods. An experiment in Emotions data set further demonstrates the interpretability of MB-MCF.

This paper discusses the problem of causal query in observational data with hidden variables, with the aim of seeking the change of an outcome when "manipulating" a variable while given a set of plausible confounding variables which affect the manipulated variable and the outcome. Such an "experiment on data" to estimate the causal effect of the manipulated variable is useful for validating an experiment design using historical data or for exploring confounders when studying a new relationship. However, existing data-driven methods for causal effect estimation face some major challenges, including poor scalability with high dimensional data, low estimation accuracy due to heuristics used by the global causal structure learning algorithms, and the assumption of causal sufficiency when hidden variables are inevitable in data. In this paper, we develop theorems for using local search to find a superset of the adjustment (or confounding) variables for causal effect estimation from observational data under a realistic pretreatment assumption. The theorems ensure that the unbiased estimate of causal effect is included in the set of causal effects estimated by the superset of adjustment variables. Based on the developed theorems, we propose a data-driven algorithm for causal query. Experiments show that the proposed algorithm is faster and produces better causal effect estimation than an existing data-driven causal effect estimation method with hidden variables. The causal effects estimated by the proposed algorithm are as accurate as those by the state-of-the-art methods using domain knowledge.

Causal feature selection has attracted much attention in recent years, as the causal features selected imply the causal mechanism related to the class attribute, leading to more reliable prediction models built using them. Currently there is a need of developing multi-source feature selection methods, since in many applications data for studying the same problem has been collected from various sources, such as multiple gene expression datasets obtained from different experiments for studying the causes of the same disease. However, the state-of-the-art causal feature selection methods generally tackle a single dataset, and a direct application of the methods to multiple datasets will result in unreliable results as the datasets may have different distributions. To address the challenges, by utilizing the concept of causal invariance in causal inference, we first formulate the problem of causal feature selection with multiple datasets as a search problem for an invariant set across the datasets, then give the upper and lower bounds of the invariant set, and finally we propose a new Multi-source Causal Feature Selection algorithm, MCFS. Using synthetic and real world datasets and 16 feature selection methods, the extensive experiments have validated the effectiveness of MCFS.

Causal feature selection has achieved much attention in recent years, which discovers a Markov boundary (MB) of the class attribute. The MB of the class attribute implies local causal relations between the class attribute and the features, thus leading to more interpretable and robust prediction models than the features selected by the traditional feature selection algorithms. Many causal feature selection methods have been proposed, and almost all of them employ conditional independence (CI) tests to identify MBs. However, many datasets from real-world applications may suffer from incorrect CI tests due to noise or small-sized samples, resulting in lower MB discovery accuracy for these existing algorithms. To tackle this issue, in this article, we first introduce a new concept of PCMasking to explain a type of incorrect CI tests in the MB discovery, then propose a cross-check and complement MB discovery (CCMB) algorithm to repair this type of incorrect CI tests for accurate MB discovery. To improve the efficiency of CCMB, we further design a pipeline machine-based CCMB (PM-CCMB) algorithm. Using benchmark Bayesian network datasets, the experiments demonstrate that both CCMB and PM-CCMB achieve significant improvements on the MB discovery accuracy compared with the existing methods, and PM-CCMB further improves the computational efficiency. The empirical study in the real-world datasets validates the effectiveness of CCMB and PM-CCMB against the state-of-the-art causal and traditional feature selection algorithms.

Learning Markov blankets (MBs) plays an important role in many machine learning tasks, such as causal Bayesian network structure learning, feature selection, and domain adaptation. Since variables included in the MB of a target variable of interest have causal relationships with the target, the MB can serve as the basis of learning the global structure of a causal Bayesian network or as a reliable and robust feature set for classification, both within the same domain or across domains. In this article, we study the problem of learning the MB of a target variable from multiple interventional data sets. Data sets attained from interventional experiments contain richer causal information than passively observed data (observational data) for MB discovery. However, almost all existing MB discovery methods are designed for learning MBs from a single observational data set. To learn MBs from multiple interventional data sets, we face two challenges: 1) unknown intervention variables and 2) nonidentical data distributions. To address these challenges, we theoretically analyze: 1) under what conditions we can find the correct MB of a target variable and 2) under what conditions we can identify the causes of the target variable via discovering its MB. Based on the theoretical analysis, we propose a new algorithm for learning MBs from multiple interventional data sets, and we present the conditions/assumptions that assure the correctness of the algorithm. To the best of our knowledge, this article is the first to present the theoretical analyses about the conditions for MB discovery in multiple interventional data sets and the algorithm to find the MBs in relation to the conditions. Using benchmark Bayesian networks and real-world data sets, the experiments have validated the effectiveness and efficiency of the proposed algorithm in this article.

With the increasing data dimensionality, feature selection has become a fundamental task to deal with high-dimensional data. Semi-supervised feature selection focuses on the problem of how to learn a relevant feature subset in the case of abundant unlabeled data with few labeled data. In recent years, many semi-supervised feature selection algorithms have been proposed. However, these algorithms are implemented by separating the processes of feature selection and classifier training, such that they cannot simultaneously select features and learn a classifier with the selected features. Moreover, they ignore the difference of reliability inside unlabeled samples and directly use them in the training stage, which might cause performance degradation. In this paper, we propose a joint semi-supervised feature selection and classification algorithm (JSFS) which adopts a Bayesian approach to automatically select the relevant features and simultaneously learn a classifier. Instead of using all unlabeled samples indiscriminately, JSFS associates each unlabeled sample with a self-adjusting weight to distinguish the difference between them, which can effectively eliminate the irrelevant unlabeled samples via introducing a left-truncated Gaussian prior. Experiments on various datasets demonstrate the effectiveness and superiority of JSFS.

Computer-aided diagnosis (CAD) is mainly used in disease diagnosis and cause analysis. For example, using CAD to make early predictions of the semantic features of lung nodules is critical for helping physicians judge the semantic features of solitary pulmonary nodules. It is an effective method to predict disease using the features calculated from CT images. But how to select the most relevant features from the large number of image features is still a challenge. In this paper, we perform feature selection using maximum-relevance minimum-redundancy criteria based on applying a support vector machine (MRMR_SVM) on four types of computed image features to predict the semantic characteristics of pulmonary nodules over seven categories. The proposed method has the following advantages. 1) It improves work efficiency and reduces costs compared to manual evaluation by radiologists. 2) It combines a few key image features with specific semantic features to provide a basis for radiologists' diagnosis. 3) It eliminates noisy data and improves accuracy of early prediction of pulmonary nodules compared to using all features. The experimental results show that the proposed method performs well at predicting the semantic features of lung nodules in terms of accuracy and running time.

The discovery of Markov blanket (MB) for feature selection has attracted much attention in recent years, since the MB of the class attribute is the optimal feature subset for feature selection. However, almost all existing MB discovery algorithms focus on either improving computational efficiency or boosting learning accuracy, instead of both. In this article, we propose a novel MB discovery algorithm for balancing efficiency and accuracy, called BAlanced Markov Blanket (BAMB) discovery. To achieve this goal, given a class attribute of interest, BAMB finds candidate PC (parents and children) and spouses and removes false positives from the candidate MB set in one go. Specifically, once a feature is successfully added to the current PC set, BAMB finds the spouses with regard to this feature, then uses the updated PC and the spouse set to remove false positives from the current MB set. This makes the PC and spouses of the target as small as possible and thus achieves a trade-off between computational efficiency and learning accuracy. In the experiments, we first compare BAMB with 8 state-of-the-art MB discovery algorithms on 7 benchmark Bayesian networks, then we use 10 real-world datasets and compare BAMB with 12 feature selection algorithms, including 8 state-of-the-art MB discovery algorithms and 4 other well-established feature selection methods. On prediction accuracy, BAMB outperforms 12 feature selection algorithms compared. On computational efficiency, BAMB is close to the IAMB algorithm while it is much faster than the remaining seven MB discovery algorithms.

It is a grand challenge to identify the outliers existing in subspaces from a high-dimensional data set. A brute-force method is computationally prohibitive since it requires examining an exponential number of subspaces. Current state-of-the-art methods explore various heuristics to significantly prune subspaces, facing the tradeoff between the subspace completeness and search efficiency. In this brief, we discuss a principal type of subspace outliers whose behaviors are different from the others on individual attributes. We formulate such outliers by a novel notion of the Markov boundary-based (MBB) outliers. The central idea is that for each attribute T in a data set, we consider only the subspace representing the knowledge needed to predict the behavior on T , which is captured by the MB of T . Then, the outliers whose behavior is different from others on T can be detected in the subspace of the MB, and thus, our approach reduces the number of possible subspaces from exponential to linear with respect to dimensionality. Using both synthetic and real data sets, we validate the effectiveness and efficiency of our method.

BACKGROUND: miRBase is the primary repository for published miRNA sequence and annotation data, and serves as the "go-to" place for miRNA research. However, the definition and annotation of miRNAs have been changed significantly across different versions of miRBase. The changes cause inconsistency in miRNA related data between different databases and articles published at different times. Several tools have been developed for different purposes of querying and converting the information of miRNAs between different miRBase versions, but none of them individually can provide the comprehensive information about miRNAs in miRBase and users will need to use a number of different tools in their analyses. RESULTS: We introduce miRBaseConverter, an R package integrating the latest miRBase version 22 available in Bioconductor to provide a suite of functions for converting and retrieving miRNA name (ID), accession, sequence, species, version and family information in different versions of miRBase. The package is implemented in R and available under the GPL-2 license from the Bioconductor website ( http://bioconductor.org/packages/miRBaseConverter/ ). A Shiny-based GUI suitable for non-R users is also available as a standalone application from the package and also as a web application at http://nugget.unisa.edu.au:3838/miRBaseConverter . miRBaseConverter has a built-in database for querying miRNA information in all species and for both pre-mature and mature miRNAs defined by miRBase. In addition, it is the first tool for batch querying the miRNA family information. The package aims to provide a comprehensive and easy-to-use tool for miRNA research community where researchers often utilize published miRNA data from different sources. CONCLUSIONS: The Bioconductor package miRBaseConverter and the Shiny-based web application are presented to provide a suite of functions for converting and retrieving miRNA name, accession, sequence, species, version and family information in different versions of miRBase. The package will serve a wide range of applications in miRNA research and could provide a full view of the miRNAs of interest.

Markov blankets (MBs) in Bayesian networks (BNs) play an important role in both local causal discovery and large-scale BN structure learning. Almost all existing MB discovery algorithms are designed under the assumption of causal sufficiency, which states that there are no latent common causes for two or more of the observed variables in data. However, latent common causes are ubiquitous in many applications, and hence, this assumption is often violated in practice. Thus, developing algorithms for discovering MBs without assuming causal sufficiency is of practical significance, and it is crucial for causal structure learning in real-world data. In this paper, we focus on addressing this problem. Specifically, we adopt a maximal ancestral graph (MAG) model to represent latent common causes and the concept of MBs without assuming causal sufficiency. Then, we propose an effective and efficient algorithm to discover the MB of a target variable in an MAG. Using benchmark and real-world data sets, the experiments validate the algorithm proposed in this paper.

It has received much attention in recent years to use Markov blankets in a Bayesian network for feature selection. The Markov blanket of a class attribute in a Bayesian network is a unique yet minimal feature subset for optimal feature selection if the probability distribution of a data set can be faithfully represented by this Bayesian network. However, if a data set violates the faithful condition, Markov blankets of a class attribute may not be unique. To tackle this issue, in this paper, we propose a new concept of representative sets and then design the selection via group alpha-investing (SGAI) algorithm to perform Markov blanket feature selection with representative sets for classification. Using a comprehensive set of real data, our empirical studies have demonstrated that SGAI outperforms the state-of-the-art Markov blanket feature selectors and other well-established feature selection methods.It has received much attention in recent years to use Markov blankets in a Bayesian network for feature selection. The Markov blanket of a class attribute in a Bayesian network is a unique yet minimal feature subset for optimal feature selection if the probability distribution of a data set can be faithfully represented by this Bayesian network. However, if a data set violates the faithful condition, Markov blankets of a class attribute may not be unique. To tackle this issue, in this paper, we propose a new concept of representative sets and then design the selection via group alpha-investing (SGAI) algorithm to perform Markov blanket feature selection with representative sets for classification. Using a comprehensive set of real data, our empirical studies have demonstrated that SGAI outperforms the state-of-the-art Markov blanket feature selectors and other well-established feature selection methods.

Feature selection is important in many big data applications. Two critical challenges closely associate with big data. First, in many big data applications, the dimensionality is extremely high, in millions, and keeps growing. Second, big data applications call for highly scalable feature selection algorithms in an online manner such that each feature can be processed in a sequential scan. We present SAOLA, a Scalable and Accurate OnLine Approach for feature selection in this paper. With a theoretical analysis on bounds of the pairwise correlations between features, SAOLA employs novel pairwise comparison techniques and maintains a parsimonious model over time in an online manner. Furthermore, to deal with upcoming features that arrive by groups, we extend the SAOLA algorithm, and then propose a new group-SAOLA algorithm for online group feature selection. The group-SAOLA algorithm can online maintain a set of feature groups that is sparse at the levels of both groups and individual features simultaneously. An empirical study using a series of benchmark real datasets shows that our two algorithms, SAOLA and group-SAOLA, are scalable on datasets of extremely high dimensionality and have superior performance over the state-of-the-art feature selection methods.

Reliable tornado forecasting with a long-lead time can greatly support emergency response and is of vital importance for the economy and society. The large number of meteorological variables in spatiotemporal domains and the complex relationships among variables remain the top difficulties for a long-lead tornado forecasting.

Many datasets from real-world applications have very high-dimensional or increasing feature space. It is a new research problem to learn and maintain a classifier to deal with very high dimensionality or streaming features. In this article, we adapt the well-known emerging-pattern--based classification models and propose a semi-streaming approach. For streaming features, it is computationally expensive or even prohibitive to mine long-emerging patterns, and it is nontrivial to integrate emerging-pattern mining with feature selection. We present an online feature selection step, which is capable of selecting and maintaining a pool of effective features from a feature stream. Then, in our offline step, separated from the online step, we periodically compute and update emerging patterns from the pool of selected features from the online step. We evaluate the effectiveness and efficiency of the proposed method using a series of benchmark datasets and a real-world case study on Mars crater detection. Our proposed method yields classification performance comparable to the state-of-art static classification methods. Most important, the proposed method is significantly faster and can efficiently handle datasets with streaming features.

Feature selection is important in many big data applications. There are at least two critical challenges. Firstly, in many applications, the dimensionality is extremely high, in millions, and keeps growing. Secondly, feature selection has to be highly scalable, preferably in an online manner such that each feature can be processed in a sequential scan. In this paper, we develop SAOLA, a Scalable and Accurate On Line Approach for feature selection. With a theoretical analysis on a low bound on the pair wise correlations between features in the currently selected feature subset, SAOLA employs novel online pair wise comparison techniques to address the two challenges and maintain a parsimonious model over time in an online manner. An empirical study using a series of benchmark real data sets shows that SAOLA is scalable on data sets of extremely high dimensionality, and has superior performance over the state-of-the-art feature selection methods.

In faithful Bayesian networks, the Markov blanket of the class attribute is a unique and minimal feature subset for optimal feature selection. However, little attention has been paid to Markov blanket feature selection in a non-faithful environment which widely exists in the real world. To tackle this issue, in this paper, we deal with non-faithful data distributions and propose the concept of representative sets instead of Markov blankets. With a standard sparse group lasso for selection of features from the representative sets, we design an effective algorithm, SRS, for Markov blanket feature Selection via Representative Sets with non-faithful data distributions. Empirical studies demonstrate that SRS outperforms the state-of-the-art Markov blanket feature selectors and other well-established feature selection methods.

The development of disastrous flood forecasting techniques able to provide warnings at a long lead-time (5-15 days) is of great importance to society. Extreme Flood is usually a consequence of a sequence of precipitation events occurring over from several days to several weeks. Though precise short-term forecasting the magnitude and extent of individual precipitation event is still beyond our reach, long-term forecasting of precipitation clusters can be attempted by identifying persistent atmospheric regimes that are conducive for the precipitation clusters. However, such forecasting will suffer from overwhelming number of relevant features and high imbalance of sample sets. In this paper, we propose an integrated data mining framework for identifying the precursors to precipitation event clusters and use this information to predict extended periods of extreme precipitation and subsequent floods. We synthesize a representative feature set that describes the atmosphere motion, and apply a streaming feature selection algorithm to online identify the precipitation precursors from the enormous feature space. A hierarchical re-sampling approach is embedded in the framework to deal with the imbalance problem. An extensive empirical study is conducted on historical precipitation and associated flood data collected in the State of Iowa. Utilizing our framework a few physically meaningful precipitation cluster precursor sets are identified from millions of features. More than 90% of extreme precipitation events are captured by the proposed prediction model using precipitation cluster precursors with a lead time of more than 5 days.

We propose a new online feature selection framework for applications with streaming features where the knowledge of the full feature space is unknown in advance. We define streaming features as features that flow in one by one over time whereas the number of training examples remains fixed. This is in contrast with traditional online learning methods that only deal with sequentially added observations, with little attention being paid to streaming features. The critical challenges for Online Streaming Feature Selection (OSFS) include 1) the continuous growth of feature volumes over time, 2) a large feature space, possibly of unknown or infinite size, and 3) the unavailability of the entire feature set before learning starts. In the paper, we present a novel Online Streaming Feature Selection method to select strongly relevant and nonredundant features on the fly. An efficient Fast-OSFS algorithm is proposed to improve feature selection performance. The proposed algorithms are evaluated extensively on high-dimensional datasets and also with a real-world case study on impact crater detection. Experimental results demonstrate that the algorithms achieve better compactness and higher prediction accuracy than existing streaming feature selection algorithms.

It is a nontrivial task to build an accurate emerging pattern (EP) classifier from high-dimensional data because we inevitably face two challenges 1) how to efficiently extract a minimal set of strongly predictive EPs from an explosive number of candidate patterns, and 2) how to handle the highly sensitive choice of the minimal support threshold. To address these two challenges, we bridge causal relevance and EP discriminability (the predictive ability of emerging patterns) to facilitate EP mining and propose a new framework of mining EPs from high-dimensional data. In this framework, we study the relationships between causal relevance in a causal Bayesian network and EP discriminability in EP mining, and then reduce the pattern space of EP mining to direct causes and direct effects, or the Markov blanket (MB) of the class attribute in a causal Bayesian network. The proposed framework is instantiated by two EPs-based classifiers, CE-EP and MB-EP, where CE stands for direct Causes and direct Effects, and MB for Markov Blanket. Extensive experiments on a broad range of data sets validate the effectiveness of the CE-EP and MB-EP classifiers against other well-established methods, in terms of predictive accuracy, pattern numbers, running time, and sensitivity analysis.

Building an accurate emerging pattern classifier with a high-dimensional dataset is a challenging issue. The problem becomes even more difficult if the whole feature space is unavailable before learning starts. This paper presents a new technique on mining emerging patterns using streaming feature selection. We model high feature dimensions with streaming features, that is, features arrive and are processed one at a time. As features flow in one by one, we online evaluate each coming feature to determine whether it is useful for mining predictive emerging patterns (EPs) by exploiting the relationship between feature relevance and EP discriminability (the predictive ability of an EP). We employ this relationship to guide an online EP mining process. This new approach can mine EPs from a high-dimensional dataset, even when its entire feature set is unavailable before learning. The experiments on a broad range of datasets validate the effectiveness of the proposed approach against other well-established methods, in terms of predictive accuracy, pattern numbers and running time.

Causal discovery is highly desirable in science and technology. In this paper, we study a new research problem of discovery of causal relationships in the context of streaming features, where the features steam in one by one. With a Bayesian network to represent causal relationships, we propose a novel algorithm called causal discovery from streaming features (CDFSF) which consists of a two-phase scheme. In the first phase, CDFSF dynamically discovers causal relationships between each feature seen so far with an arriving feature, while in the second phase CDFSF removes the false positives of each arrived feature from its current set of direct causes and effects. To improve the efficiency of CDFSF, using the symmetry properties between parents (causes) and children (effects) in a faithful Bayesian network, we present a variant of CDFSF, S-CDFSF. Experimental results validate our algorithms in comparison with the existing algorithms of causal relationship discovery.

Associative classifiers have received considerable attention due to their easy to understand models and promising performance. However, with a high dimensional dataset, associative classifiers inevitably face two challenges: (1) how to extract a minimal set of strong predictive rules from an explosive number of generated association rules, and (2) how to deal with the highly sensitive choice of the minimal support threshold. In order to address these two challenges, we introduce causality into associative classification, and propose a new framework of causal associative classification. In this framework, we use causal Bayesian networks to bridge irrelevant and redundant features with irrelevant and redundant rules in associative classification. Without loss of prediction power, the feature space involved with the antecedent of a classification rule is reduced to the space of the direct causes, direct effects, and direct causes of the direct effects, a.k.a. the Markov blanket, of the consequent of the rule in causal Bayesian networks. The proposed framework is instantiated via baseline classifiers using emerging patterns. Experimental results show that our framework significantly reduces the model complexity while outperforming the other state-of-the-art algorithms.

In this paper, we study a new research problem of causal discovery from streaming features. A unique characteristic of streaming features is that not all features can be available before learning begins. Feature generation and selection often have to be interleaved. Managing streaming features has been extensively studied in classification, but little attention has been paid to the problem of causal discovery from streaming features. To this end, we propose a novel algorithm to solve this challenging problem, denoted as CDFSF (Causal Discovery From Streaming Features) which consists of two phases: growing and shrinking. In the growing phase, CDFSF finds candidate parents or children for each feature seen so far, while in the shrinking phase the algorithm dynamically removes false positives from the current sets of candidate parents and children. In order to improve the efficiency of CDFSF, we present S-CDFSF, a faster version of CDFSF, using two symmetry theorems. Experimental results validate our algorithms in comparison with other state-of-art algorithms of causal discovery.