Clustering is a fundamental technique widely used for exploring the inherent data structure. Many studies indicate that an appropriate feature representation can effectively improve clustering performance. However, the existing feature representation methods are based on correlation to select or extract features, which makes it hard to deal with spurious correlations. The spurious correlations mislead the correlation-based methods to consider features that have no causal relationship as being correlative, which limits the clustering performance and feature interpretability. To tackle this issue, inspired by causal learning, we propose a new joint optimization Clustering Method with Causal Feature Embedding (CM-CaFE) , which utilizes the causality of features to learn more discriminative representation for clustering. Specifically, to eliminate spurious correlations among features, we first employ any state-of-the-art Markov blanket learning method to learn an undirected causal graph. Next, we extract the maximal fully connected causal subgraphs from the learned undirected causal graph and propose an approach to merge them to generate the causal matrix. Based on the causal matrix, we present an objective function that consists of a clustering loss term and a causal matrix fitting term to learn a causal transformation matrix. The causal transformation matrix is utilized to map the original data into a new space for clustering. Finally, we comprehensively compare the proposed method with some state-of-the-art clustering approaches on several datasets to demonstrate the effectiveness and interpretability of the proposed method.

Feature weighting aims to assign different weights to features based on their importance in machine learning tasks. In clustering tasks, the existing methods learn feature importance based on the clustering results derived from the collaborative contribution of all features, which overlooks the independent effect of each feature. In fact, there are underlying causal relationships between features and the clustering results, and the features with high causal effects are always more crucial for clustering. Therefore, we propose an enhancedFeatureWeighting method viaCausalEffect forClustering, calculating the causal effect of each feature on the clustering results for obtaining the independent contribution of each feature. Specifically, we start by identifying the causal relationships among the features and utilizing the causal relationships to generate a reasonable treatment group. Next, we compare the changes in the data distribution between the treatment and control groups to determine the causal effect of each feature. Finally, the causal effects of features are used for enhancing the clustering-driven weight learning. Moreover, we present a theory of relative order consistency in causal effect. Experimental results demonstrate that utilizing causal effect in weight learning facilitates efficient convergence and achieves superior accuracy compared to state-of-the-art clustering algorithms.

As a promising strategy to achieve generalizable graph learning tasks, graph invariant learning emphasizes identifying invariant subgraphs for stable predictions on biased unknown distribution by selecting the important edges/nodes based on their contributions to the predictive tasks (i.e., subgraph predictivity). However, the existing approaches solely relying on subgraph predictivity face a challenge: the learned invariant subgraph often contains numerous spurious nodes and shows poor connectivity, undermining the generalization power of Graph Neural Networks (GNNs). To tackle this issue, we propose a summary graph-induced Invariant Learning (SIL) model that innovatively adopts a summary graph to leverage both the subgraph connectivity and predictivity for learning strong connected and accurate invariant subgraphs. Specifically, SIL first learns a summary graph containing multiple strongly connected supernodes while maintaining structure consistency with the original graph. Second, the learned summary graph is disentangled into an invariant supernode and spurious counterparts to eliminate the interference of highly predictive edges and nodes. Finally, SIL identifies a potential invariant subgraph from the invariant supernode to accomplish generalization tasks. Additionally, we provide a theoretical analysis of the summary graph learning mechanism, guaranteeing that the learned summary graph is consistent with the original graph. Experimental results validate the effectiveness of the SIL model.

Improving the generalization performance of classifiers from data pre-processing perspective has recently received considerable attention in the machine learning community. Although many methods have been proposed in the past decades, most of them lack theoretical foundations and cannot guarantee better generalization performance of classifiers on processed datasets. To overcome this flaw, in this paper, we propose a method, which is supported by Bayesian decision theory and percolation theory, to improve generalization performance by removing Bayesian confusing samples (abbr. BCS). Specifically, for a training set, we define the samples that misclassified by the Bayesian optimal classifier as BCS and prove that a classifier trained on the training set after removing BCS can obtain better generalization performance. To find out BCS, we indicate that BCS can be identified according to the size of global homogeneous cluster, a set of samples with the same labels, based on percolation theory. Based on these analysis, we propose a method to construct global homogeneous clusters and remove BCS from the training set. Extensive experiments show that the proposed method is effective for a number of classical and state-of-the-art classifiers.

Inferring causal structures from experimentation is a challenging task in many fields. Most causal structure learning algorithms with unknown interventions are proposed to discover causal relationships over an identical variable set. However, often due to privacy, ethical, financial, and practical concerns, the variable sets observed by multiple sources or domains are not entirely identical. While a few algorithms are proposed to handle the partially overlapping variable sets, they focus on the case of known intervention targets. Therefore, to be close to the real-world environment, we consider discovering causal relationships over overlapping variable sets under the unknown intervention setting and exploring a scenario where a problem is studied across multiple domains. Here, we propose an algorithm for discovering the causal relationships over the integrated set of variables from unknown interventions, mainly handling the entangled inconsistencies caused by the incomplete observation of variables and unknown intervention targets. Specifically, we first distinguish two types of inconsistencies and then deal with respectively them by presenting some lemmas. Finally, we construct a fusion rule to combine learned structures of multiple domains, obtaining the final structures over the integrated set of variables. Theoretical analysis and experimental results on synthetic, benchmark, and real-world datasets have verified the effectiveness of the proposed algorithm.

Causal structure learning (CSL) from observational data is a crucial objective in various machine learning applications. Recent advances in CSL have focused on local-to-global learning, which offers improved efficiency and accuracy. The local-to-global CSL algorithms first learn the local skeleton of each variable in a dataset, then construct the global skeleton by combining these local skeletons, and finally orient edges to infer causality. However, data quality issues such as noise and small samples often result in the presence of problematicasymmetric edgesduring global skeleton construction, hindering the creation of a high-quality global skeleton. To address this challenge, we propose a novel local-to-global CSL algorithm with a progressive enhancement strategy and make the following novel contributions: 1) To construct an accurate global skeleton, we design a novel strategy to iteratively correctasymmetric edgesand progressively improve the accuracy of the global skeleton. 2) Based on the learned accurate global skeleton, we design an integrated global skeleton orientation strategy to infer the correct directions of edges for obtaining an accurate and reliable causal structure. Extensive experiments demonstrate that our method achieves better performance than the existing CSL methods.

In this study, we investigate a challenging problem, namely, robust domain adaptation, where data from only a single well-labeled source domain are available in the training phase. To address this problem, assuming that the causal relationships between the features and the class variable are robust across domains, we propose a novel causal autoencoder (CAE), which integrates a deep autoencoder and a causal structure learning model to learn causal representations using data from a single source domain. Specifically, a deep autoencoder model is adopted to learn the low-dimensional representations, and a causal structure learning model is designed to separate the low-dimensional representations into two groups: causal representations and task-irrelevant representations. Using three real-world datasets, the experiments have validated the effectiveness of CAE, in comparison with eleven state-of-the-art methods.