Data are exploding in many fields and may exist in the streaming mode. When the generation speed of massive streaming data far exceeds the processing speed of a single node and the generated data need to be processed in real time, traditional centralized learning models are challenging in meeting the efficiency requirements. Therefore, online distributed learning models emerge. As time progresses, features may continuously emerge from various sources in a distributed and heterogeneous fashion. Therefore, we study the problem of online distributed heterogeneous streaming feature selection and propose a novel framework to address it, named DHSFS. The framework comprises two main components: sub-node streaming feature selection and global information synchronization. The sub-node component uses a dynamic strategy to select strong features, discard irrelevant ones, and cache weakly relevant features. In the global information synchronization stage, each sub-node synchronizes statistics information with the master node to adjust the global thresholds dynamically. Finally, the features selected by each sub-node are summarized and output. Experiments on 16 datasets show that the DHSFS framework has both high prediction accuracy and high efficiency of online stream feature selection.

Conditional Independence (CI) tests play an essential role in causal discovery from observational data, enabling the measurement of independence between two nodes. However, traditional CI tests ignore the imbalanced occurrence probabilities of node values, which may affect the accuracy of determining independence between nodes. To address this problem, we first introduce a new concept of the Node-imbalance phenomenon to describe the imbalance of node values in the Bayesian network data and analyze the influence of the Node-imbalance phenomenon on the traditional CI tests, then we propose a Weight-Based Conditional Independence (WCI) test to improve the accuracy of CI tests in the presence of Node-imbalance. In the experiments, we verify that WCI effectively measures the dependency between nodes in the Node-imbalance phenomenon compared with the traditional independence tests, and the state-of-the-art causal discovery algorithms reduce the number of false causal orientations through WCI.

Feature selection is an essential pre-process component in data mining that aims to select the most relevant features from the target dataset. Datasets are always dynamic in real-world applications, and features may exist in stream mode. Then, online streaming feature selection methods are proposed, which deal with streaming features arriving continuously in real-time. However, most existing algorithms prioritize high accuracy and low time-consumption but overlook the stability of the selected features. Stable feature selection results are crucial for users in practice. For instance, in the medical field, unstable feature selection results can make it challenging for experts to identify the main causative factors of a disease. Motivated by this, this article proposes a new online stable streaming feature selection method via feature aggregation named OSSFS. Specifically, inspired by the cohesive MeanShift approach, OSSFS applies an incremental aggregation strategy to partition the streaming features into multiple hyperellipsoids. Then, we incrementally update and merge these hyperellipsoids with new streaming features. Finally, we select representative features from each hyperellipsoid as the final selected feature subset. Extensive experiments are conducted on several real-world datasets to compare our new method with state-of-the-art competing algorithms in cases of stability and predictive accuracy. Experimental results indicate that OSSFS achieves optimal stability without losing prediction accuracy.

Causal feature selection has recently received increasing attention in machine learning and data mining, especially in the era of Big Data. Existing causal feature selection algorithms select unique causal features of the single class label as the optimal feature subset. However, a single class label usually has multiple classes, and it is unreasonable to select the same causal features for different classes of a single class label. To address this problem, we employ the class-specific mutual information to evaluate the causal information carried by each class of the single class label, and theoretically analyze the unique relationship between each class and the causal features. Based on this, aLabel-awareCausalFeatureSelection algorithm (LaCFS) is proposed to identifies the causal features for each class of the class label. Specifically, LaCFS uses the pairwise comparisons of class-specific mutual information and the size of class-specific mutual information values from the perspective of each class, and follows a divide-and-conquer framework to find causal features. The correctness and application condition of LaCFS are theoretically proved, and extensive experiments are conducted to demonstrate the efficiency and superiority of LaCFS compared to the state-of-the-art approaches.

Fair feature selection for classification decision tasks has recently garnered significant attention from researchers. However, existing fair feature selection algorithms fall short of providing a full explanation of the causal relationship between features and sensitive attributes, potentially impacting the accuracy of fair feature identification. To address this issue, we propose a fair causal feature selection algorithm, called FairCFS . Specifically, FairCFS constructs a localized causal graph that identifies the Markov blankets of class and sensitive variables, to block the transmission of sensitive information for selecting fair causal features. Extensive experiments on seven public real-world datasets validate that FairCFS has accuracy comparable to eight state-of-the-art feature selection algorithms while presenting more superior fairness.

Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (parents and children) of each variable in a dataset, then orient edges in the skeleton. However, existing MB or PC learning methods are often computationally expensive especially with a large-sized BN, resulting in inefficient local-to-global learning algorithms. To tackle the problem, in this article, we link feature selection with local BN structure learning and develop an efficient local-to-global learning approach using filtering feature selection. Specifically, we first analyze the rationale of the well-known Minimum-Redundancy and Maximum-Relevance (MRMR) feature selection approach for learning a PC set of a variable. Based on the analysis, we propose an efficient F2SL (feature selection-based structure learning) approach to local-to-global BN structure learning. The F2SL approach first employs the MRMR approach to learn the skeleton of a DAG, then orients edges in the skeleton. Employing independence tests or score functions for orienting edges, we instantiate the F2SL approach into two new algorithms, F2SL-c (using independence tests) and F2SL-s (using score functions). Compared to the state-of-the-art local-to-global BN learning algorithms, the experiments validated that the proposed algorithms in this article are more efficient and provide competitive structure learning quality than the compared algorithms.

Clustering ensemble often provides robust and stable results without accessing original features of data, and thus has been widely studied. The conventional clustering ensemble methods often take the full multiple base partitions as inputs and provide a consensus clustering result. However, in many real-world applications, full base partitions are hard to obtain because some data may be missing in some base partitions. To tackle this problem, in this paper, we propose a novel partial clustering ensemble method, which takes the partial multiple base partitions as inputs. In this method, we simultaneously fill the missing values in the base partitions and ensemble them by fully considering the consensus and diversity. Moreover, to address the unreliability issue in the partial data scenario, we seamlessly plug it into a self-paced learning framework. The extensive experiments on benchmark data sets demonstrate the effectiveness and efficiency of the proposed method when handling incomplete data.

Causal feature selection has received increasing attention in recent years. However, the state-of-the-art causal feature selection algorithms use the conditional independence tests, which require enumerating conditioning sets, leading to an exponential increase in computational complexity along with an increase in feature space. To address this problem, in this paper, we theoretically analyze the unique performance of causal features in mutual information, and propose a novelCausalFeatureSelection algorithm usingMutualInformation, called CFS-MI. Specifically, CFS-MI separately instantiates the pairwise comparison of mutual information in two stages to reduce computational complexity, and thus improves the efficiency on high-dimensional data. Extensive experiments on 5 benchmark Bayesian networks and 16 real-world datasets validate that CFS-MI has comparable accuracy compared to 7 state-of-the-art causal feature selection algorithms, while presenting more superior computational efficiency.

Learning partial Bayesian network (BN) structure is an interesting and challenging problem. In this challenge, it is computationally expensive to use global BN structure learning algorithms, while only one part of a BN structure is interesting, local BN structure learning algorithms are not a favourable solution either due to the issue of false edge orientation. To address the problem, this article first presents a detailed analysis of the false edge orientation issue with local BN structure learning algorithms and then proposes PSL, an efficient and accurate P artial BN S tructure L earning (PSL) algorithm. Specifically, PSL divides V-structures in a Markov blanket (MB) into two types: Type-C V-structures and Type-NC V-structures, then it starts from the given node of interest and recursively finds both types of V-structures in the MB of the current node until all edges in the partial BN structure are oriented. To further improve the efficiency of PSL, the PSL-FS algorithm is designed by incorporating F eature S election (FS) into PSL. Extensive experiments with six benchmark BNs validate the efficiency and accuracy of the proposed algorithms.

The discovery of Markov blanket (MB) for feature selection has attracted much attention in recent years, since the MB of the class attribute is the optimal feature subset for feature selection. However, almost all existing MB discovery algorithms focus on either improving computational efficiency or boosting learning accuracy, instead of both. In this article, we propose a novel MB discovery algorithm for balancing efficiency and accuracy, called BAlanced Markov Blanket (BAMB) discovery. To achieve this goal, given a class attribute of interest, BAMB finds candidate PC (parents and children) and spouses and removes false positives from the candidate MB set in one go. Specifically, once a feature is successfully added to the current PC set, BAMB finds the spouses with regard to this feature, then uses the updated PC and the spouse set to remove false positives from the current MB set. This makes the PC and spouses of the target as small as possible and thus achieves a trade-off between computational efficiency and learning accuracy. In the experiments, we first compare BAMB with 8 state-of-the-art MB discovery algorithms on 7 benchmark Bayesian networks, then we use 10 real-world datasets and compare BAMB with 12 feature selection algorithms, including 8 state-of-the-art MB discovery algorithms and 4 other well-established feature selection methods. On prediction accuracy, BAMB outperforms 12 feature selection algorithms compared. On computational efficiency, BAMB is close to the IAMB algorithm while it is much faster than the remaining seven MB discovery algorithms.