Summary Hierarchical classification learning aims to exploit the hierarchical relationship between data categories. The high dimensionality and dynamic of the data feature space are the main challenges of this research. Hierarchical feature selection uses a hierarchical structure to divide large‐scale tasks into multiple small tasks, which can more effectively improve the training speed and prediction accuracy of classification models. To present, existing online feature selection methods ignore the hierarchical structure of data. In addition, the dependency relationships in the hierarchical structure can serve as auxiliary knowledge to aid feature selection. Based on this, this paper proposes an online streaming hierarchical feature selection method based on kernelized fuzzy rough sets (OFS‐HNFRS). First, we use the prior knowledge of the hierarchical structure to divide the sample set into multiple subsets. Second, the dependency relationship of hierarchical structure is extended to kernelized fuzzy rough sets, and hierarchical category dependency based on kernelized fuzzy rough sets is defined. Finally, a new online feature selection framework is proposed, which is used to evaluate the relevance, significance, and redundancy of features. We verify the effectiveness of the proposed algorithm on six hierarchical datasets and eight flat datasets.

Label distribution learning (LDL) is a novel approach that outputs labels with varying degrees of description. To enhance the performance of LDL algorithms, researchers have developed different algorithms with mining label correlations globally, locally, and both globally and locally. However, existing LDL algorithms for mining local label correlations roughly assume that samples within a cluster share same label correlations, which may not be applicable to all samples. Moreover, existing LDL algorithms apply global and local label correlations to the same parameter matrix, which cannot fully exploit their respective advantages. To address these issues, a novel LDL method based on horizontal and vertical mining of label correlations (LDL-HVLC) is proposed in this paper. The method first encodes a unique local influence vector for each sample through the label distribution of its neighbor samples. Then, this vector is extended as additional features to assist in predicting unknown instances, and a penalty term is designed to correct wrong local influence vector (horizontal mining). Finally, to capture both local and global correlations of label, a new regularization term is constructed to constrain the global label correlations on the output results (vertical mining). Extensive experiments on real datasets demonstrate that the proposed method effectively solves the label distribution problem and outperforms the current state-of-the-art methods.

Abstract Multi‐label feature selection eliminates irrelevant and redundant features, and then improves the performance of multi‐label classification models. Most multi‐label feature selection algorithms assume that the training set contains logical labels, which means that labels are equally important for instances. However, in practical applications, there are different importances with respect to labels. To solve the problem, a multi‐label feature selection method based on relative entropy and fuzzy neighborhood mutual discriminant index is proposed. Firstly, logical labels are converted to label distribution through label enhancement. Secondly, the neighborhood and relative entropy are introduced into the label distribution, the label neighborhood similarity matrix is constructed to describe the similarity of samples under label space. Finally, the fuzzy neighborhood mutual discrimination index is used to combine the candidate features with the label neighborhood similarity matrix, which is used to judge the distinguishing ability of the candidate features. Comprehensive experiment of eight multi‐label datasets shows that the proposed algorithm has better classification performance than other compared algorithms.

Abstract In hierarchical classification learning, the feature space of data has high dimensionality and is unknown with emergent features. To solve the above problems, we propose an online hierarchical feature selection algorithm based on adaptive ReliefF. Firstly, ReliefF is adaptively improved via using the density information of instances around the target sample, making it unnecessary to prespecify parameters. Secondly, the hierarchical relationship between classes is used, and a new method for calculating the feature weight of hierarchical data is defined. Then, an online correlation analysis method based on feature interaction is designed. Finally, the adaptive ReliefF algorithm is improved based on feature redundancy, and the feature weight is scaled by the correlation between features in order to achieve the dynamic updating of feature redundancy. A large number of experiments verify the effectiveness of the proposed algorithm.

Abstract Hierarchical classification learning is a hot research topic in machine learning and data mining domains, and many feature selection algorithms with category hierarchy have been proposed. However, existing algorithms assume that the feature space of data is completely obtained in advance, and ignore its uncertainty and dynamicity. To address these problems, we propose an online streaming feature selection framework with a hierarchical structure to solve the above two problems simultaneously. First, we apply the hierarchical relationship between nodes in a hierarchical structure to the Relief algorithm, so that it can be used to compute the weights of dynamic features. Second, we dynamically select important features for each internal node via comparing the magnitude of the weights of features on these nodes with their parent and sibling nodes. In addition, we perform redundancy analysis of features by calculating the covariance between features to obtain a superior online feature subset for each internal node. Finally, the proposed algorithm is compared with six online streaming feature selection methods on six hierarchical data sets, and experimental results shows that the classification performance of the proposed algorithm is effective.

Abstract Feature selection is a nontrivial preprocessing technique in many practical application domains. There are three key challenges with respect to real‐world data. Firstly, the dimensionality of data keeps growing and will achieve hundreds of millions. Secondly, the data has the characteristic of high‐dimensional and small‐size. Thirdly, practical applications need to process each feature in an online manner. However, most of the previous methods only pay much attention to solving the challenges of high dimensionality and online stream. To address all issues above, we propose OFSI, in this article, an nline streaming eature election based on feature nteraction method for feature selection. OFSI can effectively select the streaming features that are strongly related to each other in high‐dimensional and small‐size data, via using feature interaction. Furthermore, to address upcoming features that arrive by groups, we present a new group‐OFSI algorithm for online group feature selection. An extensive experiment using a series of benchmark data sets shows that the proposed two algorithms, OFSI and group‐OFSI, outperform six state‐of‐the‐art online streaming feature selection methods.