Hashing-based methods have gained widespread attention in cross-modal retrieval due to their significant potential for spatiotemporal efficiency and query accuracy, driving significant progress. However, most existing algorithms extract similarity information solely from labels, overlooking the consistent semantic similarities embedded in features. Additionally, some algorithms underutilize label semantics, neglecting inherent category relationships and their interplay with feature semantics, which compromises algorithmic effectiveness. Furthermore, current methods often suffer from weak discriminative power and semantic misalignment when embedding similarity information into hash codes, leading to learning biases. To address these short-comings, a Multi-similarity Preserving Semantic-aware Hashing (MPSH) algorithm is proposed. MPSH first learns a shared semantic space from heterogeneous features to generate dynamic feature fusion similarity. Subsequently, it explores label distributions to construct category vectors, which are collaboratively associated with isomorphic feature representations to form a semantic-aware space. Finally, manifold embedding is employed to effectively preserve multi-similarity and data distribution, yielding semantically rich hash representations. Moreover, a non-relaxed discrete optimization algorithm is introduced to accelerate hash code iteration. Extensive experiments on multiple benchmark datasets, comparing MPSH with leading algorithms, demonstrate its superior efficiency and performance.

Unsupervised anomaly detection has attracted considerable attention in complex data environments due to its independence from costly labeled data. Among various approaches, subspace sampling-based ensemble methods, such as IForest, have been widely adopted for their simplicity and computational efficiency. However, these methods typically operate under single granularity, which limits the diversity of subspaces and hinders the ability to capture hierarchical structures and complex patterns in the data. Moreover, they often overlook uncertainty information such as fuzziness among samples, which constrains their capacity to model complex relationships. To address these limitations, this paper proposes a method called Granular-Ball Subspace-based Fuzzy Neighborhood Anomaly Detector (GSFAD). The proposed method integrates granular-ball subspace ensemble learning with a fuzzy computing framework, achieving a balance between computational efficiency and the ability to model multi-granularity fuzzy structures. Specifically, the algorithm begins by performing multi-granularity aggregation with granular-balls to cover the origin data. Then, regions potentially containing anomalies are filtered out based on granular-ball characteristics before sampling. Building on this, multiple granular-ball subspaces are constructed via repeated sampling, and fuzzy relations between granular-balls are computed within each subspace. Finally, the anomaly score of each sample is assessed by fusing the fuzzy neighborhood information across all subspaces. Experimental results on 20 benchmark datasets demonstrate that GSFAD consistently outperforms existing subspace sampling methods that operate under a single granularity. In addition, it achieves superior performance compared to 15 state-of-the-art anomaly detection techniques. The code is publicly available online athttps://github.com/Caspar-lab/GSFAD.

Representation learning serves as a critical bridge between human cognition and the data world, constituting an essential component of machine learning architectures where comprehensiveness and flexibility are paramount. However, existing multi-view feature selection methods are constrained by the raw-scale representations, neglecting comprehensive depth-breadth integration and flexible adaptability. This paper presents multi-scale fuzzy fusion-based heterogeneous granular-ball flexible representation learning for multi-view feature selection (MFHGBFR). A comprehensive multi-view, multi-scale analytical foundation is established to promote depth-breadth representation learning, where views represent breadth and scales represent depth in human cognition. Heterogeneous granular ball-based flexible representations are adaptively developed via multi-scale fuzzy fusion, effectively capturing intricate data manifolds and implicit fuzzy patterns across multi-granularity spaces. Intricate data manifolds are fitted with heterogeneous rather than conventional homogeneous structures to improve the adaptability of representations' multi-granularity. Implicit fuzzy patterns are extracted with fuzzy approximation operators to mitigate fuzziness and uncertainty in a multi-granularity space. For the first time, granular-ball representation learning and feature selection are adaptively optimized in a unified one-step framework, rather than the conventional two-step frameworks, for flexible adaptability. An effective optimization algorithm with proven convergence is derived. Through the learned high-quality representations, the MFHGBFR manifests superior performance, robustness, and efficiency. Comparative experiments against contemporary state-of-the-art algorithms substantiate these advantages quantitatively and qualitatively.

With the successful application of granular computing in anomaly detection, a variety of tools including fuzzy information entropy can achieve superior detection results. However, fuzzy information entropy calculates fuzzy similarity through a global strategy, ignoring the local information in the data. To address this deficiency, this paper constructs a fuzzy$k$NN entropy theory and applies it to identify anomalies. Firstly, fuzzy$k$-similarity and fuzzy$k$NN are defined, and$k$NN entropy theory and the related information-theoretic metrics are proposed. Then, the relevant definitions and propositions of fuzzy$k$NN entropy, fuzzy$k$-joint entropy, fuzzy$k$-conditional information entropy, as well as fuzzy$k$-mutual information are elaborated. Based on the proposed theory, an anomaly detection model is constructed. At first, the fuzzy$k$-similarity relation matrix is constructed based on the fuzzy$k$-similarity in the proposed theory, and the relative fuzzy$k$NN entropy is calculated. Based on the relative fuzzy$k$NN entropy, the fuzzy$k$-relation anomaly degree is defined to characterize the anomaly intensity of fuzzy$k$NN information granules. Then, the anomaly factor based on fuzzy$k$NN entropy is built to represent the anomaly degree of data objects. Finally, the corresponding Fuzzy$k$NN Entropy-based Anomaly Detection algorithm (F$k$EAD) is designed. Comparative experiments are conducted with 11 state-of-the-art anomaly detection methods on thirty public datasets. The results reveal that the proposed method achieves better performance.

Unsupervised feature selection aims to enhance the quality of unlabeled data, thereby improving the performance of subsequent unsupervised learning models. However, most of the existing unsupervised feature selection methods rely on single-granularity modeling, which reduces the expressive capability of data to some extent. In addition, the existing studies are generally based on a forward greedy feature selection strategy, which tends to fall into a local optimum. To address these issues, this paper proposes a novel unsupervised feature selection method for handling hybrid data, called unsupervised feature selection method using fuzzy graph momentum random walk in bi-level granular-ball knowledge space. Specifically, a Bi-level Granular-ball Knowledge Space (BGKS) is first constructed by combining fine granularity and coarse granularity representations through a hybrid Gaussian kernel function. Then, a multi-granularity fuzzy graph is built on the BGKS using upper and lower fuzzy approximation operators. Based on this graph, a Momentum Random Walk (MRW) mechanism is introduced to design the Fuzzy Graph Momentum Random Walk (FGMRW) model. Finally, an iterative unsupervised feature selection algorithm is developed. Extensive experiments on 20 public datasets demonstrate that, compared with existing algorithms, the proposed method is able to maintain or even improve clustering performance while selecting fewer features, thus achieving superior overall performance. The source code of this work is publicly available athttps://github.com/HongtaoGao-code/FGMRW-UFS.

In information systems lacking decision-making information, effectively leveraging fuzzy rough sets for outlier detection in complex data is challenging, especially in capturing inherent uncertainty and multi-granularity characteristics to construct discriminative outlier scores. However, existing fuzzy rough sets-based outlier detection methods often suffer from three key limitations: (1) Local data distributions are often ignored when calculating fuzzy relation matrices, resulting in inaccurate fuzzy similarity representations; (2) Use of all objects in fuzzy upper and lower approximations can weaken noise resistance and increase computational complexity; (3) Single-granularity data processing reduces efficiency and may fail to capture the multi-granularity nature of data, thereby limiting the adaptability of these methods in complex data environments. To address these issues, we propose to fusesNatural neighbor fuzzy approximations withGranular-ball representation forOutlierDetection (NGOD), which integrates the multi-granularity granular-ball representation and fuzzy rough sets to improve the effectiveness and robustness of unsupervised outlier detection. Specifically, we first define a local distribution-aware fuzzy relation, enabling more discriminative similarity calculations between samples. To improve the effectiveness and robustness of fuzzy upper and lower approximations, we propose a multi-granularity natural neighbor fuzzy approximation model, which effectively utilizes the inherent uncertainty and local abnormal information of data in approximations. Moreover, by introducing natural neighbors, NGOD can adaptively capture local abnormal information in the data without setting neighborhoods manually. Finally, the outlier factors of each sample are calculated in NGOD to measure their outlier degrees. Extensive experiments on diverse datasets demonstrate that NGOD outperforms state-of-the-art methods, validating its superior performance and adaptability. The NGOD code and associated datasets are publicly available athttps://github.com/Mxeron/NGOD.

Multi-view unsupervised feature selection (MV-UFS) has recently gained significant attention as an effective technique for handling high-dimensional multi-view data. Despite recent progress, three flaws are present in most existing MV-UFS methods. First, they predominantly pay attention to the feature dimensionality while neglecting sample size considerations, typically resulting in cubic time complexity concerning sample size (i.e.$\mathcal {O}(n^{3})$) that hinders them in real-world applications. Second, existing methods primarily focus on the relationships between pairwise views, failing to leverage the complex interactions among multiple views effectively. Third, they generally overlook crucial graph semantic information during graph fusion. An ingenious MV-UFS method is presented in this paper, namedJoint Information Interaction and Semantic Fusion for Multi-View Unsupervised Feature Selection (JIISF), to overcome these limitations. Specifically, JIISF devises a multi-view double reduction learning framework that projects original data into a low-dimensional embedding space and dynamically picks out representative anchors, eliminating redundant information while constructing compact view-specific anchor graphs. Meanwhile, JIISF presents a cross-view information interaction scheme, which enables comprehensive information interaction among multiple views rather than being limited to pairwise views. A semantic alignment fusion strategy is presented to learn a diverse, consistent anchor graph with an explicit semantic alignment module, which can preserve diversity while maintaining consistency. Finally, an alternating optimization algorithm is devised to resolve the proposed JIISF, and experimental results on real-life datasets validate its advantage compared with some state-of-the-art UFS and MV-UFS methods.

Hashing techniques have attracted significant attention in cross-modal retrieval tasks due to their high query efficiency and low storage overhead. However, some methods map heterogeneous features and labels into the same semantic space, ignoring their complementary relationship, which leads to semantic loss. When leveraging pairwise similarity relationships, the affinity matrix incurs high computational costs, and the asymmetric proxy strategy lacks sufficient semantic degrading retrieval performance. Additionally, optimizing binary hash codes remains a challenging task. A novel algorithm, Multi-granularity Semantic Complementarity Fusion Hashing (MSCFH), is proposed to address these issues. First, it learns multi-granularity semantic representations from features and labels, then complements their semantics through a complementarity fusion strategy. Subsequently, an asymmetric semantic learning framework is constructed, guided by label similarities, to learn discriminative hash codes. Furthermore, an efficient discrete optimization algorithm is introduced to update the hash codes. Extensive experiments on three benchmark datasets show that the proposed MSCFH demonstrates the best retrieval accuracy and highly competitive training efficiency compared to state-of-the-art approaches.

Graph-based feature selection methods can capture the most discriminative subset of features in multi-label data and have made great strides in recent years. However, most existing methods have three drawbacks: (1) Only binary pairwise relationship is difficult to capture latent structural information in high-dimensional data fully. (2) Using only one distance metric cannot accurately mine the similarity relationship between instances. (3) Ignore the problem that some of the non-obvious labels are missing when labels are abundant. Therefore, this paper proposes a graph diffusion with dual-distance metrics for missing multi-label feature selection named GDMMFS. GDMMFS uses both Euclidean and cosine distances to explore instance correlations and then uses diffusion techniques to transform binary relationship into quaternary relationship, thus obtaining a stable similarity graph. Moreover, GDMMFS utilizes adaptive graphs to learn latent structure information to improve the model’s immunity to interference. Finally, the ℓ 2 , 1 -norm is used as a sparse constraint to guide the weight matrix to capture the most discriminative subset of features. To further consider the possibility of missing label data, GDMMFS constructed a logic matrix to guide the numerical labels in recovering the missing information. Comparison experiments with state-of-the-art related algorithms demonstrate the superiority of the proposed algorithm, and ablation experiments demonstrate the effectiveness of graph diffusion with two-distance metrics.

Air quality modeling and forecasting has become a key problem in environmental protection. The existing prediction models typically require large‐scale and high‐quality historical data to achieve better performance. However, insufficient data volume and significant differences between data distribution across different regions will definitely reduce the effectiveness of the model reuse. To address the above issues, we propose a novel hybrid recurrent network based on domain adversarial transfer to achieve a stronger generalization ability when training air quality data from multisource domains. The proposed model mainly consists of three fundamental modules, i.e., feature extractor, regression predictor, and domain classifier. One‐dimensional convolutional neural networks (1D‐CNNs) are used to extract temporal feature of data from source and target stations. Bi‐directional gated recurrent unit (bi‐GRU) and bi‐directional long short‐term memory (bi‐LSTM) are utilized to learn temporal dependencies pattern of multivariate time series data. Two adversarial transfer strategies are employed to ensure that our model is capable of finding domain invariant representations automatically. Experiments with different number of source domains are conducted to demonstrate the effectiveness of the proposed domain transfer strategies. The experimental results also show that our composite model has superior performance for forecasting air quality in various regions. As further evidence, the adversarial training method could promote the positive transfer and alleviate the negative effect of irrelevant source data. Besides, our model exhibits preferable generalization capability as more robust prediction results are achieved on both unseen target domains and original source domains.

Feature selection (FS) is essential in machine learning and data mining as it makes handling high-dimensional data more efficient and reliable. More attention has been paid to unsupervised feature selection (UFS) due to the extra resources required to obtain labels for data in the real world. Most of the existing embedded UFS utilize a sparse projection matrix for FS. However, this may introduce additional regularization terms, and it is difficult to control the sparsity of the projection matrix well. Moreover, such methods may seriously destroy the original feature structure in the embedding space. Instead, avoiding projecting the original data into the low-dimensional embedding space and identifying features directly from the raw features that perform well in the process of making the data show a distinct cluster structure is a feasible solution. Inspired by this, this paper proposes a model called A General Adaptive Unsupervised Feature Selection with Auto-weighting (GAWFS), which utilizes two techniques, non-negative matrix factorization, and adaptive graph learning, to simulate the process of dividing data into clusters, and identifies the features that are most discriminative in the clustering process by a feature weighting matrix Θ. Since the weighting matrix is sparse, it also plays the role of FS or a filter. Finally, experiments comparing GAWFS with several state-of-the-art UFS methods on synthetic datasets and real-world datasets are conducted, and the results demonstrate the superiority of the GAWFS.

Multiview data possess different discriminability in different views, which is challenging to catch but crucial for a feature selection model. Multiscale information, which represents vertical exploration in each view, is vital for further mining traits implied in multiview data. However, most existing studies neglect these beneficial multi-granulation characteristics. This study first embeds the multiscale information into the sparse learning framework for multiview feature selection. A class-specific discriminability and multiscale information-based multiview feature selection (CDMIMFS) method is proposed. It explores the fuzzy and uncertain class-specific discriminability which is inherently discrepant in different views by the fuzzy rough set theory. It relaxes the over strict requirement for complete consistency in multiscale information systems to make a trade-off, which further enhances discriminative feature selection. An effective iteration algorithm is proposed to solve the optimization. Both the theoretical proof and experimental demonstration of convergence are provided. Comprehensive experiments are conducted on the CDMIMFS compared with state-of-the-art algorithms. Results on different evaluation metrics exhibit the advantages of the proposed method.

Outlier detection, crucial for identifying unusual patterns with significant implications across numerous applications, has drawn considerable research interest. Existing semisupervised methods typically treat data as purely numerical and in a deterministic manner, thereby neglecting the heterogeneity and uncertainty inherent in complex, real-world datasets. This article introduces a label-informed outlier detection method for heterogeneous data based on Granular Computing and Fuzzy Sets, namely Granule Density-based Outlier Factor (GDOF). Specifically, GDOF first employs label-informed fuzzy granulation to effectively represent various data types and develops granule density for precise density estimation. Subsequently, granule densities from individual attributes are integrated for outlier scoring by assessing attribute relevance with a limited number of labeled outliers. Experimental results on various real-world datasets show that GDOF stands out in detecting outliers in heterogeneous data with a minimal number of labeled outliers. The integration of Fuzzy Sets and Granular Computing in GDOF offers a practical framework for outlier detection in complex and diverse data types.

The inherent characteristics involved in data can be mined from multi-scale information systems by extracting information from different value levels of features. In real applications, noise data and irrelevant or redundant features affect the generality of learning models. Therefore, keeping meaningful features and avoiding the effect of noise is essential for feature selection in a multi-scale information system. In bireduct, multi-scale granulation can be used to characterize the importance and correlation of features at different scales. However, little work has taken the distribution of multi-scale data into account when granulating it. In addition, these approaches focus on solving the task of multi-scale data reduction only from the dimension perspective. To this end, a fuzzy-rough bireduct with supervised multi-scale granulation (FrBSmg) is proposed. First, the supervised multi-scale fuzzy granulation based on data distribution is constructed. Then, scaled uncertainty measures are defined to describe the fuzzy relevance of each feature. Furthermore, the global and local distributions of a sample are characterized simultaneously based on the positive region, which can reflect the degree of a sample belonging to some class, and the supervised fuzzy similarity relation can describe the degree of a sample belonging to its class. A strategy of Feature-Correlated Selection and Sample-Noisy Removal is devised for bireduct. Finally, the experimental results on twenty-one public datasets show the effectiveness of FrBSmg.

In reality, the laborious nature of label annotation leads to the widespread existence of limited labeled data. Moreover, multiscale data have received widespread attention due to its rich knowledge representation. However, current research on multiscale data primarily focuses on supervised learning environments, while semisupervised feature selection for multiscale data with limited labels remains inadequately explored. Meanwhile, existing studies on multiscale data often emphasize selecting optimal scales and further conducting feature selection, but this strategy may result in losing potentially valuable information from other scales. To overcome these limitations, by adopting a multiscale fuzzy information fusion mechanism, this article proposes two new semisupervised feature selection approaches for multiscale data with limited labels from both global and local perspectives. Initially, by fusing fuzzy information at various scales, a label learning method is proposed to convert the decision of missing labels into a fuzzy decision of label distribution. Subsequently, label-distributed multiscale fuzzy global and local rough sets are constructed from global and local perspectives, respectively. Based on the two models, two semisupervised feature selection algorithms are developed based on global and local multiscale fuzzy information fusion. Experimental results demonstrate that, compared to other advanced feature selection algorithms, the two proposed algorithms can effectively handle multiscale data with limited labels and exhibit superior classification performance, computational efficiency, and robustness.

Outlier Detection (OD) has attracted extensive research due to its application in many fields. The idea of neighborhood computing is one of the widely used methods in outlier analysis. Nevertheless, these methods mainly use certainty strategies to model outlier detection, so they cannot effectively handle the fuzzy information in the dataset. Moreover, they mainly focus on dealing with outlier detection in numerical data and cannot effectively find outliers in mixed-attribute data. Fuzzy information granulation theory is an effective granular computing model that allows objects to belong to a set to a certain extent (i.e., membership degree), which makes it possible to better handle uncertainty problems such as fuzziness. In this work, we propose an outlier detection model based on fuzzy neighborhoods. First, a hybrid fuzzy similarity is constructed to granulate the set of objects to form fuzzy information granules. Second, the fuzzy$k$-nearest neighbor is defined to describe the fuzzy local information. Then, the fuzzy neighborhood density is defined to indicate the degree of aggregation of each object. The smaller the fuzzy neighborhood density of an object, the more likely it is to be an outlier. Based on this idea, the fuzzy neighborhood deviation degree is defined to quantify the degree of outliers of objects. Finally, the fuzzy deviation degree on the set of conditional attributes is constructed to indicate the outlier scores of objects. Experimental comparisons with state-of-the-art methods show that the proposed method has a significant improvement on the AUC index and applies to three types of data.

Traditional multilabel feature selection (MFS) typically relies on pre-computing global information within the feature space. However, in real-world applications, features are dynamically generated and continuously arrive over time, known as streaming features, rendering many existing approaches ineffective. Some MFS methods for streaming features have been developed, several challenges persist: (1) Previous research often uses certain strategies to model streaming feature evaluation, failing to process fuzzy information effectively; (2) The maximum correlation between features and class is emphasized, while inter-class separability is ignored, leading to inaccurate feature evaluation; (3) The continuous influx of streaming features brings the dynamics and unknowns to data distribution, has been largely overlooked in previous work; (4) Streaming feature selection requires immediate feedback on newly arriving features, posing challenges to the algorithm's real-time responsiveness. Motivated by these observations, this paper introduces a novel online MFS strategy for streaming features. First, the weighted manifold distance is designed, and the fuzzy manifold similarity learning strategy is formalized to analyze the instance relationships of unknown distribution. Second, the fuzzy manifold intra-class correlation and inter-class separability are devised to quantify feature discriminability. Finally, a novel multilabel streaming feature analysis framework is established, with feature discriminability as the guiding factor. Incoming features are categorized as weakly relevant, strongly relevant, or redundant, culminating in generating a reliable feature selection subset. Extensive experiments on fifteen public datasets demonstrate that our algorithm achieves competitive performance compared to nine state-of-the-art offline and online algorithms.

Robust knowledge acquisition approaches are one of the research hotspots in data mining. The fuzzy dominance rough sets (FDRS) model is an important knowledge acquisition tool for ordinal classification tasks. However, it has been proved in practice that this model usually performs poorly fault tolerance, and only one noisy sample can cause huge interference in acquiring knowledge. Attribute reduction is one of the nontrivial applications of the FDRS. At present, most attribute reduction methods for ordinal classification tasks mainly focus on the dependence of decision to attributes, while ignoring the information provided by the separability of classes and the correlation of attributes for ordinal classification. In view of these two issues, this article first proposes a robust FDRS model with filterable noise samples. Then, the class-separability and attribute-correlation are explored in robust fuzzy dominance rough approximation space, and corresponding attribute evaluation index is designed. Finally, an attribute reduction algorithm is designed to select the attribute subset with the highest classification performance. The experimental results show that the proposed algorithm has better robustness and classification performance.

Anomaly detection is a significant area of discovering knowledge that has shown success in the areas of fraud detection, cyber security, and medical diagnostics. The kernelized fuzzy-rough set is a key extension model in rough computing. It inherits the advantages of kernel function and fuzzy-rough set and can handle uncertain information in data more effectively. However, existing kernelized fuzzy-rough set models construct upper and lower approximation sets mainly from decision attribute data which are not available to unlabeled data. In addition, to the best of our knowledge, studies related to the use of kernelized fuzzy upper and lower approximation sets for constructing effective anomaly detection models have not yet been reported. Based on these observations, this paper constructs a kernelized fuzzy-rough set model for unlabeled data and proposes a kernelized fuzzy-rough anomaly detection (KFRAD) method. Specifically, we first optimize the kernel function to compute the fuzzy relation matrix of attribute subsets to construct the fuzzy approximation space. Then, the definition of kernelized fuzzy approximation accuracy for unlabeled data is given. Then, the granule anomaly extent is determined based on the kernelized fuzzy approximation accuracy. Finally, the granule anomaly extents and the corresponding weight values are integrated to compute the kernelized fuzzy-rough anomaly scores of the objects. On the basis of the above detection ideas, we design the KFRAD algorithm and experimentally compare it with mainstream detection algorithms. The analysis results show that the proposed algorithm has better performance. The code is publicly available online athttps://github.com/wwwuyan/KFRAD.

Feature extraction is a fundamental and challenging task in machine learning, which aims at extracting a subset of significant and discriminant features from raw data for various downstream tasks. The extraction process involves mapping the original data into a space with lower dimensions while preserving the desirable information. However, the data often has hidden manifold structures, which contain neighbor sample information within the same class. Most existing methods that extract data features without considering the potentially significant manifold structures would result in poorly discriminative features. To address these challenges, we propose a novel geodesic fuzzy rough set model (GFRS) to capture those complex manifold structures embed in the data. Given GFRS, we further design a discriminant feature extraction algorithm based on graph embedding to enhance the discriminative performance of the extracted features. Extensive experiments on fourteen real-world datasets and visualization results on modified national institute of standards and technology (MNIST) digits demonstrate the effectiveness of the proposed algorithm and its superiority over baselines methods.

Feature selection, also known as attribute reduction, plays a crucial role in machine learning and data mining tasks. Rough set theory-based feature selection methods have gained popularity due to their ability to handle imprecise and inconsistent data, ease of implementation, and generation of highly interpretable results. However, these methods still suffer from high computational cost when dealing with large-scale datasets with high dimensions. To overcome this shortcoming, we propose a fast attribute reduction method based on inconsistent equivalence classes. The presented method can accelerate those attribute reduction algorithms whose importance measures used can be computed using only inconsistent equivalence classes. Our proposed method improves attribute reduction efficiency through three key aspects: 1) transforming the original dataset into an equivalently simplified version with fewer samples, 2) accelerating the computation of core attributes, and 3) expediting the forward selection process by removing redundant objects and attributes. Experimental results demonstrate the high computational efficiency of our proposed method.

Feature selection has been studied by many researchers using information theory to select the most informative features. Up to now, however, little attention has been paid to the interactivity and complementarity between features and their relationships. In addition, most of the approaches do not cope well with fuzzy and uncertain data and are not adaptable to the distribution characteristics of data. Therefore, to make up for these two deficiencies, a novel interactive and complementary feature selection approach based on fuzzy multineighborhood rough set model (ICFS_FmNRS) is proposed. First, fuzzy multineighborhood granules are constructed to better adapt to the data distribution. Second, feature multicorrelations (i.e., relevancy, redundancy, interactivity, and complementarity) are considered and defined comprehensively using fuzzy multigranularity uncertainty measures. Next, the features with interactivity and complementarity are mined by the forward iterative selection strategy. Finally, compared with the benchmark approaches on several datasets, the experimental results show that ICFS_FmNRS effectively improves the classification performance of feature subsets while reducing the dimension of feature space.

Feature selection, as an effective dimensionality reduction technique, is favored in preprocessing data. However, most existing algorithms are solely liable for labeled or unlabeled data, whereas a limited portion of real-world data is annotated with labels. In this article, we therefore propose a novel scheme named SemiFREE, i.e., semisupervised feature selection with fuzzy relevance and redundancy. First, both labeled and unlabeled samples are assigned with fuzzy decisions that allow class membership to naturally express the fuzziness or uncertainty in data labeling. Second, sample similarities in feature space and fuzzy decision are captured to induce fuzzy information measures for redefining the feature relevance and redundancy. Finally, adhering to the principle of relevance-maximization and redundancy-minimization, SemiFREE leverages the forward sequential searching strategy to identify qualified features progressively. Extensive experiments demonstrate the superiority of SemiFREE in the presence of partially labeled data against some other well-established feature selection algorithms.

Feature selection methods with antinoise performance are effective dimensionality reduction methods for classification tasks with noise. However, there are few studies on robust feature selection methods for monotonic classification tasks. The fuzzy dominance rough set (FDRS) model is a nontrivial knowledge acquisition tool, which is widely used in feature selection of monotonic classification tasks. Nonetheless, this model has been proved in practice to be generally poorly fault-tolerance, and only one noisy sample can cause huge interference in acquiring knowledge. In view of these two issues, this article first designs an adaptive$K$-nearest neighbors strategy to calculate the density of samples. The noisy samples are identified according to their densities, and then an active antinoise FDRS model is proposed. Then, in the active antinoise fuzzy dominance rough approximation space, the class-separability is evaluated by the approximation operators of the proposed model, and the feature-redundancy is evaluated by the fuzzy ranking conditional mutual information. On this basis, a feature evaluation index is designed comprehensively considering class-separability and feature-redundancy. Finally, a feature selection algorithm is designed to select the feature subset with the highest classification performance. The experimental results show that the proposed algorithm has better robustness and classification performance.

Most extant feature selection works neglect interactive features in the form of groups, leading to the omission of some important discriminative information. Moreover, the prevalence of data with uncertainty, fuzziness, and noise poses a certain obstacle to feature selection. Driven by these two issues, a Feature Grouping and Selection approach in Robust Fuzzy Rough Approximation Space using graph theory (FGS-RFRAS) is proposed in this study. First, a robust fuzzy rough approximation space is constructed by a neighborhood adaptive$\beta$-precision fuzzy rough set model to enhance the robustness and antinoise ability of the fuzzy rough set model. Second, uncertainty measures in robust fuzzy rough approximation space are defined to analyze the interactivity and redundancy of pairwise features on graph structure. Then, a strategy ofInteractive Retainment, Weakly Correlated Removal, and Max-Dependent Selectionis devised to guide feature grouping and selection. Experiments are performed on 21 datasets to evaluate the performance of FGS-RFRAS and demonstrate its significance. The robustness test indicates that it is antinoise for mislabeling.

In practice, it is common that there will be the same decision results under different scale conditions. Therefore, knowledge representation based on a single scale feature framework is far from meeting the needs of practical applications. Based on this, multiscale data have received extensive attention. Feature selection is an important application of fuzzy multigranularity data analysis model. The existing multiscale fuzzy granulation-based feature selection methods remove redundant or irrelevant features by selecting the optimal scale. However, this will lose the information corresponding to the remaining scale fuzzy granules, which will affect the classification results or learning tasks. Inspired by this, multiscale fuzzy entropy is defined to fuse the granule information at different scales, and applied to feature selection. First, the feature with maximum multiscale fuzzy mutual information is first selected. Then, the most significant features are gradually selected by evaluation metric that simultaneously considers the redundancy, relevance, and complementary. A multiscale fuzzy entropy-based feature selection algorithm by means of this evaluation index is further designed. Finally, the proposed method is compared with some state-of-the-art methods. The experimental results show that the proposed algorithm has higher reduction efficiency than the comparison algorithms.

Fuzzy rough set theory can model uncertainty in data and has been applied to feature selection for machine learning tasks. The existence of noise in data is one of the reasons for data uncertainty. However, most classical fuzzy rough set models are often sensitive to the noise in data, which somewhat degrades their applicability to process uncertainty of data. Furthermore, a robust feature evaluation function is nontrivial in a fuzzy rough set model as a nonoptimal feature subsets may be selected due to the perturbations from redundant features. In this article, we delve into local density and indispensable features for fuzzy rough feature selection to address these challenges. We first propose a local density-based fuzzy rough set (LDFRS) model to tackle noisy data. Mutual information is then plugged into the proposed LDFRS model to evaluate uncertainty in data. A joint feature evaluation function on the indispensability and relevance of features is constructed to evaluate the significance of features. On this basis, a fuzzy rough feature selection algorithm is built upon the LDFRS model. Experimental results using four typical classifiers demonstrate the robustness and effectiveness of the proposed model including our feature selection algorithm and its superiority against baseline methods.

The performance of multilabel learning depends heavily on the quality of the input features. A mass of irrelevant and redundant features may seriously affect the performance of multilabel learning, and feature selection is an effective technique to solve this problem. However, most multilabel feature selection methods mainly emphasize removing these useless features, and the exploration of feature interaction is ignored. Moreover, the widespread existence of real-world data with uncertainty, ambiguity, and noise limits the performance of feature selection. To this end, our work is dedicated to designing an efficient and robust multilabel feature selection scheme. First, the distribution character of multilabel data is analyzed to generate robust fuzzy multineighborhood granules. By exploring the classification information implied in the data under the granularity structure, a robust multilabel$k$-nearest neighbor fuzzy rough set model is constructed, and the concept of fuzzy dependency is studied. Second, a series of fuzzy multineighborhood uncertainty measures in$k$-nearest neighbor fuzzy rough approximation spaces are studied to analyze the correlations of feature pairs, including interactivity. Third, by investigating the uncertainty measure between feature and label, between features, multilabel data is modeled as a complete weighted graph. Then, these vertices are assessed iteratively to guide the assignment of feature weights. Finally, a graph structure-based robust multilabel feature selection algorithm (GRMFS) is designed. The experiments are conducted on 15 multilabel datasets. The results verify the superior performance of GRMFS as compared with nine representative feature selection methods.

Feature selection refers to choose an optimal non-redundant feature subset with minimal degradation of learning performance and maximal avoidance of data overfitting. The appearance of large data explosion leads to the sequential execution of algorithms are extremely time-consuming, which necessitates the scalable parallelization of algorithms by efficiently exploiting the distributed computational capabilities. In this paper, we present parallel feature selection algorithms underpinned by a rough hypercuboid approach in order to scale for the growing data volumes. Metrics in terms of rough hypercuboid are highly suitable to parallel distributed processing, and fits well with the Apache Spark cluster computing paradigm. Two data parallelism strategies, namely, vertical partitioning and horizontal partitioning, are implemented respectively to decompose the data into concurrent iterative computing streams. Experimental results on representative datasets show that our algorithms significantly faster than its original sequential counterpart while guaranteeing the quality of the results. Furthermore, the proposed algorithms are perfectly capable of exploiting the distributed-memory clusters to accomplish the computation task that fails on a single node due to the memory constraints. Parallel scalability and extensibility analysis have confirmed that our parallelization extends well to process massive amount of data and can scales well with the increase of computational nodes.

The selection of prominent features for building more compact and efficient models is an important data preprocessing task in the field of data mining. The rough hypercuboid approach is an emerging technique that can be applied to eliminate irrelevant and redundant features, especially for the inexactness problem in approximate numerical classification. By integrating the meta-heuristic-based evolutionary search technique, a novel global search method for numerical feature selection is proposed in this article based on the hybridization of the rough hypercuboid approach and binary particle swarm optimization (BPSO) algorithm, namely RH-BPSO. To further alleviate the issue of high computational cost when processing large-scale datasets, parallelization approaches for calculating the hybrid feature evaluation criteria are presented by decomposing and recombining hypercuboid equivalence partition matrix via horizontal data partitioning. A distributed meta-heuristic optimized rough hypercuboid feature selection (DiRH-BPSO) algorithm is thus developed and embedded in the Apache Spark cloud computing model. Extensive experimental results indicate that RH-BPSO is promising and can significantly outperform the other representative feature selection algorithms in terms of classification accuracy, the cardinality of the selected feature subset, and execution efficiency. Moreover, experiments on distributed-memory multicore clusters show that DiRH-BPSO is significantly faster than its sequential counterpart and is perfectly capable of completing large-scale feature selection tasks that fail on a single node due to memory constraints. Parallel scalability and extensibility analysis also demonstrate that DiRH-BPSO could scale out and extend well with the growth of computational nodes and the volume of data.

Feature selection is an important topic in data mining and machine learning, which aims to select an optimal feature subset for building effective and explainable prediction models. This article introduces Rough Hypercuboid based Distributed Online Feature Selection (RHDOFS) method to tackle two critical challenges of Volume and Velocity associated with Big Data. By exploring the class separability in the boundary region of rough hypercuboid approach, a novel integrated feature evaluation criterion is proposed by examining not only the explicit patterns contained in the positive region but also the useful implicit patterns derived from the boundary region. An efficient online feature selection method for streaming feature scenario is developed to identify relevant and nonredundant features in an incremental iterative fashion. Furthermore, a parallel optimization mechanism by combining both data and computational independence is further employed to accelerate the original sequential implementation. An efficient distributed online feature selection algorithm is presented and implemented on the Apache Spark platform to scale for massive amount of data by exploiting the computational capabilities of multicore clusters. Encouraging results of extensive experiments indicate the superiority and notable advantages of the proposed algorithm over the relevant and representative online feature selection algorithms. Empirical tests on scalability and extensibility also demonstrate our distributed implementation significantly reduces the computational times requirements while maintaining the prediction accuracy, and is capable of scaling well in volume of data and number of computing nodes.

Outlier detection is one of the most important research directions in data mining. However, most of the current research focuses on outlier detection for categorical or numerical attribute data. There are few studies on the outlier detection of mixed attribute data. In this article, we introduce fuzzy rough sets (FRSs) to deal with the problem of outlier detection in mixed attribute data. Since the outlier detection model of the classical rough set is only applicable to the categorical attribute data, we use FRS to generalize the outlier detection model and construct a generalized outlier detection model based on fuzzy rough granules. First, the granule outlier degree (GOD) is defined to characterize the outlier degree of fuzzy rough granules by employing the fuzzy approximation accuracy. Then, the outlier factor based on fuzzy rough granules is constructed by integrating the GOD and the corresponding weights to characterize the outlier degree of objects. Furthermore, the corresponding fuzzy rough granules-based outlier detection (FRGOD) algorithm is designed. The effectiveness of the FRGOD algorithm is evaluated through experiments on 16 real-world datasets. The experimental results show that the algorithm is more flexible for detecting outliers and is suitable for numerical, categorical, and mixed attribute data.

Incremental feature selection approaches can improve the efficiency of feature selection used for dynamic datasets, which has attracted increasing research attention. Nevertheless, there is currently no work on incremental feature selection approaches for dynamic ordered data. Moreover, the monotonic classification effect of ordered data is easily affected by noise, so a robust feature evaluation metric is needed for feature selection algorithm. Motivated by these two issues, we investigate incremental feature selection approaches using a new conditional entropy with robustness for dynamic ordered data in this study. First, we propose a new rough set model, i.e., fuzzy dominance neighborhood rough sets (FDNRS). Second, a conditional entropy with robustness is defined based on FDNRS model, which is used as evaluation metric for features and combined with a heuristic feature selection algorithm. Finally, two incremental feature selection algorithms are designed on the basis of the above researches. Experiments are performed on ten public datasets to evaluate the robustness of the proposed metric and the performance of the incremental algorithms. Experimental results verify that the proposed metric is robust and our incremental algorithms are effective and efficient for updating reducts in dynamic ordered data.

Monotonic classification is a common task in the field of multicriteria decision-making, in which features and decision obey a monotonic constraint. The dominance-based rough set theory is an important mathematical tool for knowledge acquisition in monotonic classification tasks (MCTs). However, existing dominance-based rough set models are very sensitive to noise information, and only a misclassified sample will lead to large errors in acquiring knowledge. This unstable phenomenon does not meet the requirements of practical applications. On the other hand, feature selection is supposedly an effective dimensionality reduction approach for classification tasks. In the real world, feature combinations with multiple correlations can often provide important classification information, where the multiple correlations include redundancy, complementarity, and interaction between features. To the best of our knowledge, most of the existing feature selection methods for MCTs only consider the relevance between features and decision, while ignoring the multiple correlations. To overcome these two drawbacks, in this article, we propose a robust fuzzy dominance rough set model, and develop a feature selection method that considers multiple correlations based on the robust model for MCTs. First, a soft fuzzy dominance rough set (SFDRS) with robustness is proposed. Second, a feature evaluation index considering multiple correlations is presented. Finally, a feature selection algorithm based on SFDRS is designed to select an optimal feature subset. Extensive experiments are conducted on 12 public datasets, and the results show that the SFDRS model has good robustness and the proposed feature selection algorithm has excellent classification performance.

Aiming at the problem of effectively selecting relevant features from heterogeneous data without decision, a novel feature selection approach is studied based on fuzzy mutual information in fuzzy rough set theory. First, the fuzzy relevance of each feature is defined by using fuzzy mutual information, and then, the fuzzy conditional relevance is further given. Next, the fuzzy redundancy is defined by using the difference between the fuzzy relevance and the fuzzy conditional relevance. Thereby, the evaluation index of the feature importance is obtained by using the idea of unsupervised minimum redundancy and maximum relevance. Finally, a fuzzy-mutual-information-based unsupervised feature selection algorithm is designed to select feature sequences. Extensive experiments are conducted on public datasets, and six unsupervised feature selection algorithms are compared. The selected features are evaluated by classification, clustering, and outlier detection methods. Experimental results show that the proposed algorithm can select fewer heterogeneous features to maintain or improve the performance of learning algorithms.

Outlier detection is widely used in many fields, such as intrusion detection, credit card fraud detection, medical diagnosis, and so on. Existing outlier detection algorithms are mostly designed for dealing with numeric or categorical attributes. However, data usually exist in the form of mixed attributes in real-world applications. In this article, we propose a novel mixed attribute outlier detection method based on multigranulation relative entropy by employing the neighborhood rough set. First, the neighborhood system is constructed by optimizing the mixed distance metric and the radius of the statistical value. Second, the neighborhood entropy is introduced as an uncertainty measure of data. Furthermore, the three kinds of multigranulation relative entropy-based matrices are defined by three kinds of attribute sequences, and the multigranulation relative entropy-based outlier factor is integrated to indicate the outlier degree of every object. Based on the proposed outlier detection model, the corresponding algorithm is designed. Finally, the proposed algorithm is compared with other nine algorithms through experiments on public data. The experimental results show that the proposed technique is adaptive and effective.

This article aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretically analyze and summarize the general properties of all algorithms that are based on traditional Auto-Encoders: (1) The reconstruction error of the input cannot be lower than a lower bound, which can be viewed as a guiding principle for reconstructing the input. Additionally, when the input is corrupted with noises, the reconstruction error of the corrupted input also cannot be lower than a lower bound. (2) The reconstruction of a hidden representation achieving its ideal situation is the necessary condition for the reconstruction of the input to reach the ideal state. (3) Minimizing the Frobenius norm of the Jacobian matrix of the hidden representation has a deficiency and may result in a much worse local optimum value. We believe that minimizing the reconstruction error of the hidden representation is more robust than minimizing the Frobenius norm of the Jacobian matrix of the hidden representation. Based on the above analysis, we propose a new model termedDouble Denoising Auto-Encoders(DDAEs), which uses corruption and reconstruction on both the input and the hidden representation. We demonstrate that the proposed model is highly flexible and extensible and has a potentially better capability to learn invariant and robust feature representations. We also show that our model is more robust than Denoising Auto-Encoders (DAEs) for dealing with noises or inessential features. Furthermore, we detail how to train DDAEs with two different pretraining methods by optimizing the objective function in a combined and separate manner, respectively. Comparative experiments illustrate that the proposed model is significantly better for representation learning than the state-of-the-art models.

Attribute reduction based on rough set theory has attracted much attention recently. In real-life applications, many decision tables may vary dynamically with time, e.g., the variation of attributes, objects, and attribute values. The reduction of decision tables may change on the alteration of attribute values. The paper focuses on dynamic maintenance of attribute reduction when varying data values of multiple objects. Incremental mechanisms for knowledge granularity are proposed first, which aims to update attribute reduction effectively. Then, a group incremental reduction algorithm with varying data values is developed. When attribute values of multiple objects have been replaced by new ones in decision table, the proposed incremental algorithm can find the new reduct in a much shorter time. The time complexity analysis and experiments on different data sets from UCI have validated that the proposed incremental algorithms are efficient and effective to update the reduction with the variation of attribute values.

The rough set model over dual universes (RSMDU) as a generalized model of classical rough set theory (RST) on the two universes has been well studied with the objective to establishment of model and discussion of its corresponding properties. Approximations of a concept in RSMDU, which may further be applied to knowledge discovery or related work, need to be updated effectively under a dynamic environment. Despite recent advances in using the incremental method to speed up updating approximations of RST, there has been little effort toward incorporating the incremental method into computing approximations under RSMDU. This paper proposes an incremental learning approach for updating approximations in RSMDU when the objects of two universes vary with time. An illustration is employed to show the proposed method. Extensive experimental results on various real and synthetic data sets verify the effectiveness of the proposed incremental updating method while comparing with the nonincremental method.

Uncertainty and fuzziness generally exist in real-life data. Approximations are employed to describe the uncertain information approximately in rough set theory. Certain and uncertain rules are induced directly from different regions partitioned by approximations. Approximation can further be applied to datamining-related task, e.g., attribute reduction. Nowadays, different types of data collected from different applications evolve with time, especially new attributes may appear while new objects are added. This paper presents an approach for dynamic maintenance of approximations w.r.t. objects and attributes added simultaneously under the framework of decision-theoretic rough set (DTRS). Equivalence feature vector and matrix are defined first to update approximations of DTRS in different levels of granularity. Then, the information system is decomposed into subspaces, and the equivalence feature matrix is updated in different subspaces incrementally. Finally, the approximations of DTRS are renewed during the process of updating the equivalence feature matrix. Extensive experimental results verify the effectiveness of the proposed methods.

The lower and upper approximations in rough set theory will change dynamically over time due to the variation of the information system. Incremental methods for updating approximations in rough set theory and its extensions have received much attenti

Rule induction method based on rough set theory (RST) has received much attention recently since it may generate a minimal set of rules from the decision system for real-life applications by using of attribute reduction and approximations. The decision system may vary with time, e.g., the variation of objects, attributes and attribute values. The reduction and approximations of the decision system may alter on Attribute Values' Coarsening and Refining (AVCR), a kind of variation of attribute values, which results in the alteration of decision rules simultaneously. This paper aims for dynamic maintenance of decision rules w.r.t. AVCR. The definition of minimal discernibility attribute set is proposed firstly, which aims to improve the efficiency of attribute reduction in RST. Then, principles of updating decision rules in case of AVCR are discussed. Furthermore, the rough set-based methods for updating decision rules in the inconsistent decision system are proposed. The complexity analysis and extensive experiments on UCI data sets have verified the effectiveness and efficiency of the proposed methods.

Incremental learning is an efficient technique for knowledge discovery in a dynamic database, which enables acquiring additional knowledge from new data without forgetting prior knowledge. Rough set theory has been successfully used in information systems for classification analysis. Set-valued information systems are generalized models of single-valued information systems, which can be classified into two categories: disjunctive and conjunctive. Approximations are fundamental concepts of rough set theory, which need to be updated incrementally while the object set varies over time in the set-valued information systems. In this paper, we analyze the updating mechanisms for computing approximations with the variation of the object set. Two incremental algorithms for updating the approximations in disjunctive/conjunctive set-valued information systems are proposed, respectively. Furthermore, extensive experiments are carried out on several data sets to verify the performance of the proposed algorithms. The results indicate the incremental approaches significantly outperform non-incremental approaches with a dramatic reduction in the computational speed.

Approximations of a concept by a variable precision rough-set model (VPRS) usually vary under a dynamic information system environment. It is thus effective to carry out incremental updating approximations by utilizing previous data structures. This paper focuses on a new incremental method for updating approximations of VPRS while objects in the information system dynamically alter. It discusses properties of information granulation and approximations under the dynamic environment while objects in the universe evolve over time. The variation of an attribute's domain is also considered to perform incremental updating for approximations under VPRS. Finally, an extensive experimental evaluation validates the efficiency of the proposed method for dynamic maintenance of VPRS approximations.

In rough set theory, upper and lower approximations for a concept will change dynamically as the information system changes over time. How to update approximations based on the original information is an important task that can help improve the efficiency of knowledge discovery. This paper focuses on the approach of dynamically updating approximations when attribute values are coarsened or refined. The main contributions include: (1) defining coarsening and refining attribute values in information systems and introducing the properties and the principles of coarsening and refining attribute values; (2) analyzing the properties for dynamic maintenance in terms of upper and lower approximations with coarsening and refining attribute values; (3) proposing an incremental algorithm for updating the approximations of a concept as coarsening or refining attributes values; and finally (4) validating the efficiency of the proposed approach to handle the dynamic maintenance of the approximations for a given concept. © 2010 Wiley Periodicals, Inc.