Spectral clustering is a prevalent clustering method in which an affinity matrix is constructed based on all samples (the number is n ), leading to high computational complexity and making it infeasible for dealing with large-scale data directly. In this study, we introduce an Approximate Spectral Embedding Representation method (ASER). By employing an anchor-based strategy, the spectral embedding representation of the selected anchors (the number is m , m ≪ n ) is used to approximate the spectral embedding representation of the original samples. Unlike available methods that approximate the similarity matrix based on an anchor graph, we directly implement the approximation in the spectral embedding space. Moreover, the properties of the formed anchor graph are inherited from the original space to the spectral embedding space. The time complexity of conducting spectral clustering is significantly reduced from O ( n 3 ) to be linear with respect to n , without relying on any acceleration operations for eigenvalue decomposition. Experimental results on toy examples and benchmark datasets with large sizes demonstrate the effectiveness and efficiency of the proposed model.

Ultrasonic image anomaly detection faces significant challenges due to limited labeled data, strong structural and random noise, and highly diverse defect manifestations. To overcome these obstacles, we introduce UltraChip, a new large-scale C-scan benchmark containing about 8,000 real-world images from various chip packaging types, each meticulously annotated with pixel-level masks for cracks, holes, and layers. Building on this resource, we present FSGM-Net, a fully unsupervised framework tailored for anomaly detection. FSGM-Net leverages an adaptive Frequency-Spatial feature filtering mechanism: a learnable FFT-Spatial patch filter first suppresses noise and dynamically assigns normality weights to Vision Transformer (ViT) patch features. Subsequently, an Adaptive Gaussian Mixture Model (Ada-GMM) captures the distribution of normal features and guides a deep-shallow multi-scale interaction decoder for accurate, pixel-level anomaly inference. In addition, we propose a filter loss that enforces encoder-filter consistency and entropy-based sparse gating, together with a distributional loss that encourages both feature reconstruction and confident Gaussian mixture modeling. Extensive experiments demonstrate that FSGM-Net not only achieves state-of-the-art results on UltraChip but also exhibits superior cross-domain generalization to MVTec-AD and VisA, while supporting real-time inference on a single GPU. Together, the dataset and framework advance robust, annotation-free ultrasonic NDT in practical applications. The UltraChip dataset can be obtained via https://iiplab.net/ultrachip/.

Anchor-based strategies have been widely used to accelerate spectral clustering, yet their effectiveness is directly affected by the quality of the selected anchors. Random sampling has become one of the most important anchor determination methods due to its efficiency. However, the anchors obtained by a single random sampling often fail to adequately capture the topological structure of the original data, making it difficult for the constructed anchor graph to achieve satisfactory clustering performance. To solve this problem, we propose a novel spectral embedding representation model based on random anchor graph aggregation (RAGA), in which an aggregated anchor graph can be produced to obtain enhanced sample representation capability. Specifically, we perform multiple random samplings to make the distribution of the selected anchors approximate the original data within a reasonable sampling time. Subsequently, adaptive weighted learning is performed on the contribution of the constructed multiple anchor graphs, and then an aggregated anchor graph can be formed, which can portray the topological structure of the original samples more precisely. In addition, spectral embedding and spectral rotation are integrated into a joint learning framework to reduce the model learning error accumulation caused by the traditional two-stage framework. Notably, we propose a rigorous theorem for analyzing the approximation of samples by the selected anchors in multiple random samplings. Our proposed RAGA maintains the speed advantage of random sampling while obtaining a high-quality aggregated anchor graph, enabling it to handle large-scale data scenarios. Experimental results on several benchmark datasets show that the RAGA model outperforms other state-of-the-art (SOTA) anchor graph-based clustering methods.

3D anomaly detection has recently become a significant focus in computer vision. Several advanced methods have achieved satisfying anomaly detection performance. However, they typically concentrate on the external structure of 3D samples and struggle to leverage the internal information embedded within samples. Inspired by the basic intuition of why not look inside for more, we observed this prototype is straightforward and effective. As a result, we introduce a newly designed mode named Internal Spatial Modality Perception (ISMP) to explore the feature representation from internal views fully. Specifically, our proposed ISMP consists of a critical perception module, Spatial Insight Engine (SIE), which abstracts complex internal information of point clouds into essential global features. Besides, to better align structural information with point data, we propose an enhanced key point feature extraction method for amplifying spatial structure feature representation. Simultaneously, a novel feature filtering module is incorporated to reduce noise and redundant features for further precise spatial structure aligning. Extensive experiments validate the efficiency of our proposed method, achieving object-level and pixel-level AUROC improvements of 4.2% and 13.1%, respectively, on the Real3D-AD benchmarks. Note that the strong generalization ability of SIE has been theoretically proven and verified in both classification and segmentation tasks. Our code will be released upon acceptance.

Learning with Noisy Labels (LNL) aims to improve the model generalization when facing data with noisy labels, and existing methods generally assume that noisy labels come from known classes, called closed-set noise. However, in real-world scenarios, noisy labels from similar unknown classes, i.e., open-set noise, may occur during the training and inference stage. Such open-world noisy labels may significantly impact the performance of LNL methods. In this study, we propose a novel dual-space joint learning method to robustly handle the open-world noise. To mitigate model overfitting on closed-set and open-set noises, a dual representation space is constructed by two networks. One is a projection network that learns shared representations in the prototype space, while the other is a One-Vs-All (OVA) network that makes predictions using unique semantic representations in the class-independent space. Then, bi-level contrastive learning and consistency regularization are introduced in two spaces to enhance the detection capability for data with unknown classes. To benefit from the memorization effects across different types of samples, class-independent margin criteria are designed for sample identification, which selects clean samples, weights closed-set noise, and filters open-set noise effectively. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods and achieves an average accuracy improvement of 4.55\% and an AUROC improvement of 6.17\% on CIFAR80N.

3D Anomaly Detection (AD) is a promising means of controlling the quality of manufactured products. However, existing methods typically require carefully training a task-specific model for each category independently, leading to high cost, low efficiency, and weak generalization. This study presents a novel unified model for Multi-Category 3D Anomaly Detection (MC3D-AD) that aims to utilize both local and global geometry-aware information to reconstruct normal representations of all categories. First, to learn robust and generalized features of different categories, we propose an adaptive geometry-aware masked attention module that extracts geometry variation information to guide mask attention. Then, we introduce a local geometry-aware encoder reinforced by the improved mask attention to encode group-level feature tokens. Finally, we design a global query decoder that utilizes point cloud position embeddings to improve the decoding process and reconstruction ability. This leads to local and global geometry-aware reconstructed feature tokens for the 3D AD task. MC3D-AD is evaluated on two publicly available Real3D-AD and Anomaly-ShapeNet datasets, and exhibits significant superiority over current state-of-the-art single-category methods, achieving 3.1% and 9.3% improvement in object-level AUROC over Real3D-AD and Anomaly-ShapeNet, respectively. The code is available at https://github.com/iCAN-SZU/MC3D-AD.

Few-shot learning aims to identify novel concepts with limited annotated examples. Recent works have made significant progress on modeling the distribution of novel categories. They generally rely on the prior knowledge from base set, which is often acquired by calculating the distance between base and support classes. However, the number of samples given from novel categories is often very limited, making it difficult to acquire accurate prior, especially when they are far away from the class center. To address these issues, we explore them in a Bayesian framework, proposing a prior knowledge in Bayesian learning with semantic information networks (Se-PKBL) to enhance the reliability of prior knowledge which has been captured during training stage. Specifically, we introduce semantic similarity between classes, which are easily available, to obtain the most similar classes for the novel categories. Experimental results show that it outperforms the state-of-the-art methods by a large margin, improves around 1.96%∼5.29% than the best results in each dataset on both 1- and 5-shot tasks.

Few-shot image classification seeks to categorize novel classes using only a limited number of samples. As a key topic in deep learning, a notable approach to this challenge involves leveraging statistical analysis of base set category distributions as prior knowledge, which is subsequently employed to estimate the true distributions of categories in the novel set through probabilistic modeling. Due to the extreme scarcity of samples, especially in 1-shot scenarios, the modeling process heavily relies on the accuracy of prior information, making it a critical factor for performance. To address this issue, we propose a prior enhancement strategy based on maximum a posteriori (MAP) estimation to improve the accuracy of priors and thereby enhance classification performance. Specifically, during prior selection, we incorporate advanced semantics enriched with category-specific information to refine the priors. Furthermore, for prior fusion, we introduce a temperature-scaled softmax function to adaptively weight and combine the selected priors. Extensive experiments conducted on four benchmark few-shot datasets demonstrate the effectiveness of the proposed strategy, achieving state-of-the-art performance.

Reconstruction-based methods have demonstrated very promising results for 3D anomaly detection. However, these methods face great challenges in handling high-precision point clouds due to the large scale and complex structure. In this study, a Down-Up Sampling Networks (DUS-Net) is proposed to reconstruct high-precision point clouds for 3D anomaly detection by preserving the group center geometric structure. The DUS-Net first introduces a Noise Generation module to generate noisy patches, which facilitates the diversity of training data and strengthens the feature representation for reconstruction. Then, a Down-sampling Network (Down-Net) is developed to learn an anomaly-free center point cloud from patches with noise injection. Subsequently, an Up-sampling Network (Up-Net) is designed to reconstruct high-precision point clouds by fusing multi-scale up-sampling features. Our method leverages group centers for construction, enabling the preservation of geometric structure and providing a more precise point cloud. Extensive experiments demonstrate the effectiveness of our proposed method, achieving state-of-the-art (SOTA) performance, with an Object-level AUROC of 79.9% and 79.5% and a Point-level AUROC of 71.2% and 84.7% on the Real3D-AD and Anomaly-ShapeNet datasets, respectively.

Neighborhood rough sets are an effective model for handling numerical and categorical data entangled with vagueness, imprecision, or uncertainty. However, existing neighborhood rough set models and their feature selection methods treat each sample equally, whereas different types of samples inherently play different roles in constructing neighborhood granules and evaluating the goodness of features. In this study, the sample weight information is first introduced into neighborhood rough sets, and a novel weighted neighborhood rough set model is consequently constructed. Then, considering the lack of sample weight information in practical data, a margin-based weight optimization function is designed, based on which a gradient descent algorithm is provided to adaptively learn sample weights through maximizing sample margins. Finally, an average granule margin measure is put forward for feature selection, and a forward-adding heuristic algorithm is developed to generate an optimal feature subset. The proposed method constructs the weighted neighborhood rough sets using sample weights for the first time and is able to yield compact feature subsets with a large margin. Extensive experiments and statistical analysis on UCI datasets show that the proposed method achieves highly competitive performance in terms of feature reduction rate and classification accuracy when compared with other state-of-the-art methods.

Subspace clustering has attracted significant interest for its capacity to partition high-dimensional data into multiple subspaces. The current approaches to subspace clustering predominantly revolve around two key aspects: 1) the construction of an effective similarity matrix and 2) the pursuit of sparsity within the projection matrix. However, assessing whether the dimensionality of the projected subspace is the true dimensionality of the data is challenging. Therefore, the clustering performance may decrease when dealing with scenarios such as subspace overlap, insufficient projected dimensions, data noise, etc., since the defined dimensionality of the projected lower-dimensional space may deviate significantly from its true value. In this research, we introduce a novel reweighting strategy, which is applied to projected coordinates for the first time and propose a reweighted subspace clustering model guided by the preservation of the both local and global structural characteristics (RWSC). The projected subspaces are reweighted to augment or suppress the importance of different coordinates, so that data with overlapping subspaces can be better distinguished and the redundant coordinates produced by the predefined number of projected dimensions can be further removed. By introducing reweighting strategies, the bias caused by imprecise dimensionalities in subspace clustering can be alleviated. Moreover, global scatter structure preservation and adaptive local structure learning are integrated into the proposed model, which helps RWSC capture more intrinsic structures and its robustness and applicability can then be improved. Through rigorous experiments on both synthetic and real-world datasets, the effectiveness and superiority of RWSC are empirically verified.

Outlier detection refers to the identification of anomalous samples that deviate significantly from the distribution of normal data and has been extensively studied and used in a variety of practical tasks. However, most unsupervised outlier detection methods are carefully designed to detect specified outliers, while real-world data may be entangled with different types of outliers. In this study, we propose a fuzzy rough sets-based multi-scale outlier detection method to identify various types of outliers. Specifically, a novel fuzzy rough sets-based method that integrates relative fuzzy granule density is first introduced to improve the capability of detecting local outliers. Then, a multi-scale view generation method based on granular-ball computing is proposed to collaboratively identify group outliers at different levels of granularity. Moreover, reliable outliers and inliers determined by the three-way decision are used to train a weighted support vector machine to further improve the performance of outlier detection. The proposed method innovatively transforms unsupervised outlier detection into a semi-supervised classification problem and for the first time explores the fuzzy rough sets-based outlier detection from the perspective of multi-scale granular balls, allowing for high adaptability to different types of outliers. Extensive experiments carried out on both artificial and UCI datasets demonstrate that the proposed outlier detection method significantly outperforms the state-of-the-art methods, improving the results by at least 8.48% in terms of the Area Under the ROC Curve (AUROC) index. The source codes are released at https://github.com/Xiaofeng-Tan/MGBOD

High-resolution point clouds (HRPCD) anomaly detection (AD) plays a critical role in precision machining and high-end equipment manufacturing. Despite considerable 3D-AD methods that have been proposed recently, they still cannot meet the requirements of the HRPCD-AD task. There are several challenges: i) It is difficult to directly capture HRPCD information due to large amounts of points at the sample level; ii) The advanced transformer-based methods usually obtain anisotropic features, leading to degradation of the representation; iii) The proportion of abnormal areas is very small, which makes it difficult to characterize. To address these challenges, we propose a novel group-level feature-based network, called Group3AD, which has a significantly efficient representation ability. First, we design an Intercluster Uniformity Network (IUN) to present the mapping of different groups in the feature space as several clusters, and obtain a more uniform distribution between clusters representing different parts of the point clouds in the feature space. Then, an Intracluster Alignment Network (IAN) is designed to encourage groups within the cluster to be distributed tightly in the feature space. In addition, we propose an Adaptive Group-Center Selection (AGCS) based on geometric information to improve the pixel density of potential anomalous regions during inference. The experimental results verify the effectiveness of our proposed Group3AD, which surpasses Reg3D-AD by the margin of 5% in terms of object-level AUROC on Real3D-AD. We provide the code and supplementary information on our website: https://github.com/M-3LAB/Group3AD.

Available subspace clustering methods often contain two stages, finding low-dimensional subspaces of data and then conducting clustering in the subspaces. Therefore, how to find the subspaces that better represent the original data becomes a research challenge. However, most of the reported methods are based on the premise that the contributions of different features are equal, which may not be ideal for real scenarios, i.e., the contributions of the important features may be overwhelmed by a large amount of redundant features. In this study, a weighted subspace fuzzy clustering (WSFC) model with a locality preservation mechanism is presented, which can adaptively capture the importance of different features, achieve an optimal lower-dimensional subspace, and perform fuzzy clustering simultaneously. Since each feature can be well quantified in terms of its importance, the proposed model exhibits the sparsity and robustness of fuzzy clustering. The intrinsic geometrical structures of data can also be preserved while enhancing the interpretability of clustering tasks. Extensive experimental results show that WSFC can allocate appropriate weights to different features according to data distributions and clustering tasks and achieve superior performance compared to other clustering models on real-world datasets.

Self-expression learning methods often obtain a coefficient matrix to measure the similarity between pairs of samples. However, directly using the raw data to represent each sample under the self-expression framework may not be ideal, as noise points are inevitably involved in the process of representing clean samples. To address this issue, this work proposes a novel self-expression model called robust Self-Expression learning with adaptive Noise Perception (SENP). SENP decomposes each sample into a clean part and a noisy part, and samples with large self-expression losses can be recognized as the noise points. A reliable coefficient matrix can then be learned by using only the clean points to reconstruct the clean part of each sample. By simultaneously detecting the noisy part of each sample and noise points, and adaptively mitigating their negative impacts, the representative ability of the generated coefficient matrix is improved. Moreover, inspired by the solution of non-negative matrix factorization (NMF), an effective algorithm is formed to optimize SENP. Extensive experiments on well-known benchmark datasets demonstrate the superiority of SENP compared to several state-of-the-art methods.

Representing data in different spaces becomes more powerful and suitable for solving downstream learning tasks. The membership degrees obtained through fuzzy C-means (FCM) clustering cannot capture data structures sufficiently, as they represent samples from a single Euclidean geometrical perspective. To address this issue, we propose a novel fuzzy clustering model guided by spectral rotation and scaling (FCSR). In FCSR, both spectral embeddings and membership degrees are considered as new representations of data. They can complement each other from different perspectives which enables the model to engage more structural properties of the data. The process of solving the problem of membership degrees not only inherits the merits of traditional FCM but also preserves data neighborhood structures revealed by the spectral decomposition based on an affinity matrix. Furthermore, to improve the adaptability and extensibility of FCSR, the projected and kernel versions of FCSR (FCSR-P and FCSR-K) are formed. We demonstrate that FCSR-P is suitable for high-dimensional scenarios and FCSR-K can improve the linear separability among data. Extensive experiments conducted on various well-known data sets illustrate the validity of the proposed ideas.

Graph-based clustering approaches, especially the family of spectral clustering, have been widely used in machine learning areas. The alternatives usually engage a similarity matrix that is constructed in advance or learned from a probabilistic perspective. However, unreasonable similarity matrix construction inevitably leads to performance degradation, and the sum-to-one probability constraints may make the approaches sensitive to noisy scenarios. To address these issues, the notion of typicality-aware adaptive similarity matrix learning is presented in this study. The typicality (possibility) rather than the probability of each sample being a neighbor of other samples is measured and adaptively learned. By introducing a robust balance term, the similarity between any pairs of samples is only related to the distance between them, yet it is not affected by other samples. Therefore, the impact caused by the noisy data or outliers can be alleviated, and meanwhile, the neighborhood structures can be well captured according to the joint distance between samples and their spectral embeddings. Moreover, the generated similarity matrix has block diagonal properties that are beneficial to correct clustering. Interestingly, the results optimized by the typicality-aware adaptive similarity matrix learning share the common essence with the Gaussian kernel function, and the latter can be directly derived from the former. Extensive experiments on synthetic and well-known benchmark datasets demonstrate the superiority of the proposed idea when comparing with some state-of-the-art methods.

Abstract Anomaly detection is a challenging task, especially detecting and segmenting tiny defect regions in images without anomaly priors. Although deep encoder‐decoder‐based convolutional neural networks have achieved good anomaly detection results, existing methods operate uniformly on all extracted image features without considering disentangling these features. To fully explore the texture and semantic information of images, A novel unsupervised anomaly detection method is proposed. Specifically, discriminative features are extracted from images by using a deep pre‐trained network, where shallow and deep features are aggregated into texture and semantic modules, respectively. Then, a feature fusion module is developed to interactively enable feature information in two different modules. The texture and semantic segmentation results are obtained by comparing the texture features and semantic features before and after reconstruction, respectively. Finally, an anomaly segmentation module is designed to generate anomaly detection results by integrating the results of the texture and semantic modules by setting a threshold. Experimental results on benchmark datasets for anomaly detection demonstrate that our proposed method can efficiently and effectively detect anomalies, outperforming some state‐of‐the‐art methods by 2.7% and 0.6% in classification and segmentation.

The performance of k-means clustering is often degenerate when dealing with high-dimensional and noisy scenarios. In this study, an end-to-end robust clustering method with low-rank linear embedding techniques (RCLR) is presented in conjunction with k-means. Sparse coefficients and a space projection matrix can be simultaneously learned. The global structures and local neighborhood properties are well captured in the learning procedures. Both the processes of clustering and dimensionality reduction are realized at the same time. The notions of clustering, dimensionality reduction, low-rank representation, and local property preservation are seamlessly integrated into a unified model. The limitation of error accumulation encountered in the previous two-stage clustering framework involving low-rank representation can be alleviated. This is the first attempt to introduce both the global and local geometrical structures into k-means directly, as well L2,1-norm is used as a basic metric instead of the conventional F-norm to further improve the robustness and interpretation of the model. The superiority of the proposed RCLR method is demonstrated by extensive experiments completed on various well-known benchmark datasets.

Fuzzy clustering techniques, especially fuzzy C-means (FCM) and its weighted variants, are typical partitive clustering models that are widely used for revealing possible hidden structures in data. Although they can quantitatively depict the overlapping areas with a partition matrix, their performances deteriorate when dealing with high-dimensional data because the distance computations may be negatively impacted by the irrelevant features, and then the concentration effect may arise. Moreover, they are sensitive to noisy environments. To tackle these obstacles, a robust jointly sparse fuzzy clustering method (RJSFC) is proposed in this study. The representative prototypes, sparse membership grades, and an orthogonal projection matrix are simultaneously learnt when optimizing RJSFC. The obtained low-dimensional embeddings can preserve the local neighborhood structure, and the clustering is conducted in the transformed lower dimensional space rather than the original space, which improves the capability of fuzzy clustering for dealing with high-dimensional scenarios. Furthermore,${L_{2,1}}$-norm is exploited as the basic metric for both loss and regularization parts in RJSFC, the robustness of the model and the interpretability of the extracted features are enhanced. The notions of fuzzy clustering, neighborhood structure preservation, and feature extraction are seamlessly integrated into a unified model. The limitation of the previous two-stage clustering framework when dealing with high-dimensional data entailing dimensionality reduction and clustering procedures separately can be effectively addressed. Extensive experimental results on various well-known datasets demonstrate the usefulness of RJSFC when comparing with some state-of-the-art methods.

Recently, heatmap regression has been widely explored in facial landmark detection and obtained remarkable performance. However, most of the existing heatmap regression-based facial landmark detection methods neglect to explore the high-order feature correlations, which is very important to learn more representative features and enhance shape constraints. Moreover, no explicit global shape constraints have been added to the final predicted landmarks, which leads to a reduction in accuracy. To address these issues, in this article, we propose a multiorder multiconstraint deep network (MMDN) for more powerful feature correlations and shape constraints' learning. Especially, an implicit multiorder correlating geometry-aware (IMCG) model is proposed to introduce the multiorder spatial correlations and multiorder channel correlations for more discriminative representations. Furthermore, an explicit probability-based boundary-adaptive regression (EPBR) method is developed to enhance the global shape constraints and further search the semantically consistent landmarks in the predicted boundary for robust facial landmark detection. It is interesting to show that the proposed MMDN can generate more accurate boundary-adaptive landmark heatmaps and effectively enhance shape constraints to the predicted landmarks for faces with large pose variations and heavy occlusions. Experimental results on challenging benchmark data sets demonstrate the superiority of our MMDN over state-of-the-art facial landmark detection methods.

Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, Haifeng Wang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Unsupervised domain adaptation (UDA) aims to transfer knowledge between different domains. Most of the existing UDA methods try to align the conditional distribution between the source and target domains by utilizing the information of pseudo labels induced from the target domain. To tackle the negative transfer caused by inaccurate pseudo labels, we propose a novel UDA method named progressive distribution alignment based on label correction (PDALC). Specifically, PDALC uses the class discriminative information to perform subspace learning to obtain the domain invariance subspace. Furthermore, a new mechanism of pseudo label correction is introduced to measure the reliability of pseudo labels and utmostly correct the inaccurate pseudo labels. By combining subspace learning with label correction, the performance of PDALC can be continuously improved, which in turn reduces the generation of inaccurate pseudo labels. The experimental results show that the proposed method outperforms the state-of-the-art UDA methods.

Heatmap regression (HR) has become one of the mainstream approaches for face alignment and has obtained promising results under constrained environments. However, when a face image suffers from large pose variations, heavy occlusions and complicated illuminations, the performances of HR methods degrade greatly due to the low resolutions of the generated landmark heatmaps and the exclusion of important high-order information that can be used to learn more discriminative features. To address the alignment problem for faces with extremely large poses and heavy occlusions, this paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model, which are seamlessly integrated into a novel multi-order high-precision hourglass network (MHHN). The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT). At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information. To the best of our knowledge, this is the first study to explore heatmap subpixel regression for robust and high-precision face alignment. The experimental results from challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

Recently, sentiment analysis has seen remarkable advance with the help of pre-training approaches. However, sentiment knowledge, such as sentiment words and aspect-sentiment pairs, is ignored in the process of pre-training, despite the fact that they are widely used in traditional sentiment analysis approaches. In this paper, we introduce Sentiment Knowledge Enhanced Pre-training (SKEP) in order to learn a unified sentiment representation for multiple sentiment analysis tasks. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair. Experiments on three kinds of sentiment tasks show that SKEP significantly outperforms strong pre-training baseline, and achieves new state-of-the-art results on most of the test datasets. We release our code at https://github.com/baidu/Senta.

Shadowed sets provide a meaningful description of information granules by abstracting the corresponding fuzzy sets into three categories: full acceptance, full rejection, and uncertain (represented by shadows). One of the main motivating points to derive shadowed sets from fuzzy sets is the determination and explanation of the separation thresholds based on a specific optimization mechanism. The available optimization objective functions are mainly discussed on semantic interpretations and their mathematical properties; constructive algorithms for optimal solutions have rarely been reported. In this paper, the continuous and convex properties of Pedrycz's optimization objective function to construct shadowed sets, as well as the existence and uniqueness of solution points, are analyzed in detail. It is demonstrated that different approximation region partitions would be generated even under the same optimization model, which requires further criteria to make the constructed shadowed sets well-defined. To address this limitation, the notions of passive and active constrained shadowed sets are introduced. A fast algorithm to obtain the proposed constrained shadowed sets is also designed based on the analyzed mathematical properties. Its performance is then illustrated by some typical fuzzy sets and some real data from the UCI repository.

The ensemble of feature selections facilitates to improve data generalization for learning tasks. However, existing feature selection ensemble methods have the following drawbacks. First, focusing on the numerical data, the works on the feature selection ensemble for symbolic and mixed-type data are very limited. Second, the voting-based ensemble strategies tend to select the top significant features but the consistency between the ensemble result and the diverse feature selections cannot be guaranteed. Aiming to handle these problems, we propose an feature selection ensemble method based on Analytic Hierarchy Process (AHP) in this paper. The AHP-based ensemble method can integrate diverse feature selections into a consistent one under the multiple criteria of feature discernibility and independence. Moreover, the ensemble methodology is helpful to implement the feature selection on distributed data and involve domain knowledge through extending criteria. Experimental results validate that the proposed ensemble method of feature selections is effective for symbolic data classification.