Generalized Category Discovery (GCD) is a critical task in open-world computing scenarios, aiming to automatically classify partially labeled data by recognizing both known and novel categories. However, existing GCD methods usually suffer from inherent bias toward known categories due to the exclusive pre-training on them and the absence of labeled data of novel categories. This bias can lead to significant misclassification and clustering errors for novel categories. Although recent approaches leverage pseudo-label training and contrastive learning to address this, they still lack explicit supervision to disentangle novel and known categories, resulting in performance bottlenecks. To address these limitations, we propose an Expectation-Maximization-driven Contrastive Disentanglement (EMCD) framework designed to explicitly disentangle novel and known categories. We particularly formulate the identification of novel categories as a latent variable estimation problem. Specifically, it incorporates an EM-disentangling regularization to softly identify novel category samples and a consistency regularization to enhance generalization. In addition, we leverage dual contrastive constraints, including a cluster-sample contrastive constraint and a sample-sample contrastive contrastive, to pull close samples of novel categories while pushing apart ambiguous samples near decision boundaries. Empirical results on 3 commonly used datasets demonstrate that our model is effective and outperforms previous state-of-the-art methods. Our code is available at https://github.com/YWY-only/EMCD.

Multi-label text classification (MLTC) assigns multiple labels to a sentence, with the key challenge being capturing label correlations. Existing models prioritize leveraging correlations but often overlook overfitting, while plug-and-play regularization methods fail to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlations and implicit semantic correlations, and propose regularizations on prototypical label embeddings for correlation preservation. Specifically, we first generate the prototypical embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical regularization on the distance between the sentence embedding and corresponding prototypical embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods.

Knowledge Graph Completion (KGC) aims to complete KGs by predicting missing entities. A common solution for KGC is Knowledge Graph Embedding (KGE), which assumes that semantical similar entities or relationships should possess similar representations in high-dimensional space. In KGE, a heuristic score function of the head entity and its relation with different operators is required. A typical technique is regularization for tensor factorization, such as the Nuclear-p norm and the Frobenius norm of the query/entity embedding, which significantly improve the KGE model performance on the KGC task. However, the Co-Relation s, including the association between tail entities ( Co-Query Relation ) and the association between queries ( Co-Entity Relation ), desirable for KGC are not fully considered in existing embedding regularization techniques. In this article, we theoretically interpret the role of Co-Relation in KGE and propose a novel ConR regularization approach to learn embedding that takes Co-Relations into account. Extensive experiments show that our model improves static and temporal KGC tasks over decomposition-based models, ComplEx and TuckER. Further analysis of the score cumulative distribution function and embedding visualization demonstrates the effectiveness of ConR .

Temporal Knowledge Graph Forecasting (TKGF) aims to forecast the missing entities or relations at a specific timestamp when only the historical information is observed. It is crucial to accurately identify the historical information of complex temporal relational graphs related to the query. Existing works, e.g., TANGO, have exploited the Neural Ordinary Differential Equation (NODE) to TKGF. However, TANGO encounters two limitations. First, TANGO observes historical facts with only one timestamp at each step, leading to a long-term forgetting problem. Second, TANGO gives the same weight to the entire history graph, including facts that are not relevant to the query. To tackle the above limitations, this paper utilizes Attentional Neural Integral Equation for TKGF (tIE), enabling the global interaction between query-related historical graph sequences. To achieve this, we employ the Relational Graph Convolutional Network and Fourier-type Transformer to model the graph structure and temporal evolution of TKG. The Iterative Integral Equation Solver is exploited to enhance the accuracy and robustness of numerical solutions. The proposed method outperforms baseline models regarding several metrics and inference speed on four benchmark datasets, especially on the long horizontal link forecasting task with irregular time intervals.

Text classification is one vital tool assisting web content mining. Semi-supervised text classification (SSTC) offers an approach to alleviate the burden of annotation costs by training on a few labeled texts alongside many unlabeled texts. Unsolved challenges in SSTC are the overfitting problem caused by the limited labeled data and the mislabeling problem of unlabeled texts. To address these issues, this paper proposes a Self-Paced PairWise representation learning (SPPW) model. Concretely, SPPW alleviates the overfitting problem by replacing the overfitting-prone learning of a parameterized classifier with representation learning in a pair-wise manner. Besides, we propose a novel self-paced text filtering method that effectively integrates both label confidence and text hardness to reduce mislabeled texts synergistically. Extensive experiments on 3 benchmark SSTC datasets show that SPPW outperforms baselines and is effective in mitigating overfitting and mislabeling problems.

Text classification is a fundamental task in web content mining. Although the existing supervised contrastive learning (SCL) approach combined with pre-trained language models (PLMs) has achieved leading performance in text classification, it lacks fundamental principles. Theoretically motivated by a derived lower bound of mutual information maximization, we propose a dual contrastive learning framework DualCL that satisfies three properties, i.e., parameter-free, augmentation-easy and label-aware. DualCL generates classifier parameters from the PLM and simultaneously uses them for classification and as augmented views of the input text for supervised contrastive learning. Extensive experiments conclusively demonstrate that DualCL excels in learning superior text representations and consistently outperforms baseline models.

Author name disambiguation (AND) is an essential task for online academic retrieval systems. Recent models adopt representation learning in the author's name disambiguation. Despite achieving remarkable success, these methods may be limited in two aspects. First, the heuristically constructed paper association graphs used for representation learning contain uncertainties that may cause negative supervision. Second, existing algorithms, such as binary cross-entropy loss, used to train representation learning models may not produce sufficiently high-quality representations for AND. To tackle the above problems, we propose an association refining and compositional contrasting (ARCC) framework for AND tasks. ARCC first adopts an iterative graph structure refinement process to dynamically reduce the uncertainties in paper graphs. Then, a compositional contrastive learning method is proposed to encourage learning more discriminative representations for AND. Empirical studies on two benchmark datasets suggest that ARCC is effective for AND and outperforms the state-of-the-art models.

Meta-learning has recently promoted few-shot text classification, which identifies target classes based on information transferred from source classes through a series of small tasks or episodes. Existing works constructing their meta-learner on Prototypical Networks need improvement in learning discriminative text representations between similar classes that may lead to conflicts in label prediction. The overfitting problems caused by a few training instances need to be adequately addressed. In addition, efficient episode sampling procedures that could enhance few-shot training should be utilized. To address the problems mentioned above, we first present a contrastive learning framework that simultaneously learns discriminative text representations via supervised contrastive learning while mitigating the overfitting problem via unsupervised contrastive regularization, and then we build an efficient self-paced episode sampling approach on top of it to include more difficult episodes as training progresses. Empirical results on eight few-shot text classification datasets show that our model outperforms the current state-of-the-art models. The extensive experimental analysis demonstrates that our supervised contrastive representation learning and unsupervised contrastive regularization techniques improve the performance of few-shot text classification. The episode-sampling analysis reveals that our self-paced sampling strategy improves training efficiency.

Semi-supervised text classification (STC) has been extensively researched and reduces human annotation. However, existing research assuming that unlabeled data only contains in-distribution texts is unrealistic. This paper extends STC to a more practical Open-set Semi-supervised Text Classification (OSTC) setting, which assumes that the unlabeled data contains out-of-distribution (OOD) texts. The main challenge in OSTC is the false positive inference problem caused by inadvertently including OOD texts during training. To address the problem, we first develop baseline models using outlier detectors for hard OOD-data filtering in a pipeline procedure. Furthermore, we propose a Latent Outlier Softening (LOS) framework that integrates semi-supervised training and outlier detection within probabilistic latent variable modeling. LOS softens the OOD impacts by the Expectation-Maximization (EM) algorithm and weighted entropy maximization. Experiments on 3 created datasets show that LOS significantly outperforms baselines.

Word Sense Disambiguation (WSD) which aims to identify the correct sense of a target word appearing in a specific context is essential for web text analysis. The use of glosses has been explored as a means for WSD. However, only a few works model the correlation between the target context and gloss. We add to the body of literature by presenting a model that employs a multi-head attention mechanism on deep contextual features of the target word and candidate glosses to refine the target word embedding. Furthermore, to encourage the model to learn the relevant part of target features that align with the correct gloss, we recursively alternate attention on target word features and that of candidate glosses to gradually extract the relevant contextual features of the target word, refining its representation and strengthening the final disambiguation results. Empirical studies on the five most commonly used benchmark datasets show that our proposed model is effective and achieves state-of-the-art results.

Multi-label text classification (MLTC) has a wide range of real-world applications. Neural networks recently promoted the performance of MLTC models. Training these neural-network models relies on sufficient accurately labelled data. However, manually annotating large-scale multi-label text classification datasets is expensive and impractical for many applications. Weak supervision techniques have thus been developed to reduce the cost of annotating text corpus. However, these techniques introduce noisy labels into the training data and may degrade the model performance. This paper aims to deal with such noise-label problems in MLTC in both single-instance and multi-instance settings. We build a novel Neural Expectation-Maximization Framework (nEM) that combines neural networks with probabilistic modelling. The nEM framework produces text representations using neural-network text encoders and is optimized with the Expectation-Maximization algorithm. It naturally considers the noisy labels during learning by iteratively updating the model parameters and estimating the distribution of the ground-truth labels. We evaluate our nEM framework in multi-instance noisy MLTC on a benchmark relation extraction dataset constructed by distant supervision and in single-instance noisy MLTC on synthetic noisy datasets constructed by keywords supervision and label flipping. The experimental results demonstrate that nEM significantly improves upon baseline models in both single-instance and multi-instance noisy MLTC tasks. The experiment analysis suggests that our nEM framework efficiently reduces the noisy labels in MLTC datasets and significantly improves model performance.

Entity alignment is a vital task in knowledge fusion, which aims to align entities from different knowledge graphs and merge them into one single graph. Existing entity alignment models focus on local features and try to minimize the distance between pairs of pre-aligned entities. Despite their success, these models heavily rely on the number of existing pre-aligned entity pairs and the topology information from the rest large set of unaligned entities is still largely unexplored. To overcome the limitation of existing models, we propose a model, termed Global Alignment and Local Information Aggregation, or GALA. GALA constructs global features for the knowledge graphs to be aligned using entity embeddings. It aligns the entities in the graphs by forcing their global features to match with each other and progressively updating the entity embeddings by aggregating local information from the other network. Empirical studies on commonly-used KG alignment data sets confirm the effectiveness of the proposed model.

Constructing a robust conversation graph based false information detection model is crucial for real social platforms. Recently, graph neural network (GNN) methods for false information detection have achieved significant advances. However, we empirically find that slight perturbations in the conversation graph can cause the predictions of existing models to collapse. To address this problem, we present RDCL, a contrastive learning framework for false information detection on social networks, to obtain robust detection results. RDCL leverages contrastive learning to maximize the consistency between perturbed graphs from the same original graph and minimize the distance between perturbed and original graphs from the same class, forcing the model to improve resistance to data perturbations. Moreover, we prove the importance of hard positive samples for contrastive learning and propose a hard positive sample pairs generation method (HPG) for conversation graphs, which can generate stronger gradient signals to improve the contrastive learning effect and make the model more robust. Experiments on various GNN encoders and datasets show that RDCL outperforms the current state-of-the-art models.