The rapid growth of online short texts has made specialized analysis essential, as these texts are sparse and information-limited. Short text clustering (STC) is critical for automatically grouping unlabeled texts into meaningful clusters, supporting applications such as sentiment analysis, spam filtering, and social media personalization. In the context of massive online content, deep clustering seeks to uncover semantic categories by measuring distances in the representation space. Consequently, aligning clustering pseudo-labels with the true category distribution is crucial for effective self-supervised training, particularly under class imbalance and distribution skew commonly observed in web data. To address this challenge, we propose the Cluster Identification-Guided Dual Correction (CIDC) framework, which generates reliable pseudo-labels to guide deep clustering. Specifically, given cluster partitions and model-estimated class distributions, we perform Cluster Category Identification (CCI) at each training epoch to determine the most probable category for each cluster. This identification provides the foundation for the Pseudo-Label Correction (PLC) and Prototype-Based Correction (PBC) modules, which jointly enhance pseudo-label reliability and representation learning. In the PLC module, samples whose model-estimated class distributions conflict with the assigned cluster category are corrected, thereby improving semantic alignment within clusters. In the PBC module, representative and reliable prototypes are selected according to cluster categories and model predictions to guide training, further strengthening representation discriminability. Extensive experiments demonstrate that CIDC consistently outperforms existing methods in terms of clustering accuracy and mutual information, particularly in unsupervised settings characterized by class imbalance and noisy data.

Deep learning models usually require large labeled datasets to generalize well, but this is computationally and financially costly. Cold-start few-shot data selection enables fast model generalization by selecting a few diverse, representative samples from an unlabeled data pool. To achieve this goal, previous work usually divides the training data into several clusters and performs sampling from these clusters. Yet, such a way tends to have two issues. First, imbalanced data distribution in the training data pool still exists in the selected subset, causing models' performance biases and suboptimal generalization ability. Second, these methods improve sample diversity in each cluster by considering either the feature dissimilarity among instances, or model uncertainty for individual instance. They ignore the entire representativeness of samples within a cluster. To tackle these challenges, we propose a novel framework HCDS : Hierarchical Clustering for Cold-Start Few-Shot Data Selection. Specifically, we first perform class-level clustering, using pseudo-labels for class supervision and applying contrastive clustering to derive class-rich features. We then refine these features within the class-level clusters into semantically meaningful features and perform representation-level clustering. Finally, we sample data from the representation-level clusters based on global similarity to ensure representativeness. Experimental results on six public datasets, including both balanced and imbalanced ones, show that HCDS achieves state-of-the-art performance, particularly with limited and imbalanced data.

Information extraction (IE) aims to extract meaningful structured tuples from unstructured text. Existing studies usually utilize a pre-trained generative language model that rephrases the original sentence into a target sequence, which can be easily decoded as tuples. However, traditional evaluation metrics treat a slight error within the tuple as an entire prediction failure, which is unable to perceive the correctness extent of a tuple. For this reason, we first propose a novel IE evaluation metric called Matching Score to evaluate the correctness of the predicted tuples in more detail. Moreover, previous works have ignored the effects of semantic uncertainty when focusing on the generation of the target sequence. We argue that leveraging the built-in semantic uncertainty of language models is beneficial for improving its robustness. In this work, we propose Binomial distribution guided counterpart sequence (BCS) method, which is a model-agnostic approach. Specifically, we propose to quantify the built-in semantic uncertainty of the language model by bridging all local uncertainties with the whole sequence. Subsequently, with the semantic uncertainty and Matching Score, we formulate a unique binomial distribution for each local decoding step. By sampling from this distribution, a counterpart sequence is obtained, which can be regarded as a semantic complement to the target sequence. Finally, we employ the Kullback-Leibler divergence to align the semantics of the target sequence and its counterpart. Extensive experiments on 14 public datasets over 5 information extraction tasks demonstrate the effectiveness of our approach on various methods. Our code and dataset are available at https://github.com/byinhao/BCS.

Named Entity Recognition (NER) serves as a fundamental task in natural language understanding, bearing direct implications for web content analysis, search engines, and information retrieval systems. Fine-tuned NER models exhibit satisfactory performance on standard NER benchmarks. However, due to limited fine-tuning data and lack of knowledge, it performs poorly on unseen entity recognition. As a result, the usability and reliability of NER models in web-related applications are compromised. Instead, Large Language Models (LLMs) like GPT-4 possess extensive external knowledge, but research indicates that they lack specialty for NER tasks. Furthermore, non-public and large-scale weights make tuning LLMs difficult. To address these challenges, we propose a framework that combines small fine-tuned models with LLMs (LinkNER) and an uncertainty-based linking strategy called RDC that enables fine-tuned models to complement black-box LLMs, achieving better performance. We experiment with both standard NER test sets and noisy social media datasets. LinkNER enhances NER task performance, notably surpassing SOTA models in robustness tests. We also quantitatively analyze the influence of key components like uncertainty estimation methods, LLMs, and in-context learning on diverse NER tasks, offering specific web-related recommendations.