Large Language Models demonstrate strong reasoning capabilities, which can be effectively compressed into smaller models. However, existing datasets and fine-tuning approaches still face challenges that lead to catastrophic forgetting, particularly for models smaller than 8B. First, most datasets typically ignore the relationship between training data knowledge and the model's inherent abilities, making it difficult to preserve prior knowledge. Second, conventional training objectives often fail to constrain inherent knowledge preservation, which can result in forgetting of previously learned skills. To address these issues, we propose a comprehensive solution that alleviates catastrophic forgetting from both the data and fine-tuning approach perspectives. On the data side, we construct a dataset of 5K instances that covers multiple reasoning tasks and incorporates metacognitive knowledge, making it more tolerant and effective for distillation into smaller models. We annotate the metacognitive knowledge required to solve each question and filter the data based on task knowledge and the model's inherent skills. On the training side, we introduce GDPO (Group Direction Preference Optimization), which is better suited for resource-limited scenarios and can efficiently approximate the performance of GRPO. Guided by the large model and by implicitly constraining the optimization path through a reference model, GDPO enables more effective knowledge transfer from the large model and constrains excessive parameter drift. Extensive experiments demonstrate that our approach significantly alleviates catastrophic forgetting and improves reasoning performance on smaller models.

Multimodal aspect-based sentiment classification (MABSC) requires fine-grained reasoning over both textual and visual content to infer sentiments toward specific aspects.However, existing methods often rely on superficial correlations-particularly between aspect terms and sentiment labels-leading to poor generalization and vulnerability to spurious cues.To address this limitation, we propose DPCI, a novel Dual-Path Counterfactual Integration framework that enhances model robustness by explicitly modeling counterfactual reasoning in multimodal contexts.Specifically, we design a dual counterfactual generation module that simulates two types of interventions: replacing aspect terms and rewriting descriptive content, thereby disentangling the spurious dependencies from causal sentiment cues.We further introduce a sample-aware counterfactual selection strategy to retain high-quality, diverse counterfactuals tailored to each generation path.Finally, a confidence-guided integration mechanism adaptively fuses counterfactual signals into the main prediction stream.Extensive experiments on standard MABSC benchmarks demonstrate that DPCI not only achieves stateof-the-art performance but also significantly improves model robustness.

Multimodal aspect-based sentiment classification (MABSC) is a challenging task emerging in recent years, which aims to combine text and image to identify the sentiment polarity of each aspect. There exists a potential irrelevance between aspects and images, and mistakenly focusing on irrelevant image regions will introduce redundant and misalignment noise. Besides, existing methods implicitly mix visual and textual features, which may lead to the loss or blurring of modality-specific information. To address these challenges, we propose a Bottleneck-constrained Contrastive Decoupled Network (BCDN) for the MABSC task. We first design a bottleneck-constrained visual consistency module to reduce redundancy and misalignment noise in aspect-related visual features. Additionally, we employ modality decoupling to fully capture inter-modality knowledge. Specifically, we first decouple the aspect-related visual and textual representations into modality-invariant and modality-specific features. Afterwards, we propose novel contrastive regularizations to optimize the decoupled features. Extensive experiments on benchmark datasets demonstrate that our BCDN achieves superior performance and verify the effectiveness of our BCDN. The codes are released at https://github.com/ruiliu2020/BCDN.

Multimodal recommendation, which utilizes rich multimodal information to learn user preferences, has attracted significant attention. Most works focus on designing powerful encoders for extracting multimodal features, and simply aggregate the learned features together to make prediction. Consequently, they have a limited capacity to learn the inter-modality knowledge including the modality-shared and modality-unique knowledge. In fact, learning the modality-shared knowledge enables us to align cross-modality data for fusing heterogeneous modality features. Learning the modality-unique knowledge is equally important when recommendation tasks only involve a small amount of shared features and the necessary information is contained within specific modality. In this article, we propose Contrastive Modality-Disentangled Learning (CMDL) to overcome this critical limitation. CMDL exactly captures the inter-modality knowledge by achieving modality disentanglement. Specifically, CMDL first disentangles the initial representation into the modality-invariant and modality-specific representations. Afterwards, CMDL introduces a novel manner of contrastive learning to approximate the MI upper bounds for achieving disentanglement regularization. Building upon the proposed regularization, CMDL encourages the modality-invariant and modality-specific representations to capture the modality-shared and modality-unique knowledge respectively and to be statistically independent to each other. Empirically, extensive experiments are conducted on benchmark datasets, demonstrating the superior performance of CMDL compared with strong multimodal recommenders.

Aspect Sentiment Triplet Extraction (ASTE) is an emerging task to extract a given sentence's triplets, which consist of aspects, opinions, and sentiments. Recent studies tend to address this task with a table-filling paradigm, wherein word relations are encoded in a two-dimensional table, and the process involves clarifying all the individual cells to extract triples. However, these studies ignore the deep interaction between neighbor cells, which we find quite helpful for accurate extraction. To this end, we propose a novel model for the ASTE task, called Prompt-based Tri-Channel Graph Convolution Neural Network (PT-GCN), which converts the relation table into a graph to explore more comprehensive relational information. Specifically, we treat the original table cells as nodes and utilize a prompt attention score computation module to determine the edges' weights. This enables us to construct a target-aware gridlike graph to enhance the overall extraction process. After that, a triple-channel convolution module is conducted to extract precise sentiment knowledge. Extensive experiments on the benchmark datasets show that our model achieves state-of-the-art performance. The code is available at https://github.com/KunPunCN/PT-GCN.

Stance detection aims to identify the attitude from an opinion towards a certain target. Despite the significant progress on this task, it is extremely time-consuming and budget-unfriendly to collect sufficient high-quality labeled data for every new target under fully-supervised learning, whereas unlabeled data can be collected easier. Therefore, this paper is devoted to few-shot stance detection and investigating how to achieve satisfactory results in semi-supervised settings. As a target-oriented task, the core idea of semi-supervised few-shot stance detection is to make better use of target-relevant information from labeled and unlabeled data. Therefore, we develop a novel target-aware semi-supervised framework. Specifically, we propose a target-aware contrastive learning objective to learn more distinguishable representations for different targets. Such an objective can be easily applied with or without unlabeled data. Furthermore, to thoroughly exploit the unlabeled data and facilitate the model to learn target-relevant stance features in the opinion content, we explore a simple but effective target-aware consistency regularization combined with a self-training strategy. The experimental results demonstrate that our approach can achieve state-of-the-art performance on multiple benchmark datasets in the few-shot setting.

Zero-shot and few-shot stance detection (ZFSD) aims to automatically identify the users’ stance toward a wide range of continuously emerging targets without or with limited labeled data. Previous works on in-target and cross-target stance detection typically focus on extremely limited targets, which is not applicable to the zero-shot and few-shot scenarios. Additionally, existing ZFSD models are not good at modeling the relationship between seen and unseen targets. In this paper, we propose a unified end-to-end framework with a discrete latent topic variable that implicitly establishes the connections between targets. Moreover, we apply supervised contrastive learning to enhance the generalization ability of the model. Comprehensive experiments on the ZFSD task verify the effectiveness and superiority of our proposed method.

Conversational Causal Emotion Entailment aims to detect causal utterances for a non-neutral targeted utterance from a conversation. In this work, we build conversations as graphs to overcome implicit contextual modelling of the original entailment style. Following the previous work, we further introduce the emotion information into graphs. Emotion information can markedly promote the detection of causal utterances whose emotion is the same as the targeted utterance. However, it is still hard to detect causal utterances with different emotions, especially neutral ones. The reason is that models are limited in reasoning causal clues and passing them between utterances. To alleviate this problem, we introduce social commonsense knowledge (CSK) and propose a Knowledge Enhanced Conversation graph (KEC). KEC propagates the CSK between two utterances. As not all CSK is emotionally suitable for utterances, we therefore propose a sentiment-realized knowledge selecting strategy to filter CSK. To process KEC, we further construct the Knowledge Enhanced Directed Acyclic Graph networks. Experimental results show that our method outperforms baselines and infers more causes with different emotions from the targeted utterance.

Jiahao Cao, Rui Liu, Huailiang Peng, Lei Jiang, Xu Bai. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task designed to identify the polarity of a target aspect. Some works introduce various attention mechanisms to fully mine the relevant context words of different aspects, and use the traditional cross-entropy loss to fine-tune the models for the ABSA task. However, the attention mechanism paying partial attention to aspect-unrelated words inevitably introduces irrelevant noise. Moreover, the cross-entropy loss lacks discriminative learning of features, which makes it difficult to exploit the implicit information of intra-class compactness and inter-class separability. To overcome these challenges, we propose an Aspect Feature Distillation and Enhancement Network (AFDEN) for the ABSA task. We first propose a dual-feature extraction module to extract aspect-related and aspect-unrelated features through the attention mechanisms and graph convolutional networks. Then, to eliminate the interference of aspect-unrelated words, we design a novel aspect-feature distillation module containing a gradient reverse layer that learns aspect-unrelated contextual features through adversarial training, and an aspect-specific orthogonal projection layer to further project aspect-related features into the orthogonal space of aspect-unrelated features. Finally, we propose an aspect-feature enhancement module that leverages supervised contrastive learning to capture the implicit information between the same sentiment labels and between different sentiment labels. Experimental results on three public datasets demonstrate that our AFDEN model achieves state-of-the-art performance and verify the effectiveness and robustness of our model.

Keyphrase prediction is a crucial task that can effectively provide underlying support for numerous downstream Natural Language Processing (NLP) tasks, e.g., information retrieval and document summarization. Existing keyphrase prediction approaches mostly focus on either extractive or generative methods. Extractive methods directly extract keyphrases that present in the document, while they cannot obtain absent keyphrases. Generative methods are designed to generate both present and absent keyphrases. However, the absent keyphrases are generated at the cost of hurting the performance of the present keyphrase prediction. The generation of present keyphrases mainly relies on the copying mechanism, ignoring the interdependence of the overall decisions. In contrast, the extractive model that directly extracts a text span from the document is more suitable for predicting the present keyphrase. Therefore, it is necessary to coordinate the extractive and generative patterns to obtain accurate and comprehensive keyphrases. Specifically, we divide the keyphrase prediction into two subtasks, i.e., present keyphrase extraction (PKE) and absent keyphrase generation (AKG), and propose a joint inference framework to exploit their respective advantages fully. For PKE, we treat it as a sequence labeling problem and apply a BERT-based sentence selector to select salient sentences that contain present keyphrases. For AKG, we introduce a Transformer-based architecture equipped with a gated fusion attention module, which fully integrates the present keyphrase knowledge learned from PKE by the fine-tuned BERT. The experimental results demonstrate that our approach can achieve state-of-the-art performance on all benchmark datasets.

Yanfu Xu, Zheng Lin, Yuanxin Liu, Rui Liu, Weiping Wang, Dan Meng. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.