Facial expression recognition plays a crucial role in understanding human emotions and behavior. However, existing models often exhibit biases and imbalance towards diverse expression classes. To address this problem, we propose an Adaptive Mask-Guide Supervised Network (AMGSN) to enhance the uniform performance of the facial expression recognition models. We propose an adaptive mask guidance mechanism to mitigate bias and ensure uniform performance across different expression classes. AMGSN focuses on learning the ability to distinguish facial features with under-expressed expressions by dynamically generating masks during pre-training. Specifically, we employ an asymmetric encoder–decoder architecture, where the encoder encodes only the unmasked visible regions, while the lightweight decoder reconstructs the original image using latent representations and mask markers. By utilizing dynamically generated masks and focusing on informative regions, these models effectively reduce the interference of confounding factors, thus enhancing the discriminative power of the learned representation. In the pre-training stage, we introduce the Attention-Based Mask Generator (ABMG) to identify salient regions of expressions. Additionally, we advance the Mask Ratio Update Strategy (MRUS), which utilizes image reconstruction loss, to adjust the mask ratio for each image during pre-training. In the finetune stage, debiased center loss and contrastive loss are introduced to optimize the network to ensure the overall performance of expression recognition. Extensive experimental results on several standard datasets demonstrate that the proposed AMGSN significantly improves both balance and accuracy compared to state-of-the-art methods. For example, AMGSN reached 89.34% on RAF-DB, and 62.83% on AffectNet, respectively, with a standard deviation of only 0.0746 and 0.0484. This demonstrates the effectiveness of our improvements 1 .

Mitigating human-like biases and social stereotypes in pre-trained language models (PLMs) has become a crucial task in Chinese toxic speech detection. While PLMs have achieved state-of-the-art results in mitigating explicit bias (e.g., bias on sensitive attribute words), the study of implicit bias (e.g., bias against specific demographic groups) is still under-explored. Besides, existing debiasing methods focus on the tradeoff between debiasing efficiency and model performance, while ignoring robustness against noisy data. Therefore, a debiasing method that effectively reduces biases while maintaining robustness against noisy data is needed. In this article, we propose Unbiased Contrastive Learning (UCL), which can mitigate explicit and implicit bias while maintaining robustness to noisy data. Specifically, we first analyze the bias representation problem constrained by contrastive learning objective and implement unbiased contrastive objective for learning unbiased text representations to mitigate explicit and implicit biases in Chinese toxic speech detection tasks. UCL inherits the idea of supervised contrastive learning, which encourages representations of the same sensitive attribute to be closer than those of different sensitive attributes and ensures unbiasedness by penalizing the sensitive attribute information contained in the representations. Furthermore, we design conditional normalization to reduce biased classification caused by the imbalanced distribution of demographic groups in the data. Experimental results on Chinese and English datasets show that the proposed method outperforms the state-of-the-art methods and achieves the competitive performance.

Unnatural text correction aims to automatically detect and correct spelling errors or adversarial perturbation errors in sentences. Existing methods typically rely on fine-tuning or adversarial training to correct errors, which have achieved significant success. However, these methods exhibit poor generalization performance due to the difference in data distribution between training data and real-world scenarios, known as the exposure bias problem. In this paper, we propose a self-correct adversarial training framework for learning from mistakes (LIMIT), which is a task- and model-independent framework to correct unnatural errors or mistakes. Specifically, we fully utilize errors generated by the model that are actively exposed during the inference phase, i.e., predictions that are inconsistent with the target. This training method not only simulates potential errors in real application scenarios, but also mitigates the exposure bias of the traditional training process. Meanwhile, we design a novel decoding intervention strategy to maintain semantic consistency. Extensive experimental results on Chinese unnatural text error correction datasets show that our proposed method can correct multiple forms of errors and outperforms the state-of-the-art text correction methods. In addition, extensive results on Chinese and English datasets validate that LIMIT can serve as a plug-and-play defense module and can extend to new models and datasets without further training.

Current Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in understanding multimodal data, but their potential remains underexplored for deepfake detection due to the misalignment of their knowledge and forensics patterns. To this end, we present a novel framework that unlocks LVLMs’ potential capabilities for deepfake detection. Our framework includes a Knowledge-guided Forgery Detector (KFD), a Forgery Prompt Learner (FPL), and a Large Language Model (LLM). The KFD is used to calculate correlations between image features and pristine/deepfake image description embeddings, enabling forgery classification and localization. The outputs of the KFD are subsequently processed by the Forgery Prompt Learner to construct fine-grained forgery prompt embeddings. These embeddings, along with visual and question prompt embeddings, are fed into the LLM to generate textual detection responses. Extensive experiments on multiple benchmarks, including FF++, CDF2, DFD, DFDCP, DFDC, and DF40, demonstrate that our scheme surpasses state-of-the-art methods in generalization performance, while also supporting multi-turn dialogue capabilities.

Harmful meme detection presents a significant multimodal challenge that necessitates contextual background knowledge and comprehensive inference. Although some research studies have been related to harmful meme detection in English, detecting harmful memes in Chinese is also an unresolved issue. In this paper, to bridge this gap, we constructed a Chinese harmful meme detection dataset, named CHMEMES. Furthermore, existing multimodal alignment methods have shown poor performance in tasks involving harmful meme detection, where there is a mismatch between the image and text components. To improve the task, we propose a multimodal framework Semantic Contrastive Alignment fRamEwork (SCARE), which enables fully representing both cross-modal and intra-modal information. For cross-modal information, we introduce a cross-modal contrast alignment objective to maximize the mutual information between image and text. For intra-modal information, we design a new intra-modal contrast objective to achieve more robust visual and textual representation learning. Moreover, we present a simple yet efficient vision prompt tuning paradigm for parameter-efficient harmful meme detection. We conduct extensive experiments on the constructed Chinese dataset and the existing English dataset. Experimental results show that our method outperforms state-of-the-art baselines in harmful meme detection.

The powerful ability of ChatGPT has caused widespread concern in the academic community. Malicious users could synthesize dummy academic content through ChatGPT, which is extremely harmful to academic rigor and originality. The need to develop ChatGPT-written content detection algorithms calls for large-scale datasets. In this paper, we initially investigate the possible negative impact of ChatGPT on academia, and present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) to support the development of detection algorithms. In particular, the ChatGPT-written abstract dataset contains 35,304 synthetic abstracts, with$Generation$,$Polish$, and$Fusion$as prominent representatives. Based on these data, we perform a thorough analysis of the existing text synthesis detection algorithms. We show that ChatGPT-written abstracts are detectable with well-trained detectors, while the detection difficulty increases with more human guidance involved.

Multi-domain fake news detection aims to identify whether various news from different domains is real or fake and has become urgent and important. However, existing methods are dedicated to improving the overall performance of fake news detection, ignoring the fact that unbalanced data leads to disparate treatment for different domains, i.e., the domain bias problem. To solve this problem, we propose the Dual-Teacher De-biasing Distillation framework (DTDBD) to mitigate bias across different domains. Following the knowledge distillation methods, DTDBD adopts a teacher-student structure, where pre-trained large teachers instruct a student model. In particular, the DTDBD consists of an unbiased teacher and a clean teacher that jointly guide the student model in mitigating domain bias and maintaining performance. For the unbiased teacher, we introduce an adversarial de-biasing distillation loss to instruct the student model in learning unbiased domain knowledge. For the clean teacher, we design domain knowledge distillation loss, which effectively incentivizes the student model to focus on representing domain features while maintaining performance. Moreover, we present a momentum-based dynamic adjustment algorithm to trade off the effects of two teachers. Extensive experiments on Chinese and English datasets show that the proposed method substantially outperforms the state-of-the-art baseline methods in terms of bias metrics while guaranteeing competitive performance11Our codes are available at https://github.com/ningljy/DTDBD.

Conversational Recommendation Systems (CRSs) enable recommender systems to explicitly acquire user preferences during multi-turn interactions, providing more accurate and personalized recommendations. However, the data imbalance in CRSs, due to inconsistent interaction history among users, may lead to disparate treatment for disadvantaged user groups. In this paper, we investigate the discriminate problems in CRS from the user’s perspective, called as user-oriented fairness. To reveal the unfairness problems of different user groups in CRS, we conduct extensive empirical analyses. To mitigate user unfairness, we propose a user-oriented fairness framework, named FairCRS, which is a model-agnostic framework. In particular, we develop a user-embedding reconstruction mechanism that enriches user embeddings by incorporating more interaction information, and design a user-oriented fairness strategy that optimizes the recommendation quality differences among user groups while alleviating unfairness. Extensive experimental results on English and Chinese datasets show that FairCRS outperforms state-of-the-art CRSs in terms of overall recommendation performance and user fairness.

Non-normative texts and euphemisms are widely spread on the Internet, making it more difficult to moderate the content. These phenomena result from misspelling errors or deliberate textual attacks by users. However, current methods lack a robust design and are vulnerable to adversarial attacks. Thus, exploring robust text correction is of great significance. Text correction aims to automatically detect and correct errors in sentences, which can be employed as a defensive method against adversarial attacks. In this work, we provide PROTECT: a robust Chinese language model for more general Chinese text correction. In particular, we propose a simple but effective self-supervised learning approach for calibrating generalizable representations, which is robust to adapt to the changing adversarial texts. S pecifically, we develop an adversarial-aware multi-feature representation method that establishes auxiliary supervision. To the best of our knowledge, the PROTECT is the first model to represent perfect pinyin, abbreviation pinyin, character split, visual, and phonetic features simultaneously. Based on the generated adversarial examples, PROTECT is trained from scratch through a unified text-to-text generation paradigm. This empowers the model to correct multi-type text errors and inconsistent lengths simultaneously. After obtaining adversarially robust representations, we design a novel parameter-efficient tuning method consisting of context-specific adaptive prefix and semantic-consistent low-rank adaptation modules to implement zero-shot and few-shot learning. Extensive experimental results have shown that by tuning the parameters by 0.2%, PROTECT achieves best performance in the full-data and in the low-resource setting.

Ethical understanding aims at morally analyzing and discriminating ethical scenarios described in natural language. By classifying behaviors that occur in ethical scenarios as ethical or unethical, ethical understanding empowers artificial intelligence systems to understand human values so as to discern right from wrong morally. However, most existing ethical understanding methods lack fine-grained analysis and cannot handle the problem that an ethical scenario may contain multiple behaviors with multiple polarities. In this paper, we introduce a novel natural language processing task, behavior-based ethical understanding (BEU), for mining the purpose relation(s) and ethical polarity of a specific behavior from the social news. It contains three subtasks: behavior term extraction (BTE) to extracts behavior terms, purpose relation inference (PRI) to identifies purposive relations among behaviors, and polarity discrimination (PD) to predicts the ethical polarities of behaviors, respectively. To perform this task, we constructed a Chinese BEU dataset, named FG-ETHICS. Besides, we propose a three-stage framework, BEU-BERT, based on the pre-trained language model BERT and deliberately designed downstream models for three subtasks. Experimental results show that the proposed framework achieves the best performance from the BTE and PD tasks, and achieves a promising performance of 75% on the PRI task.