While RAG systems are designed to enhance factual fidelity by grounding LLMs in provided sources, the application of current watermarking techniques creates a conflict.These methods, being inherently fact-agnostic, force the model to deviate from the very source documents it is supposed to follow.This leads to "faithfulness hallucinations", which refer to a critical flaw where the generated output contradicts its own grounding context.Consequently, these watermarks undermine the core value of RAG, rendering even the most secure schemes untrustworthy for high-stakes applications.To resolve this RAG-specific conflict, we introduce the Dual Factual Shield (DFS), a three-stage post-hoc pipeline for factualitypreserving watermarking in RAG.It adopts a defense-in-depth design that combines a sourceanchored algorithmic safeguard for protecting critical tokens from retrieved context with prompt-based semantic guidance to mitigate factual corruption.Experiments show that our framework drastically reduces the Knowledge Corruption Rate (KCR), a new metric we introduce to quantify factual fidelity, while maintaining strong security and robustness, paving the way for responsible deployment of traceable AI in knowledge-critical domains.

Backdoor attacks pose a significant threat to large language models (LLMs) by embedding malicious triggers that manipulate model behavior. However, existing defenses primarily rely on prior knowledge of backdoor triggers or targets and offer only superficial mitigation strategies, thus struggling to fundamentally address the inherent reliance on unreliable features. To address these limitations, we propose a novel defense strategy, \textit{RepGuard}, that strengthens LLM resilience by adaptively separating abnormal features from useful semantic representations, rendering the defense agnostic to specific trigger patterns. Specifically, we first introduce a dual-perspective feature localization strategy that integrates local consistency and sample-wise deviation metrics to identify suspicious backdoor patterns. Based on this identification, an adaptive mask generation mechanism is applied to isolate backdoor-targeted shortcut features by decomposing hidden representations into independent spaces, while preserving task-relevant semantics. With a multi-objective optimization framework, our method can inherently mitigates backdoor attacks. Across \textit{Target Refusal} and \textit{Jailbreak} tasks under four types of attacks, RepGuard consistently reduced the attack success rate on poisoned data by nearly 80\% on average, while maintaining near-original task performance on clean data. Extensive experiments demonstrate that RepGuard provides a scalable and interpretable solution for safeguarding LLMs against sophisticated backdoor threats.

Alert underreporting presents a significant challenge to the reconstruction of attack chains, as it often leads to the absence of critical information necessary for fully presenting the entire attack. To address this issue, this paper proposes an approach for attack chain context inference and completion based on Large Language Models. By integrating an attack knowledge base, this approach leverages LLM-driven inference to identify missing attack stages and uncover potential attack behaviors. Experimental results demonstrate that this approach can effectively detect omitted alerts and complete the attack chain, thereby enhancing the integrity of attack detection.

Machine Learning (ML) based malware classifiers are vulnerable to exploitation during the training phase due to the necessity of regular retraining with samples collected from the wild. Recent studies have highlighted the efficacy of backdoor attacks in the malware domain, where attackers can manipulate the model during training by injecting samples embedded with specific triggers, causing the model to establish an association between the trigger and a designated class, thereby achieving evasion of detection. While research on backdoor attacks has been extensively explored in the field of computer vision, it has been largely overlooked in the malware domain. Unlike in the computer vision domain, the threat model in the malware domain typically restricts attackers to employing clean-label attacks (i.e., attackers do not have control over the labeling of poisoned data). However, clean-label attack methods are generally less effective compared to those that involve embedding triggers and altering sample labels to the target class (called corrupted-label attacks). To address this limitation, we propose a simple yet effective method that involves Poisoning Malware-Similar Benignware (PMSB) instead of random selection, thereby approximating the scenario of corrupted-label attacks and enhancing the effectiveness of clean-label attacks. Additionally, we introduce three similarity measurement methods based on feature-based distance, distribution-based distance, and contribution-based difference to select malware-similar benignware. Comprehensive evaluations across three different trigger types and three datasets demonstrate the superiority and general applicability of PMSB.

Large Language Models (LLMs) have achieved impressive results in Machine Translation by simply following instructions, even without training on parallel data. However, LLMs still face challenges on low-resource languages due to the lack of pre-training data. In real-world situations, humans can become proficient in their native languages through abundant and meaningful social interactions and can also learn foreign languages effectively using well-organized textbooks. Drawing inspiration from human learning patterns, we introduce the Translate After LEarNing Textbook (TALENT) approach, which aims to enhance LLMs’ ability to translate low-resource languages by learning from a textbook. TALENT follows a step-by-step process: (1) Creating a Textbook for low-resource languages. (2) Guiding LLMs to absorb the Textbook’s content for Syntax Patterns. (3) Enhancing translation by utilizing the Textbook and Syntax Patterns. We thoroughly assess TALENT’s performance using 112 low-resource languages from FLORES-200 with two LLMs: ChatGPT and BLOOMZ. Evaluation across three different metrics reveals that TALENT consistently enhances translation performance by 14.8% compared to zero-shot baselines. Further analysis demonstrates that TALENT not only improves LLMs’ comprehension of low-resource languages but also equips them with the knowledge needed to generate accurate and fluent sentences in these languages.

Community-based Question Answering (CQA) summarization aims to generate a summary from a collection of QA pairs about a specific entity. Unlike well-structured texts such as dialogues, a set of QA pairs often contains significant redundancy, including repetitive questions and similar answers. The above property of QA pairs makes it difficult for abstractive summarizers to concentrate on salient pieces, resulting in duplication of content and omission of important information. In this paper, we propose a Focus Rectification SUMmarizer (FRSum), which employs different strategies at both the sentence level and token level to rectify the focus on representative QA pairs and distinctive tokens. The experimental results on the COQASUM dataset show that our model can generate concise and informative summaries, outperforming state-of-the-art baselines in automatic and human evaluations.

Visual dialog is a multi-turn on-going conversation, where a critical challenge is how to understand questions with co-reference ambiguities and reply with personality consistency through dialog. Previous works with history encoding inevitably introduce unexpected noise since the whole dialog history is considered via implicit attention. Moreover, there is still no works focus on the role of personality at present in visual dialog. In this paper, we conduct in-depth study of the dialog history and propose two novel modules. Specifically, in Explicit Co-Reference Resolution module, we perform co-reference resolution by modeling it as a sequence tagging task, which helps to locate the relevant history for the current question. In User Personality Modeling module, we make the first attempt to model the user’s interaction style in visual dialog in terms of how detailedly the user answered questions in the previous dialog rounds and generate the user preference score for each answer candidate. Note that both of our proposed modules are model-agnostic so they are applicable in any VisDial model. By applying these two modules in several representative baseline models, we show significant boosts on all the evaluation metrics, achieving new state-of-the-art results on VisDial v1.0 and even outperforming the pre-training models such as VD-BERT [1].

In today's digital age, accessing information across language barriers poses a significant challenge, with conventional search systems often struggling to interpret and retrieve multilingual content accurately. Addressing this issue, our study introduces a novel integration of applying Large Language Models (LLMs) as Cross-lingual Readers in information retrieval systems, specifically targeting the complexities of cross-lingual information retrieval (CLIR). We present an innovative approach: Activation Steered Multilingual Retrieval (ASMR) that employs "steering activations''-a method to adjust and direct the LLM's focus-enhancing its ability to understand user queries and generate accurate, language-coherent responses. ASMR adeptly combines a Multilingual Dense Passage Retrieval (mDPR) system with an LLM, overcoming the limitations of traditional search engines in handling diverse linguistic inputs. This approach is particularly effective in managing the nuances and intricacies inherent in various languages. Rigorous testing on established benchmarks such as XOR-TyDi QA, and MKQA demonstrates that ASMR not only meets but surpasses existing standards in CLIR, achieving state-of-the-art performance. The results of our research hold significant implications for understanding the inherent features of how LLMs understand and generate natural languages, offering an attempt towards more inclusive, effective, and linguistically diverse information access on a global scale.

Machine learning methods have shown significant advantages in detecting malicious traffic, particularly identifying zero-day attacks and unknown threats. However, existing detection methods based on statistical features are susceptible to deception and evasion by attackers, resulting in reduced detection accuracy and challenges in achieving fine-grained detection. To address these issues, we propose FREDet, a fine-grained malicious traffic detection method based on frequency domain features. FREDet employs discrete wavelet transform to extract frequency domain features from network traffic and incorporates a new traffic aggregation method, significantly enhancing feature representativeness and detection accuracy for robust, fine-grained detection. Experiments on a multi-type attack dataset demonstrate that FREDet can detect malicious traffic with high precision, outperforming existing state-of-the-art methods, achieving accuracies of over 98.63% and 99.93% in detecting fine-grained attack categories and known attack subcategories, respectively.

In the contemporary digital landscape, search engines play an invaluable role in information access, yet they often face challenges in Cross-Lingual Information Retrieval (CLIR). Though attempts are made to improve CLIR, current methods still leave users grappling with issues such as misplaced named entities and lost cultural context when querying in non-native languages. While some advances have been made using Neural Machine Translation models and cross-lingual representation, these are not without limitations. Enter the paradigm shift brought about by Large Language Models (LLMs), which have transformed search engines from simple retrievers to generators of contextually relevant information. This paper introduces the Multilingual Information Model for Intelligent Retrieval (MIMIR). Built on the power of LLMs, MIMIR directly responds in the language of the user's query, reducing the need for post-search translations. Our model's architecture encompasses a dual-module system: a retriever for searching multilingual documents and a responder for crafting answers in the user's desired language. Through a unique unified training framework, with the retriever serving as a reward model supervising the responder, and in turn, the responder producing synthetic data to refine the retriever's proficiency, MIMIR's retriever and responder iteratively enhance each other. Performance evaluations via CLEF and MKQA benchmarks reveal MIMIR's superiority over existing models, effectively addressing traditional CLIR challenges.

A critical challenge for open-domain dialogue agents is to generate persona-relevant and consistent responses. Due to the nature of persona sparsity in conversation scenarios, previous persona-based dialogue agents trained with Maximum Likelihood Estimation tend to overlook the given personas and generate responses irrelevant or inconsistent with personas. To address this problem, we propose a two-stage coarse-to-fine persona-aware training framework to improve the persona consistency of a dialogue agent progressively. Specifically, our framework first trains the dialogue agent to answer the constructed persona-aware questions, making it highly sensitive to the personas to generate persona-relevant responses. Then the dialogue agent is further trained with a contrastive learning paradigm by explicitly perceiving the difference between the consistent and the generated inconsistent responses, forcing it to pay more attention to the key persona information to generate consistent responses. By applying our proposed training framework to several representative baseline models, experimental results show significant boosts on both automatic and human evaluation metrics, especially the consistency of generated responses.

Large Language Models have shown great capability to comprehend natural language and provide reasonable responses. However, previous researches have shown weak performance of these models on low-resource (long-tail) languages. It remains to be a problem to mitigate the performance gap between long-tail languages and rich-resource ones, which is referred to as long-tail language representation collapsing. Though some previous works can generate pseudo-parallel corpora with the auto-regressive generation, this generation progress is time-consuming and remains low quality, particularly for long-tail languages. In this paper, we propose a (X) Cross-lingual Bootstrapped Unsupervised Fine-tuning Framework (X-BUFF) to mitigate long-tail language representation collapsing. X-BUFF iteratively updates cross-lingual PLMs in a curriculum way. In each iteration of X-BUFF, we (1) select sentences with complementary semantics from monolingual corpora in long-tail languages. (2) match these selected sentences with semantic equivalent sentences in many other languages to create parallel sentence pairs, which we then merge with previous sentence pairs to build a larger and more difficult bootstrapped parallel queue. (3) fine-tune the PLMs with the bootstrapped parallel queue. Extensive experiments show that X-BUFF can mitigate the long-tail language representation collapsing problem in cross-lingual PLMs and achieve significant improvements over the previous baselines on several cross-lingual evaluation benchmarks.

It is a critical challenge for open-domain dialogue agents to generate context-coherent responses which can present a consistent personality. However, existing methods mainly focus on the penalty of the persona-inconsistent responses, leaving out considering the context-incoherence problem caused by wrong persona selection. In this paper, we propose the Think-Before-You-Speak (TBYS) model, consisting of Concept-guided Persona Reasoning module and Consistent Dialogue Generation module, to explicitly select persona sentences semantically relevant to the current turn and generate responses based on the selection results. The experimental results on Persona-Chat show that TBYS can generate coherent and consistent responses, outperforming state-of-the-art baselines in both automatic and human evaluations.

Controllable story generation is a challenging task in the field of NLP, which has attracted increasing research interest in recent years. However, most existing works generate a whole story conditioned on the appointed keywords or emotions, ignoring the psychological changes of the protagonist. Inspired by psychology theories, we introduce global psychological state chains, which include the needs and emotions of the protagonists, to help a story generation system create more controllable and well-planned stories. In this paper, we propose a Psychology-guided Controllable Story Generation System (PICS) to generate stories that adhere to the given leading context and desired psychological state chains for the protagonist. Specifically, psychological state trackers are employed to memorize the protagonist’s local psychological states to capture their inner temporal relationships. In addition, psychological state planners are adopted to gain the protagonist’s global psychological states for story planning. Eventually, a psychology controller is designed to integrate the local and global psychological states into the story context representation for composing psychology-guided stories. Automatic and manual evaluations demonstrate that PICS outperforms baselines, and each part of PICS shows effectiveness for writing stories with more consistent psychological changes.

Story Ending Generation (SEG) is a challenging task in natural language generation. Recently, methods based on Pre-trained Language Models (PLM) have achieved great prosperity, which can produce fluent and coherent story endings. However, the pre-training objective of PLM-based methods is unable to model the consistency between story context and ending. The goal of this paper is to adopt contrastive learning to generate endings more consistent with story context, while there are two main challenges in contrastive learning of SEG. First is the negative sampling of wrong endings inconsistent with story contexts. The second challenge is the adaptation of contrastive learning for SEG. To address these two issues, we propose a novel Contrastive Learning framework for Story Ending Generation (CLseg)†, which has two steps: multi-aspect sampling and story-specific contrastive learning. Particularly, for the first issue, we utilize novel multi-aspect sampling mechanisms to obtain wrong endings considering the consistency of order, causality, and sentiment. To solve the second issue, we well-design a story-specific contrastive training strategy that is adapted for SEG. Experiments show that CLseg outperforms baselines and can produce story endings with stronger consistency and rationality.

Emotional support conversation aims at reducing the emotional distress of the help-seeker, which is a new and challenging task. It requires the system to explore the cause of help-seeker's emotional distress and understand their psychological intention to provide supportive responses. However, existing methods mainly focus on the sequential contextual information, ignoring the hierarchical relationships with the global cause and local psychological intention behind conversations, thus leads to a weak ability of emotional support. In this paper, we propose a Global-to-Local Hierarchical Graph Network to capture the multi-source information (global cause, local intentions and dialog history) and model hierarchical relationships between them, which consists of a multi-source encoder, a hierarchical graph reasoner, and a global-guide decoder. Furthermore, a novel training objective is designed to monitor semantic information of the global cause. Experimental results on the emotional support conversation dataset, ESConv, confirm that the proposed GLHG has achieved the state-of-the-art performance on the automatic and human evaluations.

Distant supervision (DS) is widely used in relation extraction which can automatically generate large-scale training data by aligning a knowledge base with an unlabeled corpus. However, it suffers from the label noise problem. In this paper, we propose a novel explicit training loss based DS relation extraction de-noising method to generate a cleansed dataset. Specifically, we firstly design a noise detector to select noisy bags using average loss during cyclical training, which is based on the idea that the noisy samples will have different loss variation process during training compared to the clean samples. Then we propose a label corrector to generate right labels for the selected samples, which can keep more useful information to a great extent. Finally, the experimental results show that our de-noising method is robust that can detect the label noise quite well on both large scale and small scale dataset and the generated cleansed dataset significantly improves the performance of previous distant supervision models.

Task-oriented dialogues have one-to-many property from semantic and syntactic perspectives, with many suitable dialogue acts and syntactic forms for a given post. However, current state-of-the-art task-oriented dialogue systems attempt to improve the quality of dialogues in terms of the most popular metrics (i.e., BLEU and entity F1), measuring the similarity between the generated responses and the human annotations, while the diversity of task-oriented dialogues remains less explored. This paper aims to improve the diversity of task-oriented dialogues from both semantic and syntactic perspectives by proposing a structural causal model to learn the causality composition of the dialogue acts and syntactic forms. Specifically, the disentangled understanding module decouples the dialogue into semantic and syntactic spaces and learns one-to-many property with multiple reference training. Then the casual collaboration generation module is proposed to apply Structural Causal Mechanism (SCM) to learn the causality composition relationship of the semantic and syntactic representations to generate the diverse response. Extensive experiments on the MultiWOZ datasets demonstrate that the proposed method achieves significantly better diversity than solid competitors.