Large Language Models (LLMs) demonstrate impressive capabilities across diverse tasks, yet their safety mechanisms remain susceptible to adversarial exploitation of cognitive biases---systematic deviations from rational judgment. Unlike prior studies focusing on isolated biases, this work highlights the overlooked power of multi-bias interactions in undermining LLM safeguards. Specifically, we propose CognitiveAttack, a novel red-teaming framework that adaptively selects optimal ensembles from 154 human social psychology-defined cognitive biases, engineering them into adversarial prompts to effectively compromise LLM safety mechanisms. Experimental results reveal systemic vulnerabilities across 30 mainstream LLMs, particularly open-source variants. CognitiveAttack achieves a substantially higher attack success rate than the SOTA black-box method PAP (60.1% vs. 31.6%), exposing critical limitations in current defenses. Through quantitative analysis of successful jailbreaks, we further identify vulnerability patterns in safety-aligned LLMs under synergistic cognitive biases, validating multi-bias interactions as a potent yet underexplored attack vector. This work introduces a novel interdisciplinary perspective by bridging cognitive science and LLM safety, paving the way for more robust and human-aligned AI systems.

Multi-task learning (MTL) enables joint learning over multiple tasks based on shared representations, but suffers from task interference issue during optimization.Existing works mainly focus on task balancing or probabilistic modeling but fail to address the issue since they struggle to learn sufficient representations for all target tasks.To address this, we propose a multi-task representation alignment (MTRA) framework to achieve task-specific alignment and self-alignment on the shared representations from a mutual information perspective.MTRA ensures that the learned representations contain task-relevant features while mitigating the negative effects of task-irrelevant features.First, we design a task-specific alignment objective to align the shared representations and task-specific representations with the expected targets of all tasks via information maximization.Besides, we design a self-alignment objective to eliminate task-irrelevant features via conditional information minimization.Experiments on two multi-task language benchmarks show that MTRA outperforms 13 representative MTL methods under the same settings, particularly under label-noisy and dataconstrained conditions.Further analysis shows that the learned shared representations exhibit sufficient task informativeness and superior alignment properties.

Large Language Model (LLM)-based agents are increasingly deployed for complex, toolbased tasks where long-term memory is critical to driving actions.Existing benchmarks, however, primarily test an agent's ability to passively retrieve isolated facts in response to explicit questions.They fail to evaluate the more crucial capability of actively applying memory to execute tasks.To address this gap, we introduce MEM2ACTBENCH, a benchmark for evaluating whether agents can proactively leverage long-term memory to execute tool-based actions by selecting appropriate tools and grounding their parameters.The benchmark simulates persistent assistant usage, where users mention the same topic across long, interrupted interactions and expect previously established preferences and task states to be implicitly applied.We build the dataset with an automated pipeline that merges heterogeneous sources (ToolACE, BFCL, OASST1), resolves conflicts via consistency modeling, and synthesizes 2,029 sessions with 12 user-assistant-tool turns on average.From these memory chains, a reverse-generation method produces 400 tooluse tasks, with human evaluation confirming 91.3% are strongly memory-dependent.Experiments on seven memory frameworks show that current systems remain inadequate at actively utilizing memory for parameter grounding, highlighting the need for more effective approaches to evaluate and improve memory application in task execution.

Bingkang Shi, Jen-tse Huang, Luo Long, Tianyu Zong, Hongzhu Yi, Yuanxiang Wang, Songlin Hu, Xiaodan Zhang, Zhongjiang Yao. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Prevailing safety alignment methods still leave Large Language Models (LLMs) vulnerable to sophisticated jailbreak attacks.To bolster defenses, explicit reasoning mechanisms like Safety-oriented Chain-of-Thought (SCoT) have emerged, significantly enhancing robustness.However, this transparency introduces a critical trade-off: the exposed reasoning process itself becomes a new attack surface, risking the leakage of harmful information and revealing the model's safety logic to adversaries.This paper directly confronts this dilemma, asking: Can we achieve the full benefits of deliberative safety without the costs of explicit reasoning generation?We propose Safety Reasoning Internalization to make the deliberative process in SCoT "available but not visible".This approach is grounded in a key theoretical insight: the corrective influence of an SCoT can be effectively approximated by a targeted, low-rank update to the model's Feed-Forward Network (FFN) layers.We operationalize this through Hierarchical Internalization of Adversarially-Guided Reasoning (HIAR), a layer-wise safety alignment framework that internalizes safety reasoning into an implicit computational pathway using Low-Rank Adaptation (LoRA).HIAR enables the model to reach a safe conclusion within a single forward pass, entirely eliminating the need to generate vulnerable SCoT text.Extensive experiments on various LLMs demonstrate that HIAR achieves a 43% lower Attack Success Rate (ASR) against distinct jailbreak attacks compared to strong baselines.

To address the increasingly severe safety risk of large language models (LLMs), reasoningbased safety alignment methods have emerged.These methods overcome the limitations of 'shallow alignment' by exposing the model's Chain-of-Thought (CoT), enabling auditability of safety reasoning process through both training-phase supervision and post-generation verification.However, this transparency creates a critical vulnerability, a tension we define as the Security Auditability Dilemma: while explicit reasoning is a prerequisite for safety, its textual Auditable paradoxically transforms it into an optimization target for adaptive attackers and induces the model to unintentionally copy harmful content from its own reasoning context.To address this, we propose Auditable Latent CoT Alignment (ALCA), a framework that decouples internal reasoning from external output.ALCA shifts the safety deliberation process into a continuous latent space.This allows the safety reasoning process to guide the generation of harmless outputs, while eliminates the discrete textual surface that facilitates internal copying and adaptive attack.Yet, this process is not a black box.we introduce a restricted Self-Decoding mechanism that allows the model to reconstruct its latent reasoning into human-readable text for supervision under specific guidance.Extensive experiments show that ALCA achieves robustness alignment, reducing the success rate of adaptive jailbreak attacks by over 40% compared to strong baselines, while preserving performance.Our framework presents a path toward building LLMs that are both robustly secure and auditable.

Propagation-based fake news detectors primarily extract structural patterns from news propagation trees via graph neural networks (GNNs), which are crucial for trustworthy information access on social platforms. However, these systems remain vulnerable to adversarial message injection, increasingly enabled by large language models (LLMs). Such attacks pollute both semantic and structural signals, causing GNN-based aggregators to fuse logically conflicting content and yield unreliable representations. To address this, we propose LLM-TKT, a novel framework that distills LLM-based narrative reasoning into lightweight GNNs for robust fake news detection. The framework operates in two stages. First, we construct an offline LLM-driven narrative hub to synthesize global propagation narratives and diagnose local node-level coherence. Second, we design a dual-level narrative alignment to learn the semantic invariance of propagation with the guidance of propagation narratives. It filters unreliable neighbor nodes via local consistency and optimizes graph representations via global anchoring. Experiments on three real-world datasets demonstrate that LLM-TKT significantly outperforms existing methods, particularly in defending against sophisticated LLM-driven injection attacks without incurring runtime LLM inference costs.

This paper proposes a new principled multi-task representation learning framework (InfoMTL) to extract noise-invariant sufficient representations for all tasks. It ensures sufficiency of shared representations for all tasks and mitigates the negative effect of redundant features, which can enhance language understanding of pre-trained language models (PLMs) under the multi-task paradigm. Firstly, a shared information maximization principle is proposed to learn more sufficient shared representations for all target tasks. It can avoid the insufficiency issue arising from representation compression in the multi-task paradigm. Secondly, a task-specific information minimization principle is designed to mitigate the negative effect of potential redundant features in the input for each task. It can compress task-irrelevant redundant information and preserve necessary information relevant to the target for multi-task prediction. Experiments on six classification benchmarks show that our method outperforms 12 comparative multi-task methods under the same multi-task settings, especially in data-constrained and noisy scenarios. Extensive experiments demonstrate that the learned representations are more sufficient, data-efficient, and robust.

The rapid development of social platforms exacerbates the dissemination of misinformation, which stimulates the research in fact verification. Recent studies tend to leverage semantic features to solve this problem as a single-hop task. However, the process of verifying a claim requires several pieces of evidence with complicated inner logic and relations to verify the given claim in real-world situations. Recent studies attempt to improve both understanding and reasoning abilities to enhance the performance, but they overlook the crucial relations between entities that benefit models to understand better and facilitate the prediction. To emphasize the significance of relations, we resort to Large Language Models (LLMs) considering their excellent understanding ability. Instead of other methods using LLMs as the predictor, we take them as relation extractors, for they do better in understanding rather than reasoning according to the experimental results. Thus, to solve the challenges above, we propose a novel Structured Knowledge-Augmented LLM-based Network (LLM-SKAN) for multi-hop fact verification. Specifically, we utilize an LLM-driven Knowledge Extractor to capture fine-grained information, including entities and their complicated relations. Besides, we leverage a Knowledge-Augmented Relation Graph Fusion module to interact with each node and learn better claim-evidence representations comprehensively. The experimental results on four common-used datasets demonstrate the effectiveness and superiority of our model.

Large Language Model-based Dense Retrieval (LLM-DR) optimizes over numerous heterogeneous fine-tuning collections from different domains. However, the discussion about its training data distribution is still minimal. Previous studies rely on empirically assigned dataset choices or sampling ratios, which inevitably lead to sub-optimal retrieval performances. In this paper, we propose a new task-level Distributionally Robust Optimization (tDRO) algorithm for LLM-DR fine-tuning, targeted at improving the universal domain generalization ability by end-to-end reweighting the data distribution of each task. The tDRO parameterizes the domain weights and updates them with scaled domain gradients. The optimized weights are then transferred to the LLM-DR fine-tuning to train more robust retrievers. Experiments show optimal improvements in large-scale retrieval benchmarks and reduce up to 30% dataset usage after applying our optimization algorithm with a series of different-sized LLM-DR models.

Social media platforms like X(Twitter) and Reddit are vital to global communication. However, advancements in Large Language Model (LLM) technology give rise to social media bots with unprecedented intelligence. These bots adeptly simulate human profiles, conversations, and interactions, disseminating large amounts of false information and posing significant challenges to platform regulation. To better understand and counter these threats, we innovatively design BotSim, a malicious social botnet simulation powered by LLM. BotSim mimics the information dissemination patterns of real-world social networks, creating a virtual environment composed of intelligent agent bots and real human users. In the temporal simulation constructed by BotSim, these advanced agent bots autonomously engage in social interactions such as posting and commenting, effectively modeling scenarios of information flow and user interaction. Building on the BotSim framework, we construct a highly human-like, LLM-driven bot dataset called BotSim-24 and benchmark multiple bot detection strategies against it. The experimental results indicate that detection methods effective on traditional bot datasets perform worse on BotSim-24, highlighting the urgent need for new detection strategies to address the cybersecurity threats posed by these advanced bots.

Multi-task learning (MTL) enhances efficiency by sharing representations across tasks, but task dissimilarities often cause partial learning, where some tasks dominate while others are neglected.Existing methods mainly focus on balancing loss or gradients but fail to fundamentally address this issue due to the representation discrepancy in latent space.In this paper, we propose variance-invariant probabilistic decoding for multi-task learning (VIP-MTL), a framework that ensures impartial learning by harmonizing representation spaces across tasks.VIP-MTL decodes shared representations into task-specific probabilistic distributions and applies variance normalization to constrain these distributions to a consistent scale.Experiments on two language benchmarks show that VIP-MTL outperforms 12 representative methods under the same multi-task settings, especially in heterogeneous task combinations and dataconstrained scenarios.Further analysis shows that VIP-MTL is robust to sampling distributions, efficient on optimization process, and scale-invariant to task losses.Additionally, the learned task-specific representations are more informative, enhancing the language understanding abilities of pre-trained language models under the multi-task paradigm.

As Large Language Models (LLMs) scale up and gain powerful Chain-of-Thoughts (CoTs) reasoning abilities, practical resource constraints drive efforts to distill these capabilities into more compact Smaller Language Models (SLMs).We find that CoTs consist mainly of simple reasoning forms, with a small proportion (≈ 4.7%) of key reasoning steps that truly impact conclusions.However, previous distillation methods typically involve supervised fine-tuning student SLMs only on correct CoTs data produced by teacher LLMs, resulting in students struggling to learn the key, instead imitating the teacher's reasoning forms and making errors or omissions in reasoning.To address these issues, drawing an analogy to human learning, where analyzing mistakes according to correct solutions often reveals the crucial steps leading to successes or failures, we propose mistakE-Driven key reasonIng step distillaTion (EDIT), a novel method that further aids SLMs learning key reasoning steps rather than mere simple fine-tuning.Firstly, to expose the crucial steps in CoTs, we carefully design specific prompts to generate dual CoTs data with similar reasoning paths but divergent conclusions.Then, we apply the minimum edit distance algorithm on the dual CoTs data to locate these key steps and optimize the likelihood on these tokens.Extensive experiments and analysis validate the effectiveness of EDIT across both in-domain(IND) and out-ofdomain(OOD) benchmark reasoning datasets 1 .* Kun Li is the corresponding author. 1 Code can be found at https://github.com/ C-W-D/EDIT On the floor, I see two magenta scrunchiephone chargers, one gold fidget spinner, three gold booklets, three silver scrunchiephone chargers, two silver booklets, and two magenta booklets.If I remove all the magenta items from the floor, how many scrunchiephone chargers remain on it?Op ons: (A) zero (B) one (C) two (D) three (E) four (F) five (G) six According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items from the floor, we are le with 2 silver scrunchiephone chargers.Therefore, the answer is (C).According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items, we are le with only the silver scrunchiephone chargers.Therefore, the number of scrunchiephone chargers that remain on the floor is three.Therefore, the answer is (D).Teacher Ques on Unthinking Student On the floor, I see two magenta scrunchiephone chargers, one gold fidget spinner...According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items from the floor, we are le with 2 silver scrunchiephone chargers.Therefore, the answer is (C).According to this ques on, there are a total of 5 scrunchiephone chargers on the floor: 2 magentascrunchiephone chargers and 3 silver scrunchiephone chargers.If we remove all the magenta items from the floor, we remove the 2 magenta scrunchiephone chargers.Therefore, the number of scrunchiephone chargers remaining on the floor is 3. Therefore, the answer is (D). Key Steps in Correct CoTs Data Key Steps in Wrong CoTs Data

Autonomous agents powered by large language models (LLMs) hold significant potential across various domains. The Reflection framework is designed to help agents learn from past mistakes in complex tasks. While previous research has shown that reflection can enhance performance, our investigation reveals a key limitation: meaningful self-reflection primarily occurs at the beginning of iterations, with subsequent attempts failing to produce further improvements. We term this phenomenon “Early Stop Reflection,” where the reflection process halts prematurely, limiting the agent’s ability to engage in continuous learning. To address this, we propose the DORA method (Dynamic and Optimized Reflection Advice), which generates task-adaptive and diverse reflection advice. DORA introduces an external open-source small language model (SLM) that dynamically generates prompts for the reflection LLM. The SLM uses feedback from the agent and optimizes the prompt generation process through a non-gradient Bayesian Optimization (BO) algorithm, ensuring the reflection process evolves and adapts over time. Our experiments in the MiniWoB++ and Alfworld environments confirm that DORA effectively mitigates the “Early Stop Reflection” issue, enabling agents to maintain iterative improvements and boost performance in long-term, complex tasks. Code are available at https://anonymous.4open.science/r/DORA-44FB/.

Large language models (LLMs) have demonstrated significant progress in role-playing tasks, yet evaluating their ability to simulate personality traits remains a challenge. Traditional psychological questionnaires-based method have been used to assess LLMs' personality traits. However, these approaches have limitations when applied to LLM-driven role-playing agents (RPAs), as they are designed for humans and rely on stable, self-assessed personality traits. To bridge this gap, we extend simple self-perception questionnaires to more objective, real-world evaluations. In this paper, we introduce APEE, a new dataset consisting of 473 instances across three real-world scenario types: practical goal planning, social media behavior, and leaderless group discussions (LGD). In addition to evaluating whether LLMs adhere to predefined character traits, we introduce two key metrics: Stability and Differentiation. These metrics assess how consistently LLMs express personality traits across different scenarios (Stability) and how effectively they differentiate their behavior when assuming multiple roles (Differentiation). We conducted experiments on 339 different roles using 11 advanced LLMs with the APEE dataset. Discuss the impact of factors such as model size and architecture. Code and dataset are available at https://github.com/linkseed18612254945/APEE_Personality.

As Large Language Models (LLMs) are increasingly deployed as autonomous agents to accomplish complex real-world tasks, they must select appropriate actions from massive action spaces. However, in open-domain settings, LLMs often generate hallucinations when selecting from large action spaces due to their lack of practical operation experience. While existing approaches attempt to reduce action spaces through external planning mechanisms, they heavily rely on substantial training data from executable environments. To address this challenge, we propose a generative retrieval-based methodology for identifying necessary action subspaces. Our approach leverages the autoregressive capabilities of pre-trained models, enabling better utilization of knowledge and superior performance in low-data scenarios, while providing interpretable retrieval results through generated action encodings. We design a two-phase training framework that combines self-learning generative retrieval with targeted optimization. Through comprehensive experiments on multiple benchmark datasets, our approach demonstrates superior retrieval performance while effectively reducing model hallucinations and enhancing plan feasibility. Results show that our method consistently outperforms existing approaches across various metrics.

In intelligent collaborative systems, the role of Large Language Models (LLMs) is becoming increasingly significant, with their security and privacy being of paramount importance. Greedy Coordinate Gradient (GCG)-based adversarial approaches are a staple in red team testing for circumventing the security alignments of LLMs. Yet, these methods are challenged by issues such as convergence difficulties and pseudo-evasion, which can impede attack efficacy. Our research indicates that the high likelihood of rejection tokens appearing in the initial k positions of generated text is a major contributor to adversarial failures, and their suppression can significantly improve attack success rates. Building on these insights, we present DS-GCG, an innovative adversarial attack methodology that enhances GCG attack potency. It employs adjustable-position prefilling to quell refusal responses and incite harmful outputs, coupled with a bidirectional greedy gradient search to swiftly identify adversarial suffixes. DS-GCG's universal suffix approach not only mitigates refusals but also hastens convergence, offering an efficient and robust search strategy. Our experimental results on widely-used open-source LLMs, showcased on the AdvBench dataset, confirm the cutting-edge performance of DS-GCG.

With the proliferation of misinformation on social media, the need for fact-checking has increased, garnering the attention of researchers. The main task of fact-checking is to analyze the semantic relationship and logic between the evidence and the claim, so as to determine the veracity of the claim. The semantic relationship between the claim and the evidence can be strong or weak. However, previous studies ignores the disparities in the strength of semantic relationship between evidence and claims in the process of evidence modeling for fact-checking, neglects the role of weak semantically related evidence in verifying claims. Weakly semantically related evidence logically determines the veracity of the claim sometimes. This paper proposes an evidence-claim relevance decoupling (ECRD) framework for multimodal fact-checking, where the claim and the evidence could be separated by decoupling the strong and weak relevant semantic parts in multimodal dual-stream models. Specifically, we first project text and image modalities into two distinct spaces. One space focuses on tightly coupled semantics between the claim and evidence, reducing the gap. Another space explores the loosely coupled parts, analyzing the various meaning between claim and evidence to reveal potential underlying connections. Then, the fully connected visual and textual graphs are constructed for above two spaces. Finally, the responses obtained by interacting in above graphs are fed into the fusion layer. On FACTIFY dataset, which features diverse text-image relationships, and MultiFact dataset, experimental results show that the proposed ECRD framework could improve the performance of classification significantly compared to baseline models and produces a competitive performance.

Minxuan Lv, Zhenpeng Su, Leiyu Pan, Yizhe Xiong, Zijia Lin, Hui Chen, Wei Zhou, Jungong Han, Guiguang Ding, Wenwu Ou, Di Zhang, Kun Gai, Songlin Hu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

This paper contains potentially offensive and harmful text.Large language models (LLMs) are widely deployed as zero-shot evaluators for answer grading, content moderation, and document ranking.Yet studies show that guard models (Guards)-LLMs fine-tuned for safety-remain vulnerable to "jailbreak" attacks, jeopardising downstream chatbots.We confirm this weakness on three public benchmarks (BeaverTails, XSTest, AdvBench) and trace it to representation shifts that arise in the embedding layer and cascade through the Transformer stack.To counteract the effect, we introduce Gamma-Guard: lightweight residual adapters inserted after the embeddings and at sparse intervals in the model.The adapters start with zero-scaled gates, so they retain the original behaviour; a brief adversarial finetuning phase then teaches them to denoise embeddings and refocus attention.With fewer than 0.1 % extra parameters and only a 2 % latency increase, Gamma-Guard lifts adversarial accuracy from ≤ 5 % to ≈ 95 % a 90 percentage-point gain while reducing cleandata accuracy by just 8 percentage points.Extensive ablations further show that robustness improvements persist across different layer placements and model sizes.To our knowledge, this is the first approach that directly augments large Guards with trainable adapters, providing a practical path toward safer large-scale LLM deployments.

Large Language Models often contain factually incorrect or outdated knowledge, giving rise to model editing methods for precise knowledge updates.However, current mainstream locate-then-edit approaches exhibit a progressive performance decline during sequential editing, due to inadequate mechanisms for long-term knowledge preservation.To tackle this, we model the sequential editing as a constrained stochastic programming.Given the challenges posed by the cumulative preservation error constraint and the gradually revealed editing tasks, LyapLock is proposed.It integrates queuing theory and Lyapunov optimization to decompose the long-term constrained programming into tractable stepwise subproblems for efficient solving.This is the first model editing framework with rigorous theoretical guarantees, achieving asymptotic optimal editing performance while meeting the constraints of long-term knowledge preservation.Experimental results show that our framework scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89% over SOTA baselines.Furthermore, it can be leveraged to enhance the performance of baseline methods.Our code is released on https://github.com/caskcsg/LyapLock.

Open-vocabulary object detection (OVD) is a significant task identifying objects from categories not included in the training set. Our comprehensive analysis reveals two main issues with existing OVD models: poor generalization of localization network to novel categories and poor quality of class embedding impacting accuracy. We propose two solutions: Localization Network Enhancement based on Class-Agnostic Pseudo-Labels (LNE-CAPL) and Class Embedding Enhancement based on MultiModal Hybrid Knowledge (CEE-MMHK). These methods can be used offline to significantly improve the existing OVD models without affecting their training and inference efficiency. Extensive experiments confirm their effectiveness and universality. Code and models will soon be open-sourced.

Jailbreak vulnerabilities in Large Language Models (LLMs) refer to methods that extract malicious content from the model by carefully crafting prompts or suffixes, which has garnered significant attention from the research community. However, traditional attack methods, which primarily focus on the semantic level, are easily detected by the model. These methods overlook the difference in the model’s alignment protection capabilities at different output stages. To address this issue, we propose an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on LLMs. Our method leverages the model’s instruction-following capabilities to first output pre-filled safe content, then exploits its narrative-shifting abilities to generate harmful content. Extensive black-box experiments demonstrate our method can improve the attack success rate by 47% on the widely recognized secure model (Llama2) compared to existing approaches. Our code can be found at: https://github.com/Yummy416/AdaPPA.

Identifying bots on social media has become a crucial and challenging task for regulating online discourse. Existing detection methods primarily focus on individual account-level information, identifying potential threats by detecting inconsistencies between genuine humans and anomalous bots in personal profiles, textual content, and social relationships. However, these approaches generally overlook the coordinated behavior characteristics inherent in groups of bot accounts. To address this research gap, we propose a novel Bot detection network based on Coordinated Group Perception (BotCGP), which enhances bot identification performance by uncovering the collective coordinated features among bot groups. Specifically, our method jointly models account profiles, textual content, and social relationships using a Student’s t-distribution kernel function and a differentiable modularity function to capture potential coordinated characteristics. Experimental results demonstrate that BotCGP significantly outperforms existing methods in bot detection across three real-world X/Twitter datasets. Our code is available at https://github.com/QQQQQQBY/BotCGP.

Long-term time series forecasting (LTSF) seeks to make accurate long-term predictions by leveraging extensive historical data, which is crucial for solving scientific and engineering challenges. Traditional transformer-based methods process historical segments individually, leading to a limited view that overlooks distant dependencies within the entire time series. In this paper, we introduce the Segment-Recurrent Transformer (SRTrans), designed to provide a more comprehensive understanding of historical time series dynamics. By incorporating segment-level recurrence into the Transformer, our model enhances inter-segment information flow, capturing longer-term and global dependencies. We also propose a multi-scale adaptive fusion module that efficiently integrates diverse patterns using a variable-scale chunking mechanism and a weight-mixing strategy. Additionally, our spectrum purge operation improves data preprocessing by extracting significant long-term patterns from the frequency domain. Extensive experiments on eight real-world datasets demonstrate SRTrans’s effectiveness in accuracy and efficiency, offering a promising new solution for LTSF tasks.

Recent advancements in large language models (LLMs) have highlighted the importance of extending context lengths for handling complex tasks. While traditional methods for training on long contexts often use filtered long documents, these approaches lead to domain imbalances, limiting model performance. To address this, techniques like random document concatenation (Standard) and similarity-based methods (KNN, ICLM) have been developed. However, they either sacrifice semantic coherence or diversity. To balance both aspects, we introduce Quest, a query-centric data synthesis method aggregating semantically relevant yet diverse documents. Quest uses a generative model to predict potential queries for each document, grouping documents with similar queries and keywords. Extensive experiments demonstrate Quest's superior performance on long-context tasks, achieving remarkable results with context lengths of up to 1M tokens and confirming its scalability across various model sizes.

Large language models (LLMs) with extended context windows have made significant strides yet remain a challenge due to the scarcity of long documents. Existing methods tend to synthesize long-context data but lack a clear mechanism to reinforce the long-range dependency modeling. To address this limitation, we propose NExtLong, a novel framework for synthesizing long-context data through Negative document Extension. NExtLong decomposes a document into multiple meta-chunks and extends the context by interleaving hard negative distractors retrieved from pretraining corpora. This approach compels the model to discriminate long-range dependent context from distracting content, enhancing its ability to model long-range dependencies. Extensive experiments demonstrate that NExtLong achieves significant performance improvements on the HELMET and RULER benchmarks compared to existing long-context synthesis approaches and leading models, which are trained on non-synthetic long documents. These findings highlight NExtLong's ability to reduce reliance on non-synthetic long documents, making it an effective framework for developing advanced long-context LLMs.

The dissemination of misinformation on social media is likely to lead to severe social conflicts, therefore, the prediction of individual responses to news events becomes a significant task of social importance. The existing belief-centered method overlooks the fact that people tend to respond similarly to certain types of news. To tackle this issue, we propose a Multi-View framework with Dynamic Evidential Fusion(MVDEF) for Response Forecasting. To carry out the task, we first utilize Large Language Models to extract news topics and user beliefs from the news content and user profiles. Then, we construct three single-view graphs: a user-news interaction graph, a belief-aware graph, and a topic-aware graph. We dynamically evaluate the reliability of each view for different samples and then integrate the results based on their corresponding uncertainty mass. Additionally, we introduce a pseudo-view to enhance the interaction between these three views. Extensive experiments demonstrate that our model achieves excellent performance on real-world Twitter data. Further analysis reveals the model’s capability in unseen user scenarios, underscoring its practical applicability in real-world problems.

Hallucination remains a significant challenge which constrains the development of large vision-language models (LVLMs). Therefore, reliable hallucination detection has become a critical step in LVLMs evaluation and real-world deployment. Many previous studies have explored hallucination detection in LVLMs, with uncertainty-based approaches being widely adopted due to the independence from external tools and relatively low resource consumption. However, we observe that uncertainty does not always completely correlate with hallucination. Therefore, uncertainty-based methods may fail in certain cases, such as instances exhibiting high uncertainty but non-hallucination. To address this issue, we propose a framework called Visual Perception Uncertainty Learning (VisPUL) for hallucination detection in LVLMs. Specifically, VisPUL integrates visual information into uncertainty learning directly, allowing to capture uncertainty and visual-text consistency simultaneously. VisPUL improves the insufficiency of uncertainty methods that rely only on text output, providing enhanced generalizability and reliability. Extensive experiments conducted on the M-HalDetect and POPE datasets, covering both open-ended and yes-or-no tasks. Experimental results demonstrate that VisPUL significantly outperforms several strong baseline methods across different LVLMs.

Zhenpeng Su, Xing W, Zijia Lin, Yizhe Xiong, Minxuan Lv, Guangyuan Ma, Hui Chen, Songlin Hu, Guiguang Ding. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

High-quality long-context instruction data is essential for aligning long-context large language models (LLMs). Despite the public release of models like Qwen and Llama, their long-context instruction data remains proprietary. Human annotation is costly and challenging, while template-based synthesis methods limit scale, diversity, and quality. We introduce LongMagpie, a self-synthesis framework that automatically generates large-scale long-context instruction data. Our key insight is that aligned long-context LLMs, when presented with a document followed by special tokens preceding a user turn, auto-regressively generate contextually relevant queries. By harvesting these document-query pairs and the model's responses, LongMagpie produces high-quality instructions without human effort. Experiments on HELMET, RULER, and Longbench v2 demonstrate that LongMagpie achieves leading performance on long-context tasks while maintaining competitive performance on short-context tasks, establishing it as a simple and effective approach for open, diverse, and scalable long-context instruction data synthesis.

The rapid development of online media has heightened the importance of multimodal emotion recognition (MER) in video analysis. However, practical applications often encounter challenges due to missing modalities caused by various interferences. It is difficult to predict the specific missing situations, such as the number and types of missing modalities. Current approaches to modality missing typically apply a uniform method to address various missing cases, which are insufficiently adaptive to dynamic conditions. For example, translation-based methods can efficiently complete missing text from audio, but generating audio or video features that retain the original emotional information from other modalities is challenging and may introduce additional noise. In this paper, we introduce ROSA, a novel robust self-adaptive model designed to address various missing cases with tailored approaches, leveraging available modalities effectively and reducing the introduction of additional noise. Specifically, the A-T Completion module based on the encoder-decoder architecture enables ROSA to generate missing raw text from audio rather than mere embedding representations, capturing more nuanced modal features. Additionally, we design the T-V Fusion module based on a vision-language large model for deep extraction and fusion of textual and visual features. Comprehensive experiments conducted on three widely used public datasets demonstrate the superiority and effectiveness of our model. ROSA outperforms other models in both fixed missing rate and fixed missing modality cases. The ablation studies further highlights the contribution of each designed module.

Social bot detection is essential for maintaining the safety and integrity of online social networks (OSNs). Graph neural networks (GNNs) have emerged as a promising solution. Mainstream GNN-based social bot detection methods learn rich user representations by recursively performing message passing along user-user interaction edges, where users are treated as nodes and their relationships as edges. However, these methods face challenges when detecting advanced bots interacting with genuine accounts. Interaction with real accounts results in the graph structure containing camouflaged and unreliable edges. These unreliable edges interfere with the differentiation between bot and human representations, and the iterative graph encoding process amplifies this unreliability. In this article, we propose a social Bot detection method based on Edge Confidence Evaluation (BECE). Our model incorporates an edge confidence evaluation module that assesses the reliability of the edges and identifies the unreliable edges. Specifically, we design features for edges based on the representation of user nodes and introduce parameterized Gaussian distributions to map the edge embeddings into a latent semantic space. We optimize these embeddings by minimizing Kullback-Leibler (KL) divergence from the standard distribution and evaluate their confidence based on edge representation. Experimental results on three real-world datasets demonstrate that BECE is effective and superior in social bot detection. Additionally, experimental results on six widely used GNN architectures demonstrate that our proposed edge confidence evaluation module can be used as a plug-in to improve detection performance.

This paper presents a new supervised representation learning framework, namely structured probabilistic coding (SPC), to learn compact and informative representations from input related to the target task. SPC is an encoder-only probabilistic coding technology with a structured regularization from the target space. It can enhance the generalization ability of pre-trained language models for better language understanding. Specifically, our probabilistic coding simultaneously performs information encoding and task prediction in one module to more fully utilize the effective information from input data. It uses variational inference in the output space to reduce randomness and uncertainty. Besides, to better control the learning process of probabilistic representations, a structured regularization is proposed to promote uniformity across classes in the latent space. With the regularization term, SPC can preserve the Gaussian structure of the latent code and achieve better coverage of the hidden space with class uniformly. Experimental results on 12 natural language understanding tasks demonstrate that our SPC effectively improves the performance of pre-trained language models for classification and regression. Extensive experiments show that SPC can enhance the generalization capability, robustness to label noise, and clustering quality of output representations.

This paper proposes an information-theoretic representation learning framework, named conditional information flow maximization, to extract noise-invariant sufficient representations for the input data and target task.It promotes the learned representations have good feature uniformity and sufficient predictive ability, which can enhance the generalization of pre-trained language models (PLMs) for the target task.Firstly, an information flow maximization principle is proposed to learn more sufficient representations for the input and target by simultaneously maximizing both inputrepresentation and representation-label mutual information.Unlike the information bottleneck, we handle the input-representation information in an opposite way to avoid the overcompression issue of latent representations.Besides, to mitigate the negative effect of potential redundant features from the input, we design a conditional information minimization principle to eliminate negative redundant features while preserve noise-invariant features.Experiments on 13 language understanding benchmarks demonstrate that our method effectively improves the performance of PLMs for classification and regression.Extensive experiments show that the learned representations are more sufficient, robust and transferable.

Hateful memes detection is a challenging multimodal understanding task that requires comprehensive learning of vision, language, and cross-modal interactions.Previous research has focused on developing effective fusion strategies for integrating hate information from different modalities.However, these methods excessively rely on cross-modal fusion features, ignoring the modality uncertainty caused by the contribution degree of each modality to hate sentiment and the modality imbalance caused by the dominant modality suppressing the optimization of another modality.To this end, this paper proposes an Uncertainty-guided Modal Rebalance (UMR) framework for hateful memes detection.The uncertainty of each meme is explicitly formulated by designing stochastic representation drawn from a Gaussian distribution for aggregating cross-modal features with unimodal features adaptively.The modality imbalance is alleviated by improving cosine loss from the perspectives of intermodal feature and weight vectors constraints.In this way, the suppressed unimodal representation ability in multimodal models would be unleashed, while the learning of modality contribution would be further promoted.Extensive experimental results demonstrate that the proposed UMR produces the state-of-the-art performance on four widely-used datasets.

Emotional support conversation (ESC) task aims to relieve the emotional distress of users who have high-intensity of negative emotions. However, due to the ignorance of emotion intensity modelling which is essential for ESC, previous methods fail to capture the transition of emotion intensity effectively. To this end, we propose a Multi-stream information Fusion Framework (MFF-ESC) to thoroughly fuse three streams (text semantics stream, emotion intensity stream, and feedback stream) for the modelling of emotion intensity, based on a designed multi-stream fusion unit. As the difficulty of modelling subtle transitions of emotion intensity and the strong emotion intensity-feedback correlations, we use the KL divergence between feedback distribution and emotion intensity distribution to further guide the learning of emotion intensities. Experimental results on automatic and human evaluations indicate the effectiveness of our method.

Cold-start is a significant problem in recommender systems. Recently, with the development of few-shot learning and meta-learning techniques, many researchers have devoted themselves to adopting meta-learning into recommendation as the natural scenario of few-shots. Nevertheless, we argue that recent work has a huge gap between few-shot learning and recommendations. In particular, users are locally dependent, not globally independent in recommendation. Therefore, it is necessary to formulate the local relationships between users. To accomplish this, we present a novel Few-shot learning method for Cold-Start (FCS) recommendation that consists of three hierarchical structures. More concretely, this first hierarchy is the global-meta parameters for learning the global information of all users; the second hierarchy is the local-meta parameters whose goal is to learn the adaptive cluster of local users; the third hierarchy is the specific parameters of the target user. Both the global and local information are formulated, addressing the new user’s problem in accordance with the few-shot records rapidly. Experimental results on two public real-world datasets show that the FCS method could produce stable improvements compared with the state-of-the-art.

Hate speech detection has become an urgent task with the emergence of huge multimodal harmful content (, memes) on social media platforms. Previous studies mainly focus on complex feature extraction and fusion to learn discriminative information from memes. However, these methods ignore two key points: 1) the misalignment of image and text in memes caused by the modality gap, and 2) the uncertainty between modalities caused by the contribution degree of each modality to hate sentiment. To this end, this paper proposes an uncertainty-aware cross-modal alignment (UCA) framework for modeling the misalignment and uncertainty in multimodal hate speech detection. Specifically, we first utilize the cross-modal feature encoder to capture image and text feature representations in memes. Then, a cross-modal alignment module is applied to reduce semantic gaps between modalities by aligning the feature representations. Next, a cross-modal fusion module is designed to learn semantic interactions between modalities to capture cross-modal correlations, providing complementary features for memes. Finally, a cross-modal uncertainty learning module is proposed, which evaluates the divergence between unimodal feature distributions to to balance unimodal and cross-modal fusion features. Extensive experiments on five publicly available datasets show that the proposed UCA produces a competitive performance compared with the existing multimodal hate speech detection methods.

Network alignment serves as a key methodology in data analysis, enabling a understanding of intricate structures within multiple networks. The discovery of potential aligned pairs between networks is a prerequisite for a wide spectrum of applications, e.g. cross-domain recommendation or re-identification attack. Presently, most existing network alignment methods involve only two networks, remaining challenges in scenarios with multiple networks. In addition, acquiring cross-network links often poses a significant challenge, resulting in numerous instances where priori annotation information is absent. Our research confronts these problems by proposing Multi-hop Transformation Multiple Network Alignment with Absent Priori Annotation (MT-MNA). The core idea is to simultaneously align multiple networks and address the scenarios of absent priori annotation by fully utilizing the rich-annotated network pairs. Through rigorous experiments on real-world datasets, our method demonstrates superior precision and adaptability to the challenging condition of absent priori annotation in multiple network alignment, overperforming existing state-of-the-art methods.

With the impressive performances achieved by graph representation learning models on tasks such as link prediction, their vulnerability to imperceptible adversarial perturbations has also come to light. However, adversarial attacks struggle to balance effectiveness and transferability, particularly when attempting to deliver effective attacks on various target models in one shot and achieving effective outcomes in both availability and integrity attack settings. This study explores a novel way to mitigate attack performance across various models under availability and integrity attack settings. To fulfill these objectives, we develop a Scoring & Update (SU) framework for performing adversarial attacks against link prediction on graphs. Specifically, we iteratively score perturbations through a parameter-frozen and transferable surrogate model and then update the perturbation with scoring feedback to learn a more effective and transferable adversarial perturbation. Extensive experiments on two real-world datasets show that our attack model is more effective than six attack baselines against six popular target models for graph representation under both availability and integrity attack settings. Code is available at https://github.com/anonymousaccept/STAA.

The expeditious advancement of social question-and-answer (Q&A) platforms has led to the valuable yet challenging practice of anonymous knowledge sharing. Despite implementing various anonymity techniques, the persistent threat of potential privacy breaches remains a paramount concern. To tackle this issue, we introduce the task of de-anonymization within Q&A communities and provide a bilingual dataset (Chinese and English) for research. In this paper, we propose a novel de-anonymization framework called MPT, effectively improving the model’s ability to capture long-range dependencies between nodes by integrating graph neural networks(GNNs) and language models(LMs). Specifically, we use GNN to extract structural features, and then we encode and fuse node representations from multiple meta-paths using Transformer and attention mechanisms. Extensive experiments on Zhihu and Quora data sets show that our model significantly outperforms the baseline model. In addition, our model possesses a degree of interpretability, enabling a comprehensive comprehension of the underlying factors contributing to user privacy breaches and facilitating the implementation of appropriate safeguards. The dataset1and code2utilized in this study have been made publicly accessible.

Multimodal fake news propagation on social media has emerged as a primary concern for both the public and the government. Distinguishing fake news from genuine content is challenging due to their close resemblance. Moreover, fake news often presents information that contradicts real-world facts, underscoring the necessity of incorporating comprehensive open-domain knowledge. However, current knowledge-based detection methods struggle to improve two key areas in integrating open-domain knowledge for fake news detection: i) query acquisition for retrieval and ii) utilization of retrieved knowledge. The challenges stem from limitations in understanding the semantics and reasoning about the relationship between open-domain knowledge and the content of the news. With the emergence of the Large Language Model (LLM), Natural Language Processing (NLP) has witnessed a revolution where impressive semantic understanding and reasoning abilities are significantly improved. This paper proposes a novel Open-domain Knowledge Integrated (OKI) framework for multimodal fake news detection, featuring two LLM-based agents that collaboratively leverage open-domain knowledge. One agent generates appropriate queries to retrieve knowledge, while the other filters out irrelevant retrieved knowledge. Experimental results demonstrate a significant performance improvement of OKI over established baselines on the Weibo and Twitter datasets.

Event temporal relation (TempRel) extraction is a vital branch of information extraction. However, it is expensive to construct human-labeled training datasets, which leads to the lack of high-quality training data and inadequate training of the model. To mitigate this problem, we introduce a novel RLHF-based TempRel Data Augmentation method (RTDA) which aims at generating training data with high quality in terms of diversity and semantics fluency. Experimental results on three popular datasets TBD, TDD-Man, and TDD-Auto show that the model equipped with our method achieves state-of-the-art performance.

Transformer with self-attention was initially crafted to model language sequences, where discrete tokens (i.e., words) showcase high semantic density. However, when applied to time series token inputs (i.e., datapoints) with weak-density semantics and temporal redundancy, it faces challenges as these time-domain tokens impede its ability to capture the intricate latent properties of time series dynamics. While time-frequency transformation presents a viable solution by bringing forth a new space with heightened expressive power, existing approaches fall short of fully exploiting its potential. In response to these limitations, we propose a general-purpose transformer-based model, named Scattering Transformer, for multivariate time series forecasting and self-supervised representation learning. It is based on two innovative components: i) scattering self-attention mechanism incorporating wavelet key/value and standard query to unify the learning of cross-domain relationships between the time and wavelet domains; and ii) stochastic scaling positional encoding scheme that relies solely on order information, emulating longer sequence positions to generalize up to ultra-long horizon case. Extensive experiments on eight real-world benchmarks show the potential of our Scattering Transformer as a robust and versatile solution, showcasing its quadruple efficacy of non-stationary forecasting, ultra-long horizons forecasting, representation learning, and reduction in time and space complexity.

Large language models (LLMs) exhibit enhanced reasoning at larger scales, driving efforts to distill these capabilities into smaller models via teacher-student learning.Previous works simply fine-tune student models on teachers' generated Chain-of-Thoughts (CoTs) data.Although these methods enhance indomain (IND) reasoning performance, they struggle to generalize to out-of-domain (OOD) tasks.We believe that the widespread spurious correlations between questions and answers may lead the model to preset a specific answer which restricts the diversity and generalizability of its reasoning process.In this paper, we propose Cascading Decomposed CoTs Distillation (CasCoD) to address these issues by decomposing the traditional single-step learning process into two cascaded learning steps.Specifically, by restructuring the training objectives-removing the answer from outputs and concatenating the question with the rationale as input-CasCoD's two-step learning process ensures that students focus on learning rationales without interference from the preset answers, thus improving reasoning generalizability.Extensive experiments demonstrate the effectiveness of CasCoD on both IND and OOD benchmark reasoning datasets 1 .* Kun Li is the corresponding author. 1 Code available at https://github.com/C-W-D/CasCoD(a) Answer SFT consistently outperform Std-CoT on OOD tasks.(b) A case of spurious correla on between ques ons and answers.Question: Why did someone bring a swimsuit to a ski resort?Options: (A) To swim in a heated pool.(B) To wear as an underlayer for warmth.(C) To use as a fashion statement.(D) To participate in a polar bear plunge event.Answer: (A) To swim in a heated pool. OOD IND

Automated red teaming is an effective method for identifying misaligned behaviors in large language models (LLMs).Existing approaches, however, often focus primarily on improving attack success rates while overlooking the need for comprehensive test case coverage.Additionally, most of these methods are limited to single-turn red teaming, failing to capture the multi-turn dynamics of real-world human-machine interactions.To overcome these limitations, we propose HARM (Holistic Automated Red teaMing), which scales up the diversity of test cases using a top-down approach based on an extensible, fine-grained risk taxonomy.Our method also leverages a novel fine-tuning strategy and reinforcement learning techniques to facilitate multi-turn adversarial probing in a human-like manner.Experimental results demonstrate that our framework enables a more systematic understanding of model vulnerabilities and offers more targeted guidance for the alignment process.Code and prompts available at: https://github.

Many fake news detection studies have achieved promising performance by extracting effective semantic and structure features from both content and propagation trees. However, it is challenging to apply them to practical situations, especially when using the trained propagation-based models to detect news with no propagation data. Towards this scenario, we study a new task named cold-start fake news detection, which aims to detect content-only samples with missing propagation. To achieve the task, we design a simple but effective Structure Adversarial Net (SAN) framework to learn transferable features from available propagation to boost the detection of content-only samples. SAN introduces a structure discriminator to estimate dissimilarities among learned features with and without propagation, and further learns structure-invariant features to enhance the generalization of existing propagation-based methods for content-only samples. We conduct qualitative and quantitative experiments on three datasets. Results show the challenge of the new task and the effectiveness of our SAN framework.

Multimodal fact verification is an under-explored and emerging field that has gained increasing attention in recent years. The goal is to assess the veracity of claims that involve multiple modalities by analyzing the retrieved evidence. The main challenge in this area is to effectively fuse features from different modalities to learn meaningful multimodal representations. To this end, we propose a novel model named Multi-Source Knowledge-enhanced Graph Attention Network (MultiKE-GAT). MultiKE-GAT introduces external multimodal knowledge from different sources and constructs a heterogeneous graph to capture complex cross-modal and cross-source interactions. We exploit a Knowledge-aware Graph Fusion (KGF) module to learn knowledge-enhanced representations for each claim and evidence and eliminate inconsistencies and noises introduced by redundant entities. Experiments on two public benchmark datasets demonstrate that our model outperforms other comparison methods, showing the effectiveness and superiority of the proposed model.

As an important multimodal sentiment analysis task, Joint Multimodal Aspect-Sentiment Analysis (JMASA), aiming to jointly extract aspect terms and their associated sentiment polarities from the given text-image pairs, has gained increasing concerns. Existing works encounter two limitations: (1) multilevel modality noise, i.e., instance- and feature-level noise; and (2) multi-grained semantic gap, i.e., coarse- and fine-grained gap. Both issues may interfere with accurate identification of aspect-sentiment pairs. To address these limitations, we propose a novel framework named RNG for JMASA. Specifically, to simultaneously reduce multi-level modality noise and multi-grained semantic gap, we design three constraints: (i) Global Relevance Constraint (GR-Con) based on text-image similarity for instance-level noise reduction, (ii) Information Bottleneck Constraint (IB-Con) based on the Information Bottleneck (IB) principle for feature-level noise reduction, and (iii) Semantic Consistency Constraint (SC-Con) based on mutual information maximization in a contrastive learning way for multi-grained semantic gap reduction. Extensive experiments on two datasets validate our new state-of-the-art performance.

Event temporal relation (TempRel) is a primary subject of the event relation extraction task. However, the inherent ambiguity of TempRel increases the difficulty of the task. With the rise of prompt engineering, it is important to design effective prompt templates and verbalizers to extract relevant knowledge. The traditional manually designed templates struggle to extract precise temporal knowledge. This paper introduces a novel retrieval-augmented TempRel extraction approach, leveraging knowledge retrieved from large language models (LLMs) to enhance prompt templates and verbalizers. Our method capitalizes on the diverse capabilities of various LLMs to generate a wide array of ideas for template and verbalizer design. Our proposed method fully exploits the potential of LLMs for generation tasks and contributes more knowledge to our design. Empirical evaluations across three widely recognized datasets demonstrate the efficacy of our method in improving the performance of event temporal relation extraction tasks.

The widespread use of large language models (LLMs) has brought about security risks, including biases, discrimination, and ethical concerns. Reinforcement Learning from Human Feedback (RLHF), as a method to improve model security, still faces challenges such as objective management and misaligned generalization, leading to the emergence of jailbreak attacks. Existing methods implement jailbreak attacks by optimizing adversarial prompts or leveraging the in-context learning capabilities of LLMs, but they are limited in terms of efficiency and scalability. This paper proposes a reinforcement learning-based few-shot example selection method to enhance the effectiveness and efficiency of these attacks. The proposed method extends the GPT-2 architecture with an example selection module and employs strategies such as experience replay and entropy penalty to accelerate convergence and avoid local optima. Experimental results demonstrate that, compared to existing methods, this approach achieves a 100% increase in attack success rate on Vicuna-7B and a 2.4-second reduction in the time cost per harmful instruction generation on GPT-3.5.

Multi-label image classification poses a formidable challenge due to the presence of multiple objects in each image, rendering it notably complex to decipher the visual content comprehensively. Discriminating between multiple objects necessitates the establishment of robust visual label dependencies. Previous methods attempt to formulate cross-modal interaction or one-shot co-occurrence relationship guidance. However, it not only exhibits limitations when handling occluded or blurry objects but also fails to fully leverage the diverse hierarchical properties for sustainably guiding the learning process of label dependencies. To sustainably establish hierarchical visual label dependencies, this paper introduces a Pyramidal Cross-modal Transformer framework for MLIC tasks. Specifically, the pyramidal visual guidance layer parses the visual features into a multi-resolution pyramid structure, allowing the updated visual-related information to provide sustained guidance for label semantics. This surpasses the conventional pre-processing of co-occurrence relationships. Besides, the hybrid modal interaction layer is proposed to effectively mitigate the semantic disparities between visual and label information with modal-blended indiscriminate attention, replacing vanilla self-attention. Several combination blocks consisting of these two layers are integrated and embedded within the encoder-decoder structure to facilitate the exploration of meticulous visual label dependencies. Extensive experiments on two widely-used benchmarks, including MS-COCO and PASCAL VOC 2007, consistently demonstrate that PCMT could provide state-of-the-art results.

Information diffusion prediction aims to forecast the path of information spreading in social networks by exploiting user correlations or preferences. Recent works focus on characterizing the dynamic of user preferences and propose to capture users' dynamic preferences by discretizing the diffusion process into structure snapshots. Despite their effectiveness, these works simply summarize users' dynamic preferences from partially observed structure snapshots, ignoring the continuous evolution of the preferences. Moreover, discretizing the diffusion process makes these models overlook abundant structure information across different periods, reducing their ability to discover potential participants. To address the above issues, we propose a novel Graph Neural Ordinary Differential Equation Network (GODEN) for information diffusion prediction, which incorporates neural ordinary differential equations (ODE) to model the continuous dynamics of the diffusion process. Specifically, we design two coupled ODE functions on nodes and edges to describe their co-evolution dynamics and infer users' dynamic preferences based on the solution of ODEs. To predict the future infections of the observed cascade, we represent its diffusion pattern in terms of temporal and user contexts and apply a multi-head attention module to attend to different contexts. Experimental results confirm our approach's effectiveness, with our model outperforming the state-of-the-art diffusion prediction models.

Zhenpeng Su, Xing W, Wei Zhou, Guangyuan Ma, Songlin Hu. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Masked auto-encoder pre-training has emerged as a prevalent technique for initializing and enhancing dense retrieval systems. It generally utilizes additional Transformer decoder blocks to provide sustainable supervision signals and compress contextual information into dense representations. However, the underlying reasons for the effectiveness of such a pre-training technique remain unclear. The usage of additional Transformer-based decoders also incurs significant computational costs. In this study, we aim to shed light on this issue by revealing that masked auto-encoder (MAE) pre-training with enhanced decoding significantly improves the term coverage of input tokens in dense representations, compared to vanilla BERT checkpoints. Building upon this observation, we propose a modification to the traditional MAE by replacing the decoder of a masked auto-encoder with a completely simplified Bag-of-Word prediction task. This modification enables the efficient compression of lexical signals into dense representations through unsupervised pre-training. Remarkably, our proposed method achieves state-of-the-art retrieval performance on several large-scale retrieval benchmarks without requiring any additional parameters, which provides a 67% training speed-up compared to standard masked auto-encoder pre-training with enhanced decoding.

CTR(click through rate) model has played an important role in modern recommendation systems. Most of the recommendation models in industrial scenario are trained by TensorFlow. However, we observed that, TFRecord, the native k-v data format in TensorFlow, is not the best choice for CTR training. Those keys take up to 54% of the storage space in TFRecord formatted training data. To overcome this, we introduce Apache Parquet, a column-oriented data format, into CTR tasks to improve spatial efficiency. Besides, to use in production environment, we further give some high performance implementations of Parquet training scheme. Firstly, GPU data preprocessing method is adopted in replace of original Spark based solution to generate Parquet training data and accelerate data preprocessing. Secondly, we modify data loader in TensorFlow to consume the Parquet training data with high efficiency. Experimental results show that, the size of preprocessed Criteo dataset is 95.29% smaller in comparison to TFRecord and the data preprocessing time also reduces 99.6%. Without any model performance damage, we speed up the training process by 1.45x. Our scheme has applied to our internal business and has obtained similar performance benefits.

Automatic rumor detection is critical for maintaining a healthy social media environment. The mainstream methods generally learn rich features from information cascades by modeling the cascade as a tree or graph structure where edges are built based on interactions between a tweet and retweets. Some psychology studies have empirically shown that users' various subjective factors always cause the uncertainty of interactions such as differences among interactive behavior activation thresholds or semantic relevancy. However, previous works model interactions by employing a simple fully connected layer on fixed edge weights in the graph and cannot reasonably describe this inherent uncertainty of complex interactions. In this article, inspired by the fuzzy theory, we propose a novel neuro-fuzzy method, fuzzy graph convolutional networks (FGCNs), to sufficiently understand uncertain interactions in the information cascade in a fuzzy perspective. Specifically, a new strategy of graph construction is first designed to convert each information cascade into a heterogeneous graph structure with the consideration of explicit interactive behaviors between a tweet and its retweet, as well as implicit interactive behaviors among retweets, enriching more structural clues in the graph. Then, we improve graph convolutional networks by incorporating edge fuzzification (EF) modules. The EFs adapt edge weights according to predefined membership to enhance message passing in the graph. The proposed model can provide a stronger relational inductive bias for expressing uncertain interactions and capture more discriminative and robust structural features for rumor detection. Extensive experiments demonstrate the effectiveness and superiority of FGCN on both rumor detection and early rumor detection.

Dense passage retrieval aims to retrieve the relevant passages of a query from a large corpus based on dense representations (i.e., vectors) of the query and the passages. Recent studies have explored improving pre-trained language models to boost dense retrieval performance. This paper proposes CoT-MAE (ConTextual Masked Auto-Encoder), a simple yet effective generative pre-training method for dense passage retrieval. CoT-MAE employs an asymmetric encoder-decoder architecture that learns to compress the sentence semantics into a dense vector through self-supervised and context-supervised masked auto-encoding. Precisely, self-supervised masked auto-encoding learns to model the semantics of the tokens inside a text span, and context-supervised masked auto-encoding learns to model the semantical correlation between the text spans. We conduct experiments on large-scale passage retrieval benchmarks and show considerable improvements over strong baselines, demonstrating the high efficiency of CoT-MAE. Our code is available at https://github.com/caskcsg/ir/tree/main/cotmae.

Extracting generalized and robust representations is a major challenge in emotion recognition in conversations (ERC).To address this, we propose a supervised adversarial contrastive learning (SACL) framework for learning classspread structured representations in a supervised manner.SACL applies contrast-aware adversarial training to generate worst-case samples and uses joint class-spread contrastive learning to extract structured representations.It can effectively utilize label-level feature consistency and retain fine-grained intra-class features.To avoid the negative impact of adversarial perturbations on context-dependent data, we design a contextual adversarial training (CAT) strategy to learn more diverse features from context and enhance the model's context robustness.Under the framework with CAT, we develop a sequence-based SACL-LSTM to learn label-consistent and context-robust features for ERC.Experiments on three datasets show that SACL-LSTM achieves state-of-the-art performance on ERC.Extended experiments prove the effectiveness of SACL and CAT.

The rapid development of large-scale deep neural networks has put forward an urgent demand for the efficiency of parallel training. Recently, bidirectional pipeline parallelism has been recognized as an effective approach for improving training throughput. This paper proposes MixPipe, a novel bidirectional pipeline parallelism for efficiently training large-scale models in synchronous scenarios. Compared with previous proposals, MixPipe achieves a better balance between pipeline utilization and device utilization, which benefits from the flexible regulating for the number of micro-batches injected into the bidirectional pipelines at the beginning. MixPipe also features a mixed schedule to balance memory usage and further reduce the bubble ratio. Evaluation results show that: for Transformer based language models (i.e., Bert and GPT-2 models), MixPipe improves the training throughput by up to 2.39× over the state-of-the-art synchronous pipeline approaches.

Neural network models are vulnerable to adversarial examples, and adversarial transferability further increases the risk of adversarial attacks.Current methods based on transferability often rely on substitute models, which can be impractical and costly in real-world scenarios due to the unavailability of training data and the victim model's structural details.In this paper, we propose a novel approach that directly constructs adversarial examples by extracting transferable features across various tasks.Our key insight is that adversarial transferability can extend across different tasks.Specifically, we train a sequence-to-sequence generative model named CT-GAT (Cross-Task Generative Adversarial ATtack) using adversarial sample data collected from multiple tasks to acquire universal adversarial features and generate adversarial examples for different tasks.We conduct experiments on ten distinct datasets, and the results demonstrate that our method achieves superior attack performance with small cost.You can get our code and data at: https: //github.com/xiaoxuanNLP/CT-GAT

Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training.These methods simply consider two passages from the same document to be relevant, without taking into account the potential negative impacts of weakly correlated pairs.Thus, this paper proposes query-as-context pre-training, a simple yet effective pre-training technique to alleviate the issue.Query-as-context pretraining assumes that the query derived from a passage is more likely to be relevant to that passage and forms a passage-query pair.These passage-query pairs are then used in contrastive or generative context-supervised pre-training.The pre-trained models are evaluated on largescale passage retrieval benchmarks and out-ofdomain zero-shot benchmarks.Experimental results show that query-as-context pre-training brings considerable gains for retrieval performances, demonstrating its effectiveness and efficiency.

The technology of text-to-SQL has significantly enhanced the efficiency of accessing and manipulating databases.However, limited research has been conducted to study its vulnerabilities emerging from malicious user interaction.By proposing TrojanSQL, a backdoor-based SQL injection framework for text-to-SQL systems, we show how state-of-the-art text-to-SQL parsers can be easily misled to produce harmful SQL statements that can invalidate user queries or compromise sensitive information about the database.The study explores two specific injection attacks, namely boolean-based injection and union-based injection, which use different types of triggers to achieve distinct goals in compromising the parser.Experimental results demonstrate that both medium-sized models based on fine-tuning and LLM-based parsers using prompting techniques are vulnerable to this type of attack, with attack success rates as high as 99% and 89%, respectively.We hope that this study will raise more concerns about the potential security risks of building natural language interfaces to databases.

Knowledge distillation enhances the performance of the student model by transferring knowledge from the teacher model. Moreover, the attention mechanism has been introduced recently to enable each layer of the student to learn knowledge from all teacher layers, which brings about considerable optimization. However, noted that features from different layers, such as shallow and deep layers, might have a big semantic gap, and compulsively aligning one student layer to all teacher layers would mislead the learning process. To tackle this problem, an effective framework called Semantic Stage-Wise learning for Knowledge Distillation (SSWKD) is presented in this paper. We divide all layers into shallow and deep stages, and only allow feature alignment within the same stage to alleviate semantic mismatch. In addition, with the observation that the performance of deep networks relies more on some key features rather than evenly on all of them, a crucial feature enhancement method based on KL divergence is then proposed for SSWKD, forcing the student to pay more attention to critical features of the teacher. Extensive experiments and visualizations show that our SSWKD outperforms other distillation methods on CIFAR-100 and COCO2017 datasets for image classification, object detection, and instance segmentation tasks.

With social bots evolving continually, the new bots post highly anthropomorphic posts to evade detection. For post content information, existing methods choose to extract features from single posts or use the rough characterization of the pretraining language model for bot detection. However, the evolution of bots has made them better at camouflage in previous posts processing techniques. According to our observation, the purpose of bot posting is to publicize different contents in different periods, and its posting often shows abnormal interest changes, while human posting is to express hobbies and daily life, and its interest changes are relatively stable. Therefore, we extract the interest changes between multiple posts published by users for bot detection. In this paper, we propose a social Bot detection model based on the Window Strategy(BotWS). Specifically, we first employ the window strategy to obtain the user posting windows, each containing multiple posts. Then, we extract the interest changes between the posting windows by multi-head attention mechanism. Finally, we embed the interest changes information into the user representation and construct a heterogeneous graph classification module to classify. We conduct experiments on three challenging datasets. Results indicate that our approach achieves state-of-the-art.

Harmful memes detection is a challenging task in the field of multimodal information processing due to the semantic gap between different modalities. Current research on this task mainly focuses on multimodal dual-stream models. However, the existing works ignore the misalignment of the memes caused by the modality gap. Moreover, the cross-modal interaction in the dual-stream models is insufficient to identify harmful memes. To this end, this paper proposes a scalable invariant and specific modality (ISM) representations framework via graph neural networks. The proposed ISM framework provides a comprehensive and disentangled view for memes and promotes inter-modal interaction. Specifically, ISM projects each modality to two distinct spaces. The first space is modality-invariant, learning the corresponding commonalities and reducing the modality gap. The second space is modality-specific, holding the distinctive characteristics of each modality and complementing the common latent features captured in invariant spaces. Then, we construct fully connected visual and textual graphs for each space. The unimodal graphs are fused to dynamically balance inter-modal and intra-modal relationships, which are complementary to the dual-stream models. Finally, an adaptive module is designed to weigh the proportion of each fusion graph for memes. Moreover, the mainstream multimodal dual-stream models could be employed as the backbone flexibly. Extensive experiments on five publicly available datasets show that the proposed ISM provides a stable improvement over baselines and produces a competitive performance compared with the existing harmful memes detection methods.

Aspect Sentiment Triplet Extraction (ASTE) is an emerging task of fine-grained sentiment analysis, which aims to extract aspect terms, associated opinion terms, and sentiment polarities in the form of triplets. Thus, ASTE involves two groups of subtasks: aspect/opinion term extraction and aspect-opinion-pair sentiment classification. Due to the high correlations of subtasks, three categories of joint methods have been proposed, includingend-to-end tagging-based methods,cascaded span-based methods, andsequence-to-sequence generation-based methods. These methods basically learn either a shared feature space or a shared sentence encoder to capture interactions across all subtasks by parameter sharing. However, they fail to learn deep and mutual interactive features for ASTE. In this work, we present a novel tagging scheme to cast ASTE as a unified boundary-words relation classification problem. Subsequently, we propose an end-to-end Hierarchical Interaction Model (HIM), exploiting deep and mutual interactions across subtasks mainly with two interaction modules. The first-level interaction module primarily leverages multi-task learning models to capture implicit subtask interactions. Then, the second-level interaction module, namely Gated Interaction Network (GIN), adopts a novel gated control mechanism and a newly-designed Conditional BiLSTM (Cond-BiLSTM) network to capture explicit subtask interactions. Moreover, to refine the unreliable outputs of the first-level module, we develop a General word-Pair Relationship Learning (G-PRL) component. With the task-shared features as input, G-PRL further facilitates interactions between term extraction and pair classification. We conduct experiments on two benchmarks and achieve promising results. Extensive analyses demonstrate the effectiveness and flexibility of our work.

Multimodal fake news detection has obtained increasing attention recently. Existing works generally encode multimodal contents into a deterministic point in semantic subspaces, and then fuse multimodal features by simple concatenation or attention mechanisms. However, most methods suffer from adapting to noisy multimodal contents since they neglect the robustness of modality-specific features. Besides, as different modalities usually have varying confidence levels, previous attention-based fusion models that learn modality-independent weights based on the input data feature, would limit the optimal integration of multimodal contents. To alleviate the above issues, we propose novel Multimodal Uncertainty Learning Network (MM-ULN) to enhance multimodal fake news detection by modeling both intra- and inter-modality uncertainty. Specifically, we incorporate a novel intra-modality uncertainty learning (EUL) module to better understand noisy multimodal contents. EULs provide feature regularization in a variational way, successfully alleviating the effects of data uncertainty within modalities. We design a new variational attention fusion (VAF) module to adaptively fuse multimodal contents with modality-dependent weights. The VAF module consider the relative confidence between modalities and enables to explore complementary properties for detection. Extensive experiments on two benchmark datasets demonstrate the effectiveness and superiority of MM-ULN on multimodal fake news detection.

Incorporating knowledge graphs (KGs) into recommender systems to provide explainable recommendation has attracted much attention recently. The multi-hop paths in KGs can provide auxiliary facts for improving recommendation performance as well as explainability. However, existing studies may suffer from two major challenges: error propagation and weak explainability. Considering all paths between every user-item pair might involve irrelevant ones, which leads to error propagation of user preferences. Defining meta-paths might alleviate the error propagation, but the recommendation performance would heavily depend on the pre-defined meta-paths. Some recent methods based on graph convolution network (GCN) achieve better recommendation performance, but fail to provide explainability. To tackle the above problems, we propose a novel method named K nowledge-aware R easoning with G raph C onvolution N etwork (KR-GCN). Specifically, to alleviate the effect of error propagation, we design a transition-based method to determine the triple-level scores and utilize nucleus sampling to select triples within the paths between every user-item pair adaptively. To improve the recommendation performance and guarantee the diversity of explanations, user-item interactions and knowledge graphs are integrated into a heterogeneous graph, which is performed with the graph convolution network. A path-level self-attention mechanism is adopted to discriminate the contributions of different selected paths and predict the interaction probability, which improves the relevance of the final explanation. Extensive experiments conducted on three real-world datasets show that KR-GCN consistently outperforms several state-of-the-art baselines. And human evaluation proves the superiority of KR-GCN on explainability.

Contrastive learning has been attracting much attention for learning unsupervised sentence embeddings. The current state-of-the-art unsupervised method is the unsupervised SimCSE (unsup-SimCSE). Unsup-SimCSE takes dropout as a minimal data augmentation method, and passes the same input sentence to a pre-trained Transformer encoder (with dropout turned on) twice to obtain the two corresponding embeddings to build a positive pair. As the length information of a sentence will generally be encoded into the sentence embeddings due to the usage of position embedding in Transformer, each positive pair in unsup-SimCSE actually contains the same length information. And thus unsup-SimCSE trained with these positive pairs is probably biased, which would tend to consider that sentences of the same or similar length are more similar in semantics. Through statistical observations, we find that unsup-SimCSE does have such a problem. To alleviate it, we apply a simple repetition operation to modify the input sentence, and then pass the input sentence and its modified counterpart to the pre-trained Transformer encoder, respectively, to get the positive pair. Additionally, we draw inspiration from the community of computer vision and introduce a momentum contrast, enlarging the number of negative pairs without additional calculations. The proposed two modifications are applied on positive and negative pairs separately, and build a new sentence embedding method, termed Enhanced Unsup-SimCSE (ESimCSE). We evaluate the proposed ESimCSE on several benchmark datasets w.r.t the semantic text similarity (STS) task. Experimental results show that ESimCSE outperforms the state-of-the-art unsup-SimCSE by an average Spearman correlation of 2.02% on BERT-base.

In recent years, multimodal sentiment analysis (MSA) has attracted more and more interest, which aims to predict the sentiment polarity expressed in a video. Existing methods typically 1) treat three modal features (textual, acoustic, visual) equally, without distinguishing the importance of different modalities; and 2) split the video into frames, leading to missing the global acoustic information. In this paper, we propose a global Acoustic feature enhanced Modal-Order-Aware network (AMOA) to address these problems. Firstly, a modal-order-aware network is designed to obtain the multimodal fusion feature. This network integrates the three modalities in a certain order, which makes the modality at the core position matter more. Then, we introduce the global acoustic feature of the whole video into our model. Since the global acoustic feature and multimodal fusion feature originally reside in their own spaces, contrastive learning is further employed to align them before concatenation. Experiments on two public datasets show that our model outperforms the state-of-the-art models. In addition, we also generalize our model to the sentiment with more complex semantics, such as sarcasm detection. Our model also achieves state-of-the-art performance on a widely used sarcasm dataset.

The widespread of fake news has detrimental societal effects. Recent works model information propagation as graph structure and aggregate structural features from user interactions for fake news detection. However, they usually neglect a broader propagation uncertainty issue, caused by some missing and unreliable interactions during actual spreading, and suffer from learning accurate and diverse structural properties. In this paper, we propose a novel dual graph-based model, Uncertainty-aware Propagation Structure Reconstruction (UPSR) for improving fake news detection. Specifically, after the original propagation modeling, we introduce propagation structure reconstruction to fully explore latent interactions in the actual propagation. We design a novel Gaussian Propagation Estimation to refine the original deterministic node representation by multiple Gaussian distributions and arise latent interactions with KL divergence between distributions in a multi-facet manner. Extensive experiments on two real-world datasets demonstrate the effectiveness and superiority of our model.

Fake news’s quick propagation on social media brings severe social ramifications and economic damage. Previous fake news detection usually learn semantic and structural patterns within a single target propagation tree. However, they are usually limited in narrow signals since they do not consider latent information cross other propagation trees. Motivated by a common phenomenon that most fake news is published around a specific hot event/topic, this paper develops a new concept of propagation forest to naturally combine propagation trees in a semantic-aware clustering. We propose a novel Unified Propagation Forest-based framework (UniPF) to fully explore latent correlations between propagation trees to improve fake news detection. Besides, we design a root-induced training strategy, which encourages representations of propagation trees to be closer to their prototypical root nodes. Extensive experiments on four benchmarks consistently suggest the effectiveness and scalability of UniPF.

Unsupervised contrastive sentence embedding models, e.g., unsupervised SimCSE, use the InfoNCE loss function in training. Theoretically, we expect to use larger batches to get more adequate comparisons among samples and avoid overfitting. However, increasing batch size leads to performance degradation when it exceeds a threshold, which is probably due to the introduction of false-negative pairs through statistical observation. To alleviate this problem, we introduce a simple smoothing strategy upon the InfoNCE loss function, termed Gaussian Smoothed InfoNCE (GS-InfoNCE). In other words, we add random Gaussian noise as an extension to the negative pairs without increasing the batch size. Through experiments on the semantic text similarity tasks, though simple, the proposed smoothing strategy brings improvements to unsupervised SimCSE.

Capturing emotions within a conversation plays an essential role in modern dialogue systems.However, the weak correlation between emotions and semantics brings many challenges to emotion recognition in conversation (ERC).Even semantically similar utterances, the emotion may vary drastically depending on contexts or speakers.In this paper, we propose a Supervised Prototypical Contrastive Learning (SPCL) loss for the ERC task.Leveraging the Prototypical Network, the SPCL targets at solving the imbalanced classification problem through contrastive learning and does not require a large batch size.Meanwhile, we design a difficulty measure function based on the distance between classes and introduce curriculum learning to alleviate the impact of extreme samples.We achieve state-of-the-art results on three widely used benchmarks.Further, we conduct analytical experiments to demonstrate the effectiveness of our proposed SPCL and curriculum learning strategy.We release the code at https://github.com/caskcsg/SPCL.⋆

Emotion recognition in conversations (ERC) has attracted increasing interests in recent years, due to its wide range of applications, such as customer service analysis, health-care consultation, etc. One key challenge of ERC is that users' emotions would change due to the impact of others' emotions. That is, the emotions within the conversation can spread among the communication participants. However, the spread impact of emotions in a conversation is rarely addressed in existing researches. To this end, we propose EmotionFlow for ERC with the consideration of the spread of participants' emotions during a conversation. EmotionFlow first encodes users' utterance by concatenating the context with an auxiliary question, which helps to learn user-specific features. Then, conditional random field is applied to capture the sequential information at emotional level. We conduct extensive experiments on a public dataset Multimodal EmotionLines Dataset (MELD), and the results demonstrate the effectiveness of our proposed model.

Understanding temporal relations (TempRels) between events is an important task that could benefit many downstream NLP applications. This task inevitably faces the challenges of both a limited amount of high-quality training data and a very biased distribution of TempRels. These problems will substantially hurt the performance of extraction systems because they are inclined to predict dominant TempRels when training with a limited amount of data. To alleviate those issues, we propose a Knowledge/Data Enhanced method for Event and TempRel Extraction, which integrates the temporal commonsense knowledge, data augmentation and Focal Loss function into one single extraction system. Altogether, these components improve the performance of the system on two public benchmark datasets TB-Dense and MATRES1.

Multi-label image classification task aims to predict multiple object labels in a given image and faces the challenge of variable-sized objects. Limited by the size of CNN convolution kernels, existing CNN-based methods have difficulty capturing global dependencies and effectively fusing multiple layers features, which is critical for this task. Recently, transformers have utilized multi-head attention to extract feature with long range dependencies. Inspired by this, this paper proposes a Cross-layer Aggregation with Transformers (CAT) framework, which leverages transformers to capture the long range dependencies of CNN-based features with Long Range Dependencies module and aggregate the features layer by layer with Cross-Layer Fusion module. To make the framework efficient, a multi-head pre-max attention is designed to reduce the computation cost when fusing the high-resolution features of lower-layers. On two widely-used benchmarks (i.e., VOC2007 and MS-COCO), CAT provides a stable improvement over the baseline and produces a competitive performance.

Multi-label image classification (MLIC) is a more challenging task compared with single-label image classification due to multiple concepts targets, and complex visual relationships should be formulated. Convolutional Neural Network (CNN) and Visual Transformer (ViT) have shown superior performance in local and global feature representations, respectively. However, the interactions between local and global features are neglected in current works. To further formulate the critical interactions, this paper designs a Unified Feature Interaction (UFI) framework, aiming to integrate the selected local features with global features based on CNN and ViT, simultaneously. The proposed UFI includes two key modules: Class-Related Feature Selection (CRFS) and Feature Interaction Attention (FIA) modules. Specifically, according to the activation map, CRFS selects target regions by the preliminary calculation of predicted scores. FIA enables the significant local-global feature interaction based on the selected target regions and whole image. We initially attempted to interact with local and global features for multi-label image classification. UFI provides a stable improvement over the baseline and produces a new state-of-the-art result on MS-COCO and VOC2007.

The emotion recognition in conversation (ERC) task aims to predict the emotion label of an utterance in a conversation. Since the dependencies between speakers are complex and dynamic, which consist of intra- and inter-speaker dependencies, the modeling of speaker-specific information is a vital role in ERC. Although existing researchers have proposed various methods of speaker interaction modeling, they cannot explore dynamic intra- and inter-speaker dependencies jointly, leading to the insufficient comprehension of context and further hindering emotion prediction. To this end, we design a novel speaker modeling scheme that explores intra- and inter-speaker dependencies jointly in a dynamic manner. Besides, we propose a Speaker-Guided Encoder-Decoder (SGED) framework for ERC, which fully exploits speaker information for the decoding of emotion. We use different existing methods as the conversational context encoder of our framework, showing the high scalability and flexibility of the proposed framework. Experimental results demonstrate the superiority and effectiveness of SGED.

Abstract syntax trees (ASTs) play a crucial role in source code representation. However, due to the large number of nodes in an AST and the typically deep AST hierarchy, it is challenging to learn the hierarchical structure of an AST effectively. In this paper, we propose HELoC, a hierarchical contrastive learning model for source code representation. To effectively learn the AST hierarchy, we use contrastive learning to allow the network to predict the AST node level and learn the hierarchical relationships between nodes in a self-supervised manner, which makes the representation vectors of nodes with greater differences in AST levels farther apart in the embedding space. By using such vectors, the structural similarities between code snippets can be measured more precisely. In the learning process, a novel GNN (called Residual Self-attention Graph Neural Network, RSGNN) is designed, which enables HELoC to focus on embedding the local structure of an AST while capturing its overall structure. HELoC is self-supervised and can be applied to many source code related downstream tasks such as code classification, code clone detection, and code clustering after pre-training. Our extensive experiments demonstrate that HELoC outperforms the state-of-the-art source code representation models.

Frequently happening of disk failures seriously affects the dependability and service quality of cloud data centers. Recently, machine learning (ML) based methods are popularly adopted to proactively predict forthcoming disk failures via supervised learning. However, the high imbalance of failure samples and healthy samples is a huge obstacle for existing detection methods to establish high performance detection model. This paper presents a data augmentation method MSGMD, which can efficiently generate high quality failure samples to alleviate the data imbalance of the training set, so as to effectively improve the performance of any supervised failure detection models. First, MSGMD converts failure samples (multivariate time series) into multiple univariate time series via decomposing the spatial relations among features. Then it learns the temporal correlation of each feature via a policy-based reinforcement learning model trained in an adversarial way. After that, it generates failure samples by combining feature series sampled from learned distribution. Finally, it filters out low quality generated samples with a confidence-based method. Experimental results on real-world datasets show that, through data augmentation, MSGMD can improve the FDR and F1-Score of the state-of-the-art disk failure detection model by 31.59% and 30.74% respectively on average.

Nowadays, the hate speech diffusion of texts and images in social network has become the mainstream compared with the diffusion of texts-only, raising the pressing needs of multimodal hate speech detection task. Current research on this task mainly focuses on the construction of multimodal models without considering the influence of the unbalanced and widely distributed samples for various attacks in hate speech. In this situation, introducing enhanced knowledge is necessary for understanding the attack category of hate speech comprehensively. Due to the high correlation between hate speech detection and sarcasm detection tasks, this paper makes an initial attempt of common knowledge transfer based on the above two tasks, where hate speech detection and sarcasm detection are defined as primary and auxiliary tasks, respectively. A scalable cross-domain knowledge transfer (CDKT) framework is proposed, where the mainstream vision-language transformer could be employed as backbone flexibly. Three modules are included, bridging the semantic, definition and domain gaps simultaneously between primary and auxiliary tasks. Specifically, semantic adaptation module formulates the irrelevant parts between image and text in primary and auxiliary tasks, and disentangles with the text representation to align the visual and word tokens. Definition adaptation module assigns different weights to the training samples of auxiliary task by measuring the correlation between samples of the auxiliary and primary task. Domain adaptation module minimizes the feature distribution gap of samples in two tasks. Extensive experiments show that the proposed CDKT provides a stable improvement compared with baselines and produces a competitive performance compared with some existing multimodal hate speech detection methods.

Cross-domain aspect-based sentiment analysis aims to utilize the useful knowledge in a source domain to extract aspect terms and predict their sentiment polarities in a target domain. Recently, methods based on adversarial training have been applied to this task and achieved promising results. In such methods, both the source and target data are utilized to learn domain-invariant features through deceiving a domain discriminator. However, the task classifier is only trained on the source data, which causes the aspect and sentiment information lying in the target data can not be exploited by the task classifier. In this paper, we propose an Adaptive Hybrid Framework (AHF) for cross-domain aspect-based sentiment analysis. We integrate pseudo-label based semi-supervised learning and adversarial training in a unified network. Thus the target data can be used not only to align the features via the training of domain discriminator, but also to refine the task classifier. Furthermore, we design an adaptive mean teacher as the semi-supervised part of our network, which can mitigate the effects of noisy pseudo labels generated on the target data. We conduct experiments on four public datasets and the experimental results show that our framework significantly outperforms the state-of-the-art methods.

Qianwen Ma, Chunyuan Yuan, Wei Zhou, Songlin Hu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Lingwei Wei, Dou Hu, Wei Zhou, Zhaojuan Yue, Songlin Hu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Entity alignment aims to match synonymous entities across different knowledge graphs, which is a fundamental task for knowledge integration. Recently, researchers have devoted to leveraging rich information within relations to enhance entity alignment. They explicitly incorporate relations in entity representation and alignment, demonstrating remarkable results. However, affected by the semantic assumptions from early works, these works represent a relation by combining all the entities it connects, ignoring the semantic independence between entity and relation. Moreover, since these works perform alignment by comparing embedding similarity, they fail to consider a graph level alignment and tend to find local false correspondences.

In the text style transfer task, models modify the attribute style of given texts while keeping the style-irrelevant content unchanged. Previous work has proposed many approaches on the non-parallel corpus (without style-to-style training pairs). These approaches are mostly motivated by heuristic intuition and fail to precisely control texts’ attributes, such as the amount of preserved semantics, which leaves discrepancies between training and evaluation. This paper proposes a novel training method based on the evaluation metrics to address the discrepancy issue. Specifically, the model first evaluates different aspects of the transferred texts and provides the differentiable quality approximations by employing extra supervising modules. Then the model is optimized by bridging the gap between approximations and expectations. Extensive experiments conducted on two sentiment style datasets demonstrate the effectiveness of our proposal compared with some competitive baselines.

In social media, user identity linkage is a vital information security issue of identifying users’ private information across multiple online social networks. With the popularity of behavior-rich social services, existing methods attempt to align users through encoding behaviors. However, most of the efforts suffer from the high variety and heterogeneity of behavior data across social networks, resulting in a limitation of modeling user intrinsic characteristics. To address the above issues, we focus on keyword-based topics to formulate user’s variety behaviors for user identity linkage. In this paper, a novel Topic Sequence Embedding (TSeqE) method is proposed to embed contextual information of topics to represent users’ intrinsic characteristics for identity linkage. Furthermore, we introduce a domain-adversarial training strategy to tackle the behavior heterogeneity problem. Our experiments on two real-world datasets demonstrate that TSeqE produces a significant improvement compared with several strong baselines.

This paper focuses on the word disentanglement-based approaches for text style transfer. The related systems first remove the style attribute words in the given texts and then generate the target sentences using the remained neutral templates. Previous work retrieves the style words using intuitive heuristics, which lacks in-depth analysis and can hardly provide a precise word detection. The resulting error will further affect the generation phase and leads to the failure of the style transformation. In this paper, we formalize the style detection task as a dynamic decision process; each word in the given text is classified as a style or neutral content sequentially leveraging the word property, the context, and the decision history. As no background labels indicating the attribute words, reinforcement learning is deployed in our model. An evaluation (reward) function is designed to quantify the style shift and content loss simultaneously, providing an accurate estimation for the decision process. The efficiency of the proposal is verified on two types of style transfer tasks, i.e., sentiment transfer and formality transfer. The experimental results demonstrate the model's competitive performance compared to the state-of-the-art baselines once equipped with an insertion-based generation approach.

With the progress of power energy storage technologies in capacity, cycle life and reliability, data center can optimize its utilization of energy storage battery to reduce its Total Cost of Ownership (TCO). Data center can cut the peak and fill the valley of their power consumption graphs with proper management of battery charging and discharging, while maintaining uninterruptible power system (UPS) capacity. This paper studies the control technology of data center battery charging and discharging based on Deep Reinforcement Learning (DRL). According to the electricity price and status and cycle life of batteries, the appropriate time is selected to charge and discharge batteries in order to maximize the electricity bill savings. To achieve higher benefits, the system state, charging and discharging actions, reward function and a neural network structure were designed in detail. According to the simulation results, the proposed algorithm can infer the best savings strategy in both USA and Beijing electricity price systems. Compared with the baseline algorithm, the priority experience playback Deep Q-network (DQN) can increase the energy storage savings up to 47% and 55% with the electricity prices of the USA and Beijing respectively.

The rapid development of social media changes the lifestyle of people and simultaneously provides an ideal place for publishing and disseminating rumors, which severely exacerbates social panic and triggers a crisis of social trust. Early content-based methods focused on finding clues from the text and user profiles for rumor detection. Recent studies combine the stances of users' comments with news content to capture the difference between true and false rumors. Although the user's stance is effective for rumor detection, the manual labeling process is time-consuming and labor-intensive, which limits the application of utilizing it to facilitate rumor detection. In this paper, we first finetune a pre-trained BERT model on a small labeled dataset and leverage this model to annotate weak stance labels for users' comment data to overcome the problem mentioned above. Then, we propose a novel Stance-aware Reinforcement Learning Framework (SRLF) to select high-quality labeled stance data for model training and rumor detection. Both the stance selection and rumor detection tasks are optimized simultaneously to promote both tasks mutually. We conduct experiments on two commonly used real-world datasets. The experimental results demonstrate that our framework outperforms the state-of-the-art models significantly, which confirms the effectiveness of the proposed framework.

The rapid development of social media changes the lifestyle of people and simultaneously provides an ideal place for publishing and disseminating rumors, which severely exacerbates social panic and triggers a crisis of social trust. Early content-based methods focused on finding clues from the text and user profiles for rumor detection. Recent studies combine the stances of users' comments with news content to capture the difference between true and false rumors. Although the user's stance is effective for rumor detection, the manual labeling process is time-consuming and labor-intensive, which limits the application of utilizing it to facilitate rumor detection. In this paper, we first finetune a pre-trained BERT model on a small labeled dataset and leverage this model to annotate weak stance labels for users' comment data to overcome the problem mentioned above. Then, we propose a novel Stance-aware Reinforcement Learning Framework (SRLF) to select high-quality labeled stance data for model training and rumor detection. Both the stance selection and rumor detection tasks are optimized simultaneously to promote both tasks mutually. We conduct experiments on two commonly used real-world datasets. The experimental results demonstrate that our framework outperforms the state-of-the-art models significantly, which confirms the effectiveness of the proposed framework. In this paper, we first finetune a pre-trained BERT model on a small labeled dataset and leverage this model to annotate weak stance labels for users' comment data to overcome the problem mentioned above. Then, we propose a novel Stance-aware Reinforcement Learning Framework (SRLF) to select high-quality labeled stance data for model training and rumor detection. Both the stance selection and rumor detection tasks are optimized simultaneously to promote both tasks mutually. We conduct experiments on two commonly used real-world datasets. The experimental results demonstrate that our framework outperforms the state-of-the-art models significantly, which confirms the effectiveness of the proposed framework.

Summary Convolutional neural network (CNN) inference usually runs on high‐performance graphic processing units (GPUs). Since GPU is a high power consumption unit, that makes the energy consumption increases sharply due to the deep learning tasks. The energy efficiency of CNN inference is not only related to the software and hardware configurations, but also closely related to the application requirements of inference tasks. However, it is not clear on GPUs at present. In this paper, we conduct a comprehensive study on the model‐level and layer‐level energy efficiency of popular CNN models. The results point out several opportunities for further optimization. We also analyze the parameter settings (i.e., batch size, dynamic voltage and frequency scaling) and propose a revenue model to allow an optimal trade‐off between energy efficiency and latency. Compared with the default settings, the optimal settings can improve revenue by up to 15.31 × . We obtain the following main findings: (i) GPUs do not exploit the parallelism from the model depth and small convolution kernels, resulting in low energy efficiency. (ii) Convolutional layers are the most energy‐consuming CNN layers. However, due to the cache, the power consumption of all layers is relatively balanced. (iii) The energy efficiency of TensorRT is 1.53 × than that of TensorFlow.

Social media has become a preferential place for sharing information. However, some users may create multiple accounts and manipulate them to deceive legitimate users. Most previous studies utilize verbal or behavior features based methods to solve this problem, but they are only designed for some particular platforms, leading to low universalness.In this paper, to support multiple platforms, we construct interaction tree for each account based on their social interactions which is common characteristic of social platforms. Then we propose a new method to calculate the social interaction entropy of each account and detect the accounts which are controlled by the same user. Experimental results on two real-world datasets show that the method has robust superiority over state-of-the-art methods.

Metric learning based methods have attracted extensive interests in recommender systems. Current methods take the user-centric way in metric space to ensure the distance between user and negative item to be larger than that between the current user and positive item by a fixed margin. While they ignore the relations among positive item and negative item. As a result, these two items might be positioned closely, leading to incorrect results. Meanwhile, different users usually have different preferences, the fixed margin used in those methods can not be adaptive to various user biases, and thus decreases the performance as well. To address these two problems, a novel Symmetic Metric Learning with adaptive margin (SML) is proposed. In addition to the current user-centric metric, it symmetically introduces a positive item-centric metric which maintains closer distance from positive items to user, and push the negative items away from the positive items at the same time. Moreover, the dynamically adaptive margins are well trained to mitigate the impact of bias. Experimental results on three public recommendation datasets demonstrate that SML produces a competitive performance compared with several state-of-the-art methods.

Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evidence from external knowledge and make predictions based on the evidence. Recent studies either learn to generate evidence from human-annotated evidence which is expensive to collect, or extract evidence from either structured or unstructured knowledge bases which fails to take advantages of both sources simultaneously. In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence. Specifically, we extract evidence from both structured knowledge base (i.e. ConceptNet) and Wikipedia plain texts. We construct graphs for both sources to obtain the relational structures of evidence. Based on these graphs, we propose a graph-based approach consisting of a graph-based contextual word representation learning module and a graph-based inference module. The first module utilizes graph structural information to re-define the distance between words for learning better contextual word representations. The second module adopts graph convolutional network to encode neighbor information into the representations of nodes, and aggregates evidence with graph attention mechanism for predicting the final answer. Experimental results on CommonsenseQA dataset illustrate that our graph-based approach over both knowledge sources brings improvement over strong baselines. Our approach achieves the state-of-the-art accuracy (75.3%) on the CommonsenseQA dataset.

Event-oriented news retrieval (ENR) is the task of retrieving news articles related to the specific event in response to the event-oriented query. Previous approaches usually focus on optimizing traditional retrieval models through hand-crafted features from the perspective of new articles. However, these approaches often fail to work well in reality, as they do not consider the essential natures of the event, i.e., dynamics, coupling. In this paper, we propose a novel and effective event-oriented neural ranking model for news retrieval (ENRMNR). Our model exploits a deep attention mechanism to tackle the dynamics and coupling derived from event evolution. Specifically, the word-level bidirectional attention allows the model to identify which query words about the subevent are related to the news article words, and vice-versa, in order to tackle the dynamics. Moreover, the hierarchical attention at passage-level and document-level allows it to capture fine-grained event representations for the coupling between different events within a news article. Experimental results on real-world datasets demonstrate that ENRMNR model significantly outperforms competitive models.

Script learning aims to predict the subsequent event according to the existing event chain.Recent studies focus on event co-occurrence to solve this problem.However, few studies integrate external event knowledge to solve this problem.With our observations, external event knowledge can provide additional knowledge like temporal or causal knowledge for understanding event chain better and predicting the right subsequent event.In this work, we integrate event knowledge from ASER (Activities, States, Events and their Relations) knowledge base to help predict the next event.We propose a new approach consisting of knowledge retrieval stage and knowledge integration stage.In the knowledge retrieval stage, we select relevant external event knowledge from ASER.In the knowledge integration stage, we propose three methods to integrate external knowledge into our model and infer final answers.Experiments on the widely-used Multi-Choice Narrative Cloze (MCNC) task show our approach achieves state-of-the-art performance compared to other methods.

The dissemination of fake news significantly affects personal reputation and public trust.Recently, fake news detection has attracted tremendous attention, and previous studies mainly focused on finding clues from news content or diffusion path.However, the required features of previous models are often unavailable or insufficient in early detection scenarios, resulting in poor performance.Thus, early fake news detection remains a tough challenge.Intuitively, the news from trusted and authoritative sources or shared by many users with a good reputation is more reliable than other news.Using the credibility of publishers and users as prior weakly supervised information, we can quickly locate fake news in massive news and detect them in the early stages of dissemination.In this paper, we propose a novel Structure-aware Multi-head Attention Network (SMAN), which combines the news content, publishing, and reposting relations of publishers and users, to jointly optimize the fake news detection and credibility prediction tasks.In this way, we can explicitly exploit the credibility of publishers and users for early fake news detection.We conducted experiments on three real-world datasets, and the results show that SMAN can detect fake news in 4 hours with an accuracy of over 91%, which is much faster than the state-of-the-art models.

Existing gradient compression schemes fail to automatically determine the compression ratio or are accompanied by high compression overhead. To address this, we present Tail, an automated and lightweight gradient compression framework stacked by three modules, quantization, sparsification, and encoding. Without any hand-tuned effort, quantization module automatically adjusts the compression ratio along training iterations to retain accuracy first. Then, sparsification and encoding modules are successively applied to the quantized gradient to further improve compression ratio. Moreover, Tail reduces the compression overhead by approximate computing in the automated decision-making process. Experiments validate that Tail can reduce communication traffic by an order of magnitude while retaining or even improving model accuracy.

To accelerate distributed deep learning, gradient quantization technique is widely used to reduce the communication cost. However, the existing quantization schemes suffer from either model accuracy degradation or low compression ratio (arisen from a redundant setting of quantization level or high overhead in determining the level). In this work, we propose a novel adaptive quantization scheme (AdaQS) to explore the balance between model accuracy and quantization level. AdaQS determines the quantization level automatically according to gradient's mean to standard deviation ratio (MSDR). Then, to reduce the quantization overhead, we employ a computationally-friendly way of moment estimation to calculate the MSDR. Finally, theoretical analysis of AdaQS's convergence is conducted for non-convex objectives. Experiments demonstrate that AdaQS performs excellently on very deep model GoogleNet with 2.55% accuracy improvement relative to vanilla SGD and achieves 1.8x end-to-end speedup on AlexNet in a distributed cluster with 4*4 GPUs.

In recommender systems, the user uncertain preference results in unexpected ratings. Previous approaches (e.g., BiasMF) only adjust the rating value based on the bias vector, ignoring the uncertainty of rating. This paper makes an initial attempt in integrating the influence of user uncertain degree and user rating bias into the matrix factorization framework, simultaneously. An approach based on fuzzy set, called fuZzy Matrix Factorization (ZMF), is proposed. Specifically, a fuzzy set of like is defined for each user, and the membership function is utilized to measure the degree of an item belonging to the fuzzy set. Then, the user uncertain preference matrix is obtained, which could explain and represent the user bias and uncertainty effectively. Furthermore, to enhance the computational impact on sparse matrix, the uncertain preference is formulated as a side-information for fusion. Besides, the proposed approach could be extended to others due to independency on additional data sources. Experimental results on three datasets show that ZMF produces an effective improvement.

The task of Target-oriented Opinion Words Extraction aims to extract the corresponding opinion words for a given opinion target from the sentence. Recently, the methods based on recurrent neural networks have shown promising results for this task. However, these approaches only considered the sequential information of the sentences and ignored the syntactic structure. In this paper, we propose a novel graph convolutional network with adversarial training to extract the opinion words. We present a graph convolutional network based on dependency tree to learn the syntactic representation of the input. Besides, we train our model with the mixture of original examples and adversarial examples, which can improve the robustness of the model. We conduct experiments on four benchmarking datasets and the results illustrate that our proposed model consistently outperforms the state-of-the-art methods.

Music genres are useful for indexing, organizing, searching, and recommending songs and albums. Therefore, the automatic classification of music genres is an essential part of almost all kinds of music applications. Recent works focus on exploiting text, audio, or multi-modal information for genre classification, without considering the influence of the artists' and listeners' preference. However, intuitively, artists have their composing preferences, and listeners also have their music tastes. Both of them provide helpful hints to the music genre from different views, which are crucial to improve classification performance.

Automatically labeling multiple styles for every song is a comprehensive application in all kinds of music websites. Recently, some researches explore review-driven multi-label music style classification and exploit style correlations for this task. However, their methods focus on mining the statistical relations between different music styles and only consider shallow style relations. Moreover, these statistical relations suffer from the underfitting problem because some music styles have little training data. To tackle these problems, we propose a novel knowledge relations integrated framework (KRF) to capture the complete style correlations, which jointly exploits the inherent relations between music styles according to external knowledge and their statistical relations. Based on the two types of relations, we use graph convolutional network to learn the deep correlations between styles automatically. Experimental results show that our framework significantly outperforms the state-of-the-art methods. Further studies demonstrate that our framework can effectively alleviate the underfitting problem and learn meaningful style correlations.

In recommender systems, the user uncertain preference results in unexpected ratings. This paper makes an initial attempt in integrating the influence of user uncertain degree into the matrix factorization framework. Specifically, a fuzzy set of like for each user is defined, and the membership function is utilized to measure the degree of an item belonging to the fuzzy set. Furthermore, to enhance the computational effect on sparse matrix, the uncertain preference is formulated as a side-information for fusion. Experimental results on three real-world datasets show that the proposed approach produces stable improvements compared with others.

Scripts represent knowledge of event sequences that can help text understanding. Script event prediction requires to measure the relation between an existing chain and the subsequent event. The dominant approaches either focus on the effects of individual events, or the influence of the chain sequence. However, only considering individual events will lose much semantic relations within the event chain, and only considering the sequence of the chain will introduce much noise. With our observations, both the individual events and the event segments within the chain can facilitate the prediction of the subsequent event. This paper develops self attention mechanism to focus on diverse event segments within the chain and the event chain is represented as a set of event segments. We utilize the event-level attention to model the relations between subsequent events and individual events. Then, we propose the chain-level attention to model the relations between subsequent events and event segments within the chain. Finally, we integrate event-level and chain-level attentions to interact with the chain to predict what happens next. Comprehensive experiment results on the widely used New York Times corpus demonstrate that our model achieves better results than other state-of-the-art baselines by adopting the evaluation of Multi-Choice Narrative Cloze task.

Chunyuan Yuan, Wei Zhou, Mingming Li, Shangwen Lv, Fuqing Zhu, Jizhong Han, Songlin Hu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

With the implementation of mainstream DL frameworks, scarce GPU memory resource is the primary bottleneck that hinders the trainability and training efficiency of ultra-deep neural networks (UDNN). Prior memory optimization works focus on removing the trainability restriction but leave the training efficiency out of consideration. To fill the gap, we present "AccUDNN", an accelerator that aims to make full use of finite GPU memory resource to speed up the training process of UDNN in this paper. AccUDNN mainly includes two modules: memory optimizer and hyperparameter tuner. Memory optimizer develops a novel performance-model guided dynamic swap out/in strategy to meet trainability first and further remedy the efficiency degradation in other swapping strategies. Then, a hyperparameter tuner is designed to explore the efficiency-optimal minibatch size and the matched learning rate after applying the dynamic swapping strategy. Evaluations demonstrate that AccUDNN cuts down the GPU memory requirement of ResNet-152 from more than 24GB to 8GB. In turn, given 12GB GPU memory budget, the efficiency-optimal minibatch size can reach 4.2x larger than Caffe and finally improve the scaling efficiency (speedup) of 8 GPUs' cluster by 1.9x.

The development of social media has revolutionized the way people communicate, share information and make decisions, but it also provides an ideal platform for publishing and spreading rumors. Existing rumor detection methods focus on finding clues from text content, user profiles, and propagation patterns. However, the local semantic relation and global structural information in the message propagation graph have not been well utilized by previous works. In this paper, we present a novel global-local attention network (GLAN) for rumor detection, which jointly encodes the local semantic and global structural information. We first generate a better integrated representation for each source tweet by fusing the semantic information of related retweets with the attention mechanism. Then, we model the global relationships among all source tweets, retweets, and users as a heterogeneous graph to capture the rich structural information for rumor detection. We conduct experiments on three real-world datasets, and the results demonstrate that GLAN significantly outperforms the state-of-the-art models in both rumor detection and early detection scenarios.

Opinion spam has become a widespread problem in social media, where hired spammers write deceptive reviews to promote or demote products to mislead the consumers for profit or fame. Existing works mainly focus on manually designing discrete textual or behavior features, which cannot capture complex global semantics of reviews. Although recent works apply deep learning methods to learn review-level semantic features, their models ignore the impact of the user-level and product-level information on learning review semantics and the inherent user-review-product relationship information. In this paper, we propose a Hierarchical Fusion Attention Network (HFAN) to automatically learn the semantics of reviews from user and product level. Specifically, we design a multiattention unit to extract user(product)-related review information. Then, we use orthogonal decomposition and fusion attention to learn a user, review, and product representation from the review information. Finally, we take the review as a relation between user and product entity and apply TransH to jointly encode this relationship into review representation. Experimental results obtained more than 10% absolute precision improvement over the state-of-the-art performances on four real-world datasets, which show the effectiveness and versatility of the model.

The traditional convolutional neural networks (CNNs) learn numbers of fixed kernels (filters), which are used to obtain the representations of fixed patterns. Therefore, the knowledge representations of CNNs are limited to the number of kernels. In this paper, we present a Self-Representation Convolutional (SRC) layer to obtain richer knowledge representations of images by fully considering the self-similarity between adjacent pixels. SRC layer comprises a learnable local correlation measurement which measures the importance of adjacent pixels to the current pixel and two learnable linear parameters that perform linear projection on adjacent pixels and the weighted sum vectors, respectively. Compared with regular convolutional layers, the SRC layers can not only obtain comparable knowledge representations, but also reduce by a factor of 3× to 56× in the number of learnable parameters. Empirically, CNNs with SRC layers, called Self-Representation Convolutional Neural Networks (SRCNN), achieve strong performances on a range of visual datasets (SVHN, CIFAR-10 and CIFAR-100) while enjoying significant parameters and FLOPs savings.

The utilization of the unlabeled data provides a beneficial attempt for improving the generalization ability of the convolutional neural network (CNN) model, just as what is applied in person re-identification task. Different from that, multi-label image classification aims to predict multiple labels for each given image. The unlabeled data should be properly assigned multiple labels for regularizing the training process of CNN model. To make full use of the unlabeled data, this paper proposes a soft pseudo labeling (SPL) method for multi-label image classification. Specifically, the unlabeled samples are first generated by DCGAN and WGAN-GP. Then, the virtual multiple labels of the generated unlabeled samples are assigned based on an initial confidence value by SoftMax function. Finally, both the generated samples and original training samples are fed into the network as input, in order to learn a CNN model with stronger generalization ability. On three public multi-label image classification datasets (i.e., WIDER-Attribute, NUS-WIDE and MS-COCO), SPL provides a stable improvement over the baseline and produces a competitive performance compared with some existing multi-label image classification methods.

This paper studies the energy management problem in data centers. As the most energy-consuming sub-systems, server and cooling have been considered as the key of energy saving for data center. However, due to the fact that servers are not directly connected to cooling devices, it is not trivial to optimize them jointly. In this paper, we formulate the joint server and cooling energy management problem and analyze its challenges, and design a Learning-based Energy management Scheme (LEES) to solve the joint energy management problem. Firstly, machine learning methods are investigated in the cooling power modeling and cabinet temperature modeling. Based on the models, a simple yet efficient energy management scheme is proposed to find an optimized configuration of server load distribution, server status and air supply temperature setting for the data center to reduce the total energy consumption. Extensive simulation results demonstrate that our method is able to reduce the total energy consumption of data centers significantly.

This paper focuses on the task of sentiment transfer on non-parallel text, which modifies sentiment attributes (e.g., positive or negative) of sentences while preserving their attribute-independent contents. Existing methods adopt RNN encoder-decoder structure to generate a new sentence of a target sentiment word by word, which is trained on a particular dataset from scratch and have limited ability to produce satisfactory sentences. When people convert the sentiment attribute of a given sentence, a simple but effective approach is to only replace the sentiment tokens of the sentence with other expressions indicative of the target sentiment, instead of building a new sentence from scratch. Such a process is very similar to the task of Text Infilling or Cloze. With this intuition, we propose a two steps approach: Mask and Infill. In the \emph{mask} step, we identify and mask the sentiment tokens of a given sentence. In the \emph{infill} step, we utilize a pre-trained Masked Language Model (MLM) to infill the masked positions by predicting words or phrases conditioned on the context\footnote{In this paper, \emph{content} and \emph{context} are equivalent, \emph{style}, \emph{attribute} and \emph{label} are equivalent.}and target sentiment. We evaluate our model on two review datasets \emph{Yelp} and \emph{Amazon} by quantitative, qualitative, and human evaluations. Experimental results demonstrate that our model achieve state-of-the-art performance on both accuracy and BLEU scores.

Target-Based Sentiment Analysis aims at extracting opinion targets and classifying the sentiment polarities expressed on each target. Recently, token based sequence tagging methods have been successfully applied to jointly solve the two tasks, which aims to predict a tag for each token. Since they do not treat a target containing several words as a whole, it might be difficult to make use of the global information to identify that opinion target, leading to incorrect extraction. Independently predicting the sentiment for each token may also lead to sentiment inconsistency for different words in an opinion target. In this paper, inspired by span-based methods in NLP, we propose a simple and effective joint model to conduct extraction and classification at span level rather than token level. Our model first emulates spans with one or more tokens and learns their representation based on the tokens inside. And then, a span-aware attention mechanism is designed to compute the sentiment information towards each span. Extensive experiments on three benchmark datasets show that our model consistently outperforms the state-of-the-art methods.

Ultra-deep neural network(UDNN) tends to yield higher-quality model but its training process is often difficult to handle. Scarce GPU DRAM capacity is the primary bottleneck that limits the depth of neural network and the range of trainable minibatch size. In this paper, we present a scheme that dedicates to make the utmost use of finite GPU memory resource to speed up the training process for UDNN. Firstly, a performance-model guided dynamic swap out/in strategy between GPU and host memory is carefully orchestrated to tackle the out-of-memory problem without introducing performance penalty. Then, a hyperparameter (minibatch size, learning rate) tuning policy is designed to explore the optimal configuration after applying the swap strategy from the perspectives of training time and final accuracy simultaneously. Finally, we verify the effectiveness of our scheme in both single and distributed GPU mode.

Obfuscated language is created to avoid censorship in adversarial communication such as sensitive information conveying, strong sentiment expression, secret actions plan, and illegal trading. The obfuscated sentences are usually generated by replacing one word with another to conceal the textual content. Intelligence and security agencies identify such adversarial messages by scanning with a watch-list of red-flagged terms. Though semantic expansion techniques are adopted, the precision and recall of the identification is limited due to the ambiguity and the unbounded creation way. To this end, this paper frames the obfuscated language identification problem as a text matching task, where each message is checked whether matches a red-flagged term. We propose a multimodal text matching model which combining textual and visual features. The proposed model extends a Bi-directional Long Short Term Memory network with a visual-level representation component to achieve the given task. Comparative experiments on real-world dataset demonstrate that the proposed method could achieve a better performance than the previous methods.

Apache Hive has been widely used for big data processing over large scale clusters by many companies. It provides a declarative query language called HiveQL. The efficiency of filtering out query-irrelevant data from HDFS closely affects the performance of query processing. This is especially true for multi-dimensional, high-selective, and few columns involving queries, which provides sufficient information to reduce the amount of bytes read. Indexing (Compact Index, Aggregate Index, Bitmap Index, DGFIndex, and the index in ORC file) and columnar storage (RCFile, ORC file, and Parquet) are powerful techniques to achieve this. However, it is not trivial to choosing a suitable index and columnar storage based on data and query features. In this paper, we compare the data filtering performance of the above indexes with different columnar storage formats by conducting comprehensive experiments using uniform and skew TPC-H data sets and various multi-dimensional queries, and suggest the best practices of improving multi-dimensional queries in Hive under different conditions.

Current QoS-aware automatic service composition queries over a network of web services are often one-time in nature. After a network of web services is built, such queries are issued once, and answers are found from the scratch. The underlying assumption is that the participating web services are rather static, which means that their functional and non-functional parameters seldom change. However, such an assumption is often baseless. New services come and go, service APIs change gradually, and QoS values fluctuate in practice. Therefore, a support for efficiently adapting service composition is desired. In this paper, we propose an event driven continuous query algorithm to intelligently cope with different types of dynamic services, and thus enable evolution of service composition. Evaluation using both real QoS data and synthetic web service data shows its superior performance, compared with the state-of-the-art solution which won the performance championship of Web Service Challenge in 2009 and 2010. Moreover, we generalize a new graph problem: dynamic single-source optimal-Directed Acyclic Graphs (DAG) problem based on the above work. It is argued that API recommendation for framework evolution, as another typical application, could also be modeled and addressed efficiently using the proposed new graph approach.

An entity on the web can be referred by numerous morphs that are always ambiguous, implicit and informal, which makes it challenging to accurately identify all the morphs corresponding to a specific entity. In this paper, we introduce a novel method based on knowledge graph, which takes advantage of both knowledge reasoning and statistic learning. First, we present a model to build a knowledge graph for the given entity. The knowledge graph integrates the fragmented knowledge on how humans create morphs. Then, the candidate morphs are generated based on the rules summarized from the knowledge graph. At last, we use a classification method to filter the useless candidates and identify the target morphs. The experiments conducted on real world dataset demonstrate efficiency of our proposed method in terms of precision and recall.

Virtual machine (VM) scheduling is an important technique for the efficient operation of the computing resources in a data center. Previous work has mainly focused on consolidating VMs to improve resource utilization and to optimize energy consumption. However, the interference between collocated VMs is usually ignored, which can result in much worse performance degradation of the applications running on the VMs due to the contention of the shared resources. Based on this observation, we aim at designing efficient VM assignment and scheduling strategies in which we consider optimizing both the operational cost of the data center and the performance degradation of the running applications. We then propose a general model that captures the tradeoff between the two contradictory objectives. We present offline and online solutions for this problem by exploiting the spatial and temporal information of performance interference of VM collocation, where VM scheduling is performed by jointly considering the combinations and the life-cycle overlap of the VMs. Evaluation results show that the proposed methods can generate efficient schedules for VMs, achieving low operational cost while significantly reducing the performance degradation of applications in cloud data centers.

The subscription covering optimization, whereby a general subscription quenches the forwarding of more specific ones, is a common technique to reduce network traffic and routing state in content-based routing networks. Such optimizations, however, leave the system vulnerable to unsubscriptions that trigger the immediate forwarding of all the subscriptions they had previously quenched. These subscription bursts can severely congest the network, and destabilize the system. This paper presents techniques to retain much of the benefits of subscription covering while avoiding bursty subscription traffic. Heuristics are used to estimate the similarity among subscriptions, and a distributed algorithm determines the portions of a subscription propagation tree that should be preserved. Evaluations show that these mechanisms avoid subscription bursts while maintaining relatively compact routing tables.

Hive is the most mature and prevalent data warehouse tool providing SQL-like interface in the Hadoop ecosystem. It is successfully used in many Internet companies and shows its value for big data processing in traditional industries. However, enterprise big data processing systems as in Smart Grid applications usually require complicated business logics and involve many data manipulation operations like updates and deletes. Hive cannot offer sufficient support for these while preserving high query performance. Hive using the Hadoop Distributed File System (HDFS) for storage cannot implement data manipulation efficiently and Hive on HBase suffers from poor query performance even though it can support faster data manipulation. There is a project based on Hive issue Hive-5317 to support update operations, but it has not been finished in Hive's latest version. Since this ACID compliant extension adopts same data storage format on HDFS, the update performance problem is not solved. In this paper, we propose a hybrid storage model called DualTable, which combines the efficient streaming reads of HDFS and the random write capability of HBase. Hive on DualTable provides better data manipulation support and preserves query performance at the same time. Experiments on a TPC-H data set and on a real smart grid data set show that Hive on DualTable is up to 10 times faster than Hive when executing update and delete operations.

Apache Hive has been widely used by Internet companies for big data analytics applications. It can provide the capability of compiling high-level languages into efficient MapReduce workflows, which frees users from complicated and time consuming programming. The popularity of Hive and its HiveQL-compatible systems like Impala and Shark attracts attentions from traditional enterprises as well. However, enterprise big data processing systems such as Smart Grid applications often have to migrate their RDBMS-based legacy applications to Hive rather than directly writing new logic in HiveQL. Considering their differences in syntax and cost model, manual translation from SQL in RDBMS to HiveQL is very difficult, error-prone, and often leads to poor performance.

In Smart Grid applications, as the number of deployed electric smart meters increases, massive amounts of valuable meter data is generated and collected every day. To enable reliable data collection and make business decisions fast, high throughput storage and high-performance analysis of massive meter data become crucial for grid companies. Considering the advantage of high efficiency, fault tolerance, and price-performance of Hadoop and Hive systems, they are frequently deployed as underlying platform for big data processing. However, in real business use cases, these data analysis applications typically involve multidimensional range queries (MDRQ) as well as batch reading and statistics on the meter data. While Hive is high-performance at complex data batch reading and analysis, it lacks efficient indexing techniques for MDRQ. In this paper, we propose DGFIndex, an index structure for Hive that efficiently supports MDRQ for massive meter data. DGFIndex divides the data space into cubes using the grid file technique. Unlike the existing indexes in Hive, which stores all combinations of multiple dimensions, DGFIndex only stores the information of cubes. This leads to smaller index size and faster query processing. Furthermore, with pre-computing user-defined aggregations of each cube, DGFIndex only needs to access the boundary region for aggregation query. Our comprehensive experiments show that DGFIndex can save significant disk space in comparison with the existing indexes in Hive and the query performance with DGFIndex is 2-50 times faster than existing indexes in Hive and HadoopDB for aggregation query, 2-5 times faster than both for non-aggregation query, 2-75 times faster than scanning the whole table in different query selectivity.

With the proliferation of Web services, service engineers demand automatic service composition algorithms that not only synthesize the correct service compositions from thousands of services but also satisfy the quality requirements of users. This is known as QoS-aware automatic service composition problem. Our observation is that current research of only finding the optimal service composition result has several shortcomings. Users have to utilize the optimal one, which will make it rigid, and consequently bring about problems, such as overload of “hot services” and lack of choices for users. To cope with these problems, a top k query mechanism is introduced in this paper, and a progressive and incremental Key-Path-Based Loose (KPL) algorithm with 100 percent accuracy is proposed. Our QSynth, which won the performance championship of Web Service Challenge 2009 and 2010, is extended to support top k query based on KPL algorithm. Evaluations show that, compared to the state of the art, KPL algorithm achieves superior scalability and accuracy with respect to a large variety of composition scenarios. Moreover, we generalize a new graph problem: top k DAGs (Directed Acyclic Graphs) problem based on the above work. Applications of this new graph problem contain API recommender, supply chain, and so on. KPL algorithm illustrated in this paper can address them efficiently, too.

Virtual machines (VMs) consolidation has emerged as an important method for the design of energy-efficient data centers. The purpose is to aggregate VMs to fewer physical machines and put the idle servers into power-saving mode. Existing researches mainly focus on transforming the VMs consolidation into various bin packing problems. However, VMs consolidation may cost Service Level Agreement (SLA) violations just after the migration due to the uncertainty of applications' demands. In this paper, we provide a SLA risk management framework, involving a stochastic program to solve the resource allocation for VMs and an algorithm for dynamic VMs consolidation at runtime, to optimize both the energy consumption saving and SLA violations. We validate the proposed algorithm using workloads from a real world system. The results compare with other VMs consolidation algorithms that without considering risk, and show that our SLA violations is reduced by four times from 25% to 2% - 5% while only losing little energy consumption saving.

This paper introduces the queue reorganization algorithm to alleviate congestion problem in pub/sub systems. Experimental studies show that our solution can greatly reduce the number of messages congested in the input queue, which in turn alleviates system pressure from congested queues.

Content-based publish/subscribe systems allow subscribers to specify interests based on event contents, rather than pre-assigned event topics. Since most of existing publish/subscribe systems use FIFO (First In First Out) as its queue management mechanism, it may face network congestion coming from suddenly burst of huge amount of subscription messages as well as event notifications. To promote the ability to handle congestion challenge, we present a new queue management mechanism called queue reorganization algorithm. By taking advantage of similarities among concurrent subscriptions, queue reorganization algorithm can make covering subscriptions as proxies and remove unnecessary subscriptions at each broker, which in turn greatly reduces pressure from congested input queue. Detailed evaluations under various workloads show significant benefits of the optimizations in terms of input queue size and message routing cost.

Optimized placement of clients in a distributed publish/subscribe system is an important technique to improve overall system efficiency. Current methods, like interest clustering or publisher placement, treat a client as, either a pure publisher, or subscriber, but not as both. Also, the cost of client movement is usually ignored. However, many applications based on publish/subscribe systems model clients as publisher and subscriber at the same time, which breaks the assumptions made by current approaches. Considering the complex dependency among clients, we propose a new community-oriented clustering approach, based on the forming of client clusters that exhibit intense communication relationships, while keeping client movement cost low. The evaluation based on a public data set shows that our method is efficient, adapts to different settings of experimental conditions, and wins over the popular interest clustering approach with respect to number of messages sent, propagation hop count and end-to-end latency.

Current QoS-aware automatic service composition queries over a network of Web services are often one-time innature. After a network of Web services is built, such queries are issued once, and answers are found from the scratch. The underlying assumption is that the participating Web services are rather static so that their functional and non-functional parameters seldom change. However, such an assumption is often baseless. New services come and go, service APIs change gradually, and QoS values fluctuate. Therefore, a support for efficiently handling "continuous" service composition queries is desired. In this paper, we propose an event driven continuous query algorithm for QoS-aware automatic service composition problem to cope with different types of dynamic services. Moreover, we integrated this algorithm in our service composition system, QSynth. Finally, we evaluate our proposal using both real QoS data and synthetic Web service data and show the superior performance of ours, compared to the state-of-the art solution which won the performance championship of Web Service Challenge in 2009 and 2010.

While the usage of web services has increased explosively in recent years, very few studies examined the characteristics of web services using large-scale real data for a long period of time. In this paper, we present one such a large scale longitudinal analysis of publicly available web services of SOAP-based and RESTful types. For the period of roughly one year and from five different world-wide locations, we closely monitor the ups and downs of various basic properties of web services and their QoS values using a total of 825,132 real web services.

With the proliferation of Web services, service engineers demand good automatic service composition algorithms that not only synthesize the correct work plans from thousands of services but also satisfy the quality requirements of the users. Our observation is that conventional approaches suffer from serious limitations in scalability and accuracy when addressing both requirements simultaneously. We have designed and implemented a tool QSynth to use QoS objectives of service requests as the search directives. This approach effectively prunes the search space and significantly improves the accuracy of the search results. Evaluations show that, compared to the state of the art, QSynth achieves superior scalability and accuracy with respect to a large variety of composition scenarios. Our design of QSynth won the performance championship of Web Services Challenge 2009.

Programming forums are becoming the primary tools for programmers to find answers for their programming problems. Our empirical study of popular programming forums shows that the forum users experience long waiting period for answers and a small number of experts are often overloaded with questions. To improve the usage experience, we have designed and implemented G-Finder, both an algorithm and a tool that makes intelligent routing decisions as to which participant is the expert for answering a particular programming question. Our main approach is to leverage the source code information of the software systems that forums are dedicated to, and discover latent relationships between forums users. Our algorithms construct the concept networks and the user networks from the program source and the forum data. We use programming questions to dynamically integrate these two networks and present an adaptive ranking of the potential experts. Our evaluation of G-Finder, using the data from three large programming forums, takes a retrospective view to check if G-Finder can correctly predict the experts who provided answers to programming questions. The evaluation results show that G-Finder improves the prediction precision by 25% to 74%, compared to related approaches.

This paper formalizes transactional properties for publish/subscribe client mobility and develops protocols to realize them. Evaluations show that compared to traditional protocols, those developed in this paper, in addition to supporting transactional properties, are more stable with respect to message and processing overheads. Changes in factors such as the number of moving clients have little impact, making the protocols more scalable and simpler to administer due to predictable resource requirements.