Generative summarisation models have demonstrated remarkable fluency; however, they frequently encounter hallucinations and factual inconsistencies, especially in specialised domains that involve lengthy documents. Hybrid approaches aim to alleviate these issues by integrating extractive key sentences as guidance. Nevertheless, current methods generally utilise an ‘early fusion’ strategy, which involves merely concatenating key sentences with the source text. This approach results in two significant bottlenecks: representation mismatch and position bias. This paper proposes a novel Gated Dual-Encoder Summarisation Framework: unlike unified encoders, it uses two parameter-independent encoders to separately process source text and key information, avoiding semantic confusion. Additionally, we present a Dynamic Gated Fusion Module that adaptively weighs the contributions of the source context and guidance signals through a learnable gating scalar. Experimental results on CNN/DailyMail, XSum, and PubMed demonstrate that our approach significantly outperforms state-of-the-art baselines. Specifically, on the PubMed dataset, our model achieves a ROUGE-L score of 40.05 and a FactCC score of 86.4%, outperforming the BART-Large baseline by 7.2% in factual consistency. Furthermore, our model demonstrates high robustness with a performance degradation rate (PDR) of only 2.8% under sentence shuffling tests.

Large Language Models (LLMs) have received much recent attention due to their human-level accuracy. While existing works mostly focus on either improving accuracy or testing accuracy robustness, the computation efficiency of LLMs, which is of paramount importance due to often vast generation demands and real-time requirements, has surprisingly received little attention. In this article, we make the first attempt to understand and test potential computation efficiency robustness in state-of-the-art LLMs. By analyzing the working mechanism and implementation of 20,543 public-accessible LLMs, we observe a fundamental property in LLMs that could be manipulated in an adversarial manner to reduce computation efficiency significantly. Our interesting observation is that the output length determines the computation efficiency of LLMs instead of the input, where the output length depends on two factors: an often sufficiently large yet pessimistic pre-configured threshold controlling the max number of iterations and a runtime-generated end of sentence (EOS) token. Our key motivation is to generate test inputs that could sufficiently delay the generation of EOS such that LLMs would have to go through enough iterations to satisfy the pre-configured threshold. We present LLMEffiChecker , which can work under both white-box setting and black-box setting. In the white-box scenario, LLMEffiChecker develops a gradient-guided technique that searches for a minimal and unnoticeable perturbation at character-level, token-level, and structure-level. In the black-box scenario, LLMEffiChecker employs a causal inference-based approach to find critical tokens and similarly applies three levels of imperceptible perturbation to them. Both the white-box and black-box settings effectively delay the appearance of EOS, compelling these inputs to reach the naturally unreachable threshold. To demonstrate the effectiveness of LLMEffiChecker , we conduct a systematic evaluation on nine publicly available LLMs: Google T5, AllenAI WMT14, Helsinki-NLP translator, Facebook FairSeq, UNICAMP-DL translator, MarianMT, Google FLAN-T5, MBZUAI LaMini-GPT, and Salesforce CodeGen. Experimental results show that LLMEffiChecker can increase on average LLMs’ response latency and energy consumption by 325% to 3,244% and 344% to 3,616%, respectively, by perturbing just one character or token in the input sentence. Our case study shows that inputs generated by LLMEffiChecker significantly affect the battery power in real-world mobile devices (i.e., drain more than 30 times battery power than normal inputs).