Large Language Models (LLMs) have demonstrated remarkable performance in code generation, offering new possibilities for translating natural language into executable programs. To further enhance LLMs’ code generation capabilities, Retrieval-Augmented Generation (RAG) has emerged as a promising strategy by retrieving code examples aligned with the generation intent to guide the process. However, existing RAG-based methods often suffer from unnecessary augmentation, preference misalignment, and surface-level mimicry, which undermine the effectiveness of retrieved examples in guiding LLMs toward accurate code generation. To address these challenges, we propose SRACG, a Selective Retrieval-Augmented Code Generation framework. SRACG begins with a necessity-aware selection mechanism to identify generation intents that genuinely require retrieval support, thereby avoiding degradation from indiscriminate augmentation. For intents identified as needing enhancement, it first employs a multi-objective retrieval strategy to select examples that are semantically aligned with the intent. These candidates are then further filtered by assessing their consistency with the LLM’s inherent generation preferences, ensuring alignment in both style and structure. Finally, it extracts execution plans from the filtered examples to uncover their underlying logic, guiding the LLM to better comprehend the examples instead of merely mimicking surface-level content. Experimental results on widely used benchmarks show that SRACG significantly improves the success rate of LLM-generated code and outperforms existing approaches.

ZhenChun Xu, Yi Cai, Dajun Zheng, li Yuan, Mengchen Zhao, Qixiang Wang, Jiexin Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Ronghui Yang, Jie Liu, Jiajie Zeng, Jiexin Wang, Jiuchuan Jiang, Bo An, Yi Cai, Mengchen Zhao. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Pre-trained large language models (LLMs) have emerged as a breakthrough technology in code intelligence such as code generation. Recently, many works have found that LLMs can generate a correct code solution when it is allowed to make numerous attempts. Consequently, a recent trend is to do a large-scale sampling of codes from LLMs and then rank the code to select the most suitable code, a process called code ranking. A common code ranking approach ranks the code by running it against a set of LLM-generated test cases in the form of assert statements. However, existing approaches overlook how humans design test cases through systematic behaviors, which are essential for creating reliable tests for code ranking. Moreover, humans often use input–output examples to clarify and articulate the intended functionality, rather than merely to rank code. To address these gaps, we propose RankAgent, a human-inspired agent-based framework that systematically simulates human test design behaviors. Extensive experiments on five LLMs (including both open- and closed-source models) and two benchmarks (HumanEval+ and LiveCodeBench) show that RankAgent achieves notable and consistent improvements.

Repository-level code generation aims to generate target code conditioned on the context of the specified repository. Existing approaches generally adopt the Retrieval-Augmented Generation (RAG) framework to retrieve relevant contextual information from the repository, thereby mitigating the challenge of long context windows. However, these methods typically struggle to identify truly relevant information for the current generation requirement, and they generally overlook the imperative need for understanding the repository-level code structure during the retrieval process. To address these issues, we propose RepoMind, a novel framework that advances repository-level code generation by leveraging a repository documentation library and LLM reasoning capabilities for API retrieval. Specifically, RepoMind first introduces the RepoDocs Agent. This component constructs multi-granularity, hierarchical structural documentation in a bottom-up manner, providing LLMs with a high-quality knowledge base for understanding repository functionality and structure. Built upon this hierarchical knowledge base, RepoMind further integrates the Reasoning-Retrieval Agent. This agent mimics human developer patterns by utilizing the repository documentation for layer-by-layer exploration, and ultimately precisely localizes the relevant API sets. Finally, the framework combines the functionally relevant API set retrieved via LLM reasoning with the semantically similar API set retrieved by the vector retriever, utilizing this hybrid context to boost code generation performance. Evaluation on widely-used repository-level code generation benchmarks, CoderEval and DevEval, demonstrates that RepoMind surpasses state-of-the-art methods, achieving up to a 13.50% relative improvement in Pass@1 scores. The open-source code will be released at https://github.com/ABigTrouble/RepoMind.

Repository-level code generation has attracted growing interest, yet most benchmarks and methods remain maintainer-centric, emphasizing bug fixing and feature implementation. In contrast, a common yet underexplored scenario is repository usage: external users want to build applications by correctly invoking repository-internal APIs, composing them into runnable end-to-end workflows rather than modifying the codebase. To support this setting, we introduce RUCCE, a benchmark for repository-usage code generation built from real-world Python repositories. Each instance pairs a natural-language usage instruction with grounded target APIs and a verified reference script, enabling evaluation of both API retrieval and repository-usage code generation. Building on RUCCE, we propose RUCACoder, a closed-loop multi-agent framework with a Retriever for hierarchical repository exploration, a Verifier for reranking and validation, and a Coder for feedback-driven script synthesis. Experiments across multiple backbone LLMs show that RUCACoder consistently outperforms strong retrieval and generation baselines.

Automatic code summarization aims to generate concise natural language descriptions (summary) for source code, which can free software developers from the heavy burden of manual commenting and software maintenance. Existing methods focus on learning a direct mapping from pure code to summaries, overlooking the significant heterogeneity gap between code and summary. Moreover, existing methods lack a human-like re-check process to evaluate whether the generated summaries match well with the code. To address these two limitations, we introduce RBCoSum, a novel framework that incorporates the generated Chain Of Comments (COC) as auxiliary intermediate information for the model to bridge the gap between code and summaries. Also, we propose a rethinking process where a learned ranker trained on our constructed ranking dataset scores the extent of matching between the generated summary and the code, selecting the highest-scoring summary to achieve a re-check process. We conduct extensive experiments to evaluate our approach and compare it with other automatic code summarization models as well as multiple code Large Language Models (LLMs). The experimental results show that RBCoSum is effective and outperforms baselines by a large margin. The human evaluation also proves the summaries generated with RBCoSum are more natural, informative, useful, and truthful.

The code summarization task aims to automatically generate natural language descriptions for code snippets. Recently, pre-trained code language models (CLMs) have demonstrated outstanding performance on code summarization. Additionally, researchers have shown that there is a strong correlation between code function names and summaries, and poorly defined function names lead to worse summaries generated by models. To mitigate this issue, in this paper, we propose CLCoSum, a curriculum learning-based code summarization method for CLMs that improves their performance in poorly named function scenarios. CLCoSum helps CLMs avoid overreliance on function names when they are poorly defined. First, CLCoSum employs data augmentation operators on function names to generate semantically equivalent poorly named codes, which are considered harder data and assist in reducing the model's reliance on unclear function names. Subsequently, CLCoSum uses a curriculum learning paradigm to allow the model to learn these harder codes in an organized way during finetuning. This approach enables CLMs to progress from easier to more difficult training data, similar to the human learning process. Extensive experiments on two existing datasets for Java and Python demonstrate that CLCoSum boosts the performance of various CLMs in code summarization. Specifically, the improvements in BLEU-4 score range from approximately 5% to$\mathbf{2 0. 8 \%}$. The fine-tuning speed of CLCoSum on the augmented dataset is also competitive. Our code and data are available at https://github.com/KuiH/CLCoSum.

Large language models (LLMs) have revolutionized the field of programming for developers by automatically generating code based on natural language intent (NL intent). In numerous cases, LLMs can produce correct programs after several trials. As a result, a major challenge for this task is to select the most appropriate program from the multiple samples (also called code ranking) generated by LLMs. Recent popular approaches for code ranking involve the ranker-based methods, in which we train a ranker to classify the error in code using execution results (correct or error types) of code as supervised signals select the best program. However, existing rankerbased code ranking approaches rely on classification labels, which are highly sensitive to label distribution and show weak generalization ability to other distributions. In this paper, we introduce SACL-CR to address this challenge, a novel structureaware contrastive learning framework for code ranking. This approach effectively addresses the generalization issues of existing ranker-based methods by integrating both code sequence and structural information. Encoders trained with this method can effectively identify errors in code, enhancing the model's ability to differentiate between correct and incorrect code. Our research demonstrates that SACL-CR significantly enhances the pass@k accuracy of several code generation models, including CodeLlama and DeepseekCoder, on the HumanEval and MBPP datasets. The open-source code will be released at https://github.com/Iced-Americano2001/SACL-CR.

As Code Language Models (CLMs) are increasingly used to automate multilingual code intelligence tasks, Full-Parameter Fine-Tuning (FPFT) of CLMs has become a widely adopted approach, which is both time-consuming and resource-intensive. Parameter-Efficient Fine-Tuning (PEFT) provides a more efficient alternative to FPFT. However, it struggles to capture common features shared across languages, leading to performance degradation. Recent studies have explored mixed-language training with PEFT to avoid the loss of common features. However, these methods can result in gradient conflicts due to the diverse language-specific features, causing suboptimal performance, particularly for low-resource languages. In this paper, we propose Mixture-of-Experts Multilingual Low-Rank Adaptation (MMLoRA) for multilingual code summarization. MMLoRA addresses gradient conflicts while preserving common features shared across languages by combining a universal expert with a set of specialized linguistic experts. Additionally, we introduce an expert loss function that maintains the diversity of specialized linguistic experts while balancing the learning progress. Experimental results indicate that MMLoRA achieves state-of-the-art performance in multilingual code summarization while maintaining efficient fine-tuning. The performance improvement is particularly significant in low-resource languages such as Ruby.

Grounded Multimodal Named Entity Recognition (GMNER) extends Multimodal Named Entity Recognition (MNER) by identifying named entities, their types, and corresponding image regions. Fine-grained MNER and Grounding (FMNERG) further refines entity categorization. However, existing methods struggle with scarce annotated data, particularly in low-resource scenarios, and often fail to generalize to unseen entities. While vision-language pre-training (VLP) leverages unlabeled image-caption pairs, it primarily learns generic visual-linguistic representations, overlooking fine-grained entity-region alignment crucial for entity-related tasks. To address these challenges, we propose a unified VLP framework for GMNER and FMNERG, introducing two task-specific pre-training objectives: Entity-to-Region Alignment (ETRA) for entity grounding and Region-to-Entity Alignment (RTEA) for entity reconstruction. These tasks jointly optimize fine-grained entity-region alignment. To compensate for the lack of fine-grained multimodal pre-training data, we develop an automatic labeling method that distills entity-oriented knowledge from large-scale unlabeled image-text pairs, enhancing generalization to unseen entities. Extensive experiments on GMNER and FMNERG benchmarks demonstrate that our framework outperforms existing low-resource learning approaches and achieves competitive performance in full-supervision, underscoring its effectiveness across diverse data conditions.

Bug localization aims to locate the specific software artifacts responsible for triggering bugs as reported in bug reports, assisting software developers in conserving substantial resources and time during the debugging process. In recent years, there has been a shift toward changeset-based bug localization, with current approaches increasingly utilizing deep learning models that excel at understanding semantics to capture semantic relationships, to locate bug-inducing changesets. These models have already achieved notable improvements in performance. However, these approaches still have two limitations: (1) Insufficient interaction between bug reports and changesets. The feature interaction in current approaches is restricted to the final semantic similarity calculation, which may hinder overall model performance. (2) Under-exploration of rich code structural information. Changesets and bug reports are typically treated as distinct modalities-code and text, respectively, however existing approaches primarily mine code features in text form, neglecting the rich structural information in the changesets. To address these limitations, we propose CSE-WSS, a framework to improve feature interaction and representation learning for changeset-based bug localization. Specifically, we first introduce a new semantic similarity algorithm, Weighted Semantic Similarity (WSS), to conduct a comprehensive and fine-grained token-level interaction, which can aid in obtaining a more precise relevance score. Then we design a novel contrastive training technique, Code Structure Enhancement Contrastive Learning (CSE), to incorporate the rich structural information within changesets, which can help model understand the code semantics. Furthermore, we evaluate the performance of CSE- WSS across six widely used open-source projects, experimental results have demonstrated that CSE- WSS achieves significant improvements compared to four baseline models. Moreover, ablation and generalization studies of CSE-WSS further highlight its effectiveness in the task.

In the evolving field of Natural Language Processing (NLP), understanding the temporal context of text is increasingly critical for applications requiring advanced temporal reasoning. Traditional pre-trained language models like BERT, which rely on synchronic document collections such as BookCorpus and Wikipedia, often fall short in effectively capturing and leveraging temporal information. To address this limitation, we introduce BiTimeBERT 2.0, a novel time-aware language model pre-trained on a temporal news article collection. BiTimeBERT 2.0 incorporates temporal information through three innovative pre-training objectives: Extended Time-Aware Masked Language Modeling (ETAMLM), Document Dating (DD), and Time-Sensitive Entity Replacement (TSER). Each objective is specifically designed to target a distinct dimension of temporal information: ETAMLM enhances the model’s understanding of temporal contexts and relations, DD integrates document timestamps as explicit chronological markers, and TSER focuses on the temporal dynamics of “Person” entities. Moreover, our refined corpus preprocessing strategy reduces training time by nearly 53%, making BiTimeBERT 2.0 significantly more efficient while maintaining high performance. Experimental results show that BiTimeBERT 2.0 achieves substantial improvements across a broad range of time-related tasks and excels on datasets spanning extensive temporal ranges. These findings underscore BiTimeBERT 2.0’s potential as a powerful tool for advancing temporal reasoning in NLP. 1

Defect detection is a pivotal aspect ensuring product quality and production efficiency in industrial manufacturing. Existing studies on defect detection predominantly focus on locating defects through bounding boxes and classifying defect types. However, their methods can only provide limited information and fail to meet the requirements for further processing after detecting defects. To this end, we propose a novel task called defect detection report generation, which aims to provide more comprehensive and informative insights into detected defects in the form of text reports. For this task, we propose some new datasets, which contain 16 different materials and each defect contains a detailed report of human constructs. In addition, we propose a knowledge-aware report generation model as a baseline for future research, which aims to incorporate additional knowledge to generate detailed analysis and subsequent processing related to defect in images. By constructing defect report datasets and proposing corresponding baselines, we chart new directions for future research and practical applications of this task.

Code generation aims to generate code in a general-purpose programming language, such as C++, based on natural language intents. Existing efforts primarily focus on relatively simple programming problems and fail to evaluate the thought process involved in complex programming scenarios. In this paper, we introduce “steps-guided code generation,” a task that assesses the quality of both thought steps and code implementation to evaluate the overall management of handling a complex programming problem. To support this task, we construct CodeStepsEval, a real-world scenario dataset of complex programming problems in the C++ programming language with varying levels of difficulty. Comprehensive experiments on this dataset demonstrate the importance of high-quality steps in enhancing code generation performance and the challenges faced by the code LLMs in this task.

Generating coherent and credible explanations remains a significant challenge in the field of AI. In recent years, researchers have delved into the utilization of entailment trees to depict explanations, which exhibit a reasoning process of how a hypothesis is deduced from the supporting facts. However, existing models often overlook the importance of generating intermediate conclusions with logical consistency from the given facts, leading to inaccurate conclusions and undermining the overall credibility of entailment trees. To address this limitation, we propose the logical pattern memory pre-trained model (LMPM). LMPM incorporates an external memory structure to learn and store the latent representations of logical patterns, which aids in generating logically consistent conclusions. Furthermore, to mitigate the influence of logically irrelevant domain knowledge in the Wikipedia-based data, we introduce an entity abstraction approach to construct the dataset for pre-training LMPM. The experimental results highlight the effectiveness of our approach in improving the quality of entailment tree generation. By leveraging logical entailment patterns, our model produces more coherent and reasonable conclusions that closely align with the underlying premises.

With the increasing popularity of online social applications, stickers have become common in online chats. Teaching a model to select the appropriate sticker from a set of candidate stickers based on dialogue context is important for optimizing the user experience. Existing methods have proposed leveraging emotional information to facilitate the selection of appropriate stickers. However, considering the frequent co-occurrence among sticker images, words with emotional preference in the dialogue and emotion labels, these methods tend to over-rely on such dataset bias, inducing spurious correlations during training. As a result, these methods may select inappropriate stickers that do not match users' intended expression. In this paper, we introduce a causal graph to explicitly identify the spurious correlations in the sticker selection task. Building upon the analysis, we propose a Causal Knowledge-Enhanced Sticker Selection model to mitigate spurious correlations. Specifically, we design a knowledge-enhanced emotional utterance extractor to identify emotional information within dialogues. Then an interventional visual feature extractor is employed to obtain unbiased visual features, aligning them with the emotional utterances representation. Finally, a standard transformer encoder fuses the multimodal information for emotion recognition and sticker selection. Extensive experiments on the MOD dataset show that our CKS model significantly outperforms the baseline models.

Visual question answering with natural language explanation (VQA-NLE) is a challenging task that requires models to not only generate accurate answers but also to provide explanations that justify the relevant decision-making processes. This task is accomplished by generating natural language sentences based on the given question-image pair. However, existing methods often struggle to ensure consistency between the answers and explanations due to their disregard of the crucial interactions between these factors. Moreover, existing methods overlook the potential benefits of incorporating additional knowledge, which hinders their ability to effectively bridge the semantic gap between questions and images, leading to less accurate explanations. In this paper, we present a novel approach denoted the knowledge-based iterative consensus VQA-NLE (KICNLE) model to address these limitations. To maintain consistency, our model incorporates an iterative consensus generator that adopts a multi-iteration generative method, enabling multiple iterations of the answer and explanation in each generation. In each iteration, the current answer is utilized to generate an explanation, which in turn guides the generation of a new answer. Additionally, a knowledge retrieval module is introduced to provide potentially valid candidate knowledge, guide the generation process, effectively bridge the gap between questions and images, and enable the production of high-quality answer-explanation pairs. Extensive experiments conducted on three different datasets demonstrate the superiority of our proposed KICNLE model over competing state-of-the-art approaches. Our code is available at https://github.com/Gary-code/KICNLE.

In recent years, multimodal topic models have gained significant attention in various tasks involving short texts. Despite their impressive results, most models rely on bag-of-words assumptions for each modality, neglecting the intrinsic word relations in the textual modality and the underlying object relations in the visual modality. To address this limitation, we propose a novel approach that represents each document modality as a graph, harnessing the word relations and the visual object relations to guide the topic extraction process. Our approach is grounded in the insight that, in the textual modality, words with specific relations, such as co-occurrence relations, semantic relations and syntactic relations, are more likely to be assigned to the same topic. Similarly, in the visual modality, the relations between objects, such as spatial relations and contextual relations, can also provide valuable information for topic extraction. By leveraging graph-based representations, our model captures the inherent associations between words and visual objects, resulting in the generation of more coherent and interpretable topics. To infer the model's parameters, we develop an effective algorithm that integrates neural variational inference and contrastive learning. The experimental results on three datasets verify the effectiveness of our proposed model in terms of topic coherence, topic diversity and mean average precision, confirming that incorporating word relations and object relations through graph-based representations significantly enhances the quality of the extracted topics.

Time is an important aspect of documents and is used in a range of NLP and IR tasks. In this work, we investigate methods for incorporating temporal information during pre-training to further improve the performance on time-related tasks. Compared with common pre-trained language models like BERT which utilize synchronic document collections (e.g., BookCorpus and Wikipedia) as the training corpora, we use long-span temporal news article collection for building word representations. We introduce BiTimeBERT, a novel language representation model trained on a temporal collection of news articles via two new pre-training tasks, which harnesses two distinct temporal signals to construct time-aware language representations. The experimental results show that BiTimeBERT consistently outperforms BERT and other existing pre-trained models with substantial gains on different downstream NLP tasks and applications for which time is of importance (e.g., the accuracy improvement over BERT is 155% on the event time estimation task).

In the last few years, open-domain question answering (ODQA) has advanced rapidly due to the development of deep learning techniques and the availability of large-scale QA datasets. However, the current datasets are essentially designed for synchronic document collections (e.g., Wikipedia). Temporal news collections such as long-term news archives spanning decades are rarely used in training the models despite they are quite valuable for our society. To foster the research in the field of ODQA on such historical collections, we present ArchivalQA, a large question answering dataset consisting of 532,444 question-answer pairs which is designed for temporal news QA. We divide our dataset into four subparts based on the question difficulty levels and the containment of temporal expressions, which we believe are useful for training and testing ODQA systems characterized by different strengths and abilities. The novel QA dataset-constructing framework that we introduce can be also applied to generate high-quality, non-ambiguous questions over other types of temporal document collections.

Real world events are quite often mentioned in texts. Estimating the occurrence time of event mentions has many applications in IR, QA, general document understanding and downstream NLP tasks. In this paper we propose an approach to temporal profiling of event mentions in text. Our method utilizes a news article archival collection for collecting temporal as well as textual information containing contemporary and retrospective event references. As we demonstrate in our experiments, the recent method which relies on secondary data sources like Wikipedia is insufficient to correctly estimate the event time, especially, for minor or less well-known events that happened in the past. Our method then harnesses news article archives to effectively infer the occurrence time of past events, and is able to estimate the time at different temporal granularities (e.g., day, week, month, or year). As evidenced through extensive experiments, the proposed model outperforms the existing methods by a large margin at all granularities. We also demonstrate that our approach helps to answer arbitrary questions about past events, when incorporated into a QA framework operating over news article archives.