Shihan Dou, Jiayi Chen, Chenhao Huang, Feng Chen, Wei Chengzhi, Huiyuan Zheng, Shichun Liu, Yan Liu, Chenxiao Liu, Chao Xin, Lin Yan, Zongzhang Zhang, Tao Gui, Qi Zhang, Xuanjing Huang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

We introduce EvaLearn, a pioneering benchmark designed to evaluate large language models (LLMs) on their learning capability and efficiency in challenging tasks, a critical, yet underexplored aspect of model potential. EvaLearn contains 648 challenging problems across six task types, grouped into 182 sequences, each sequence dedicated to one task type. Diverging from most existing benchmarks that evaluate models in parallel, EvaLearn requires models to solve problems sequentially, allowing them to leverage the experience gained from previous solutions. EvaLearn provides five comprehensive automated metrics to evaluate models and quantify their learning capability and efficiency. We extensively benchmark nine frontier models and observe varied performance profiles: some models, such as Claude-3.7-sonnet, start with moderate initial performance but exhibit strong learning ability, while some models struggle to benefit from experience and may even show negative transfer. Moreover, we investigate model performance under two learning settings and find that instance-level rubrics and teacher-model feedback further facilitate model learning. Importantly, we observe that current LLMs with stronger static abilities do not show a clear advantage in learning capability across all tasks, highlighting that EvaLearn evaluates a new dimension of model performance. We hope EvaLearn provides a novel evaluation perspective for assessing LLM potential and understanding the gap between models and human capabilities, promoting the development of deeper and more dynamic evaluation approaches. All datasets, the automatic evaluation framework, and the results studied in this paper are available in the supplementary materials.

Reinforcement Learning from Human Feedback (RLHF) is essential for aligning large language models (LLMs) with human preferences and values. While recent research has primarily focused on algorithmic advancements—such as reducing computational overhead or strengthening reward models to mitigate reward hacking—the critical role of prompt-data construction and its scalability has received comparatively less attention. In this paper, we address this gap by systematically exploring data-driven bottlenecks that currently hinder RLHF performance scaling, focusing specifically on the challenges posed by reward hacking and decreasing response diversity. To mitigate reward hacking, we introduce a hybrid reward system combining reasoning task verifiers (RTV) and a generative reward model (GenRM). This approach not only exhibits enhanced resistance to reward hacking, but also enables accurate assessment of responses against clearly defined ground-truth solutions. Additionally, in order to ensure response diversity and enhance learning effectiveness, we propose a novel prompt-selection method named \textbf{Pre-PPO}, explicitly identifying training prompts that are inherently challenging and thus less prone to reward hacking. Furthermore, we find that \textbf{prioritizing mathematical and coding tasks during the early phases of RLHF training} significantly boosts performance, given that these tasks naturally encode fine-grained response distinctions and possess clearly defined ground truths. Through comprehensive experiments conducted across two model sizes, we validate the effectiveness and scalability of our proposed methods. Results show that RTV exhibits the strongest resistance to reward hacking, followed by GenRM with ground truth, and finally GenRM relying on SFT Best-of-N responses. Moreover, our proposed strategies enable the model to rapidly capture subtle task-specific distinctions, leading to substantial improvements in overall RLHF performance. This work underscores the importance of careful data construction and provides practical methodologies to overcome critical performance barriers in RLHF.

Semantic code repair refers to automatically fix bugs where the actual program code compiles and executes successfully but fail to generate the output the programmer intends. This problem has not been solved very well so far. In this paper, we present a semantic code repair service using a deep attentional sequence-to-sequence model to predict related information about bugs and generate potential fixes without running the program actually. We evaluate the real performance of the semantic code repair service, and verify the feasibility and effectiveness of the service.

MOOCs are popular for online education because of its convenience and excellent educational resources. At present, MOOCs just provide very limited online practical environments such as online tests, quizzes and online judge etc., but they have not provided students with some dynamic and operable online practical environments, such as interactive education tools etc. Online education tools can satisfy the needs of online practicing, mainly because of its abundance, interactivity and convenience, but MOOCs have not made full use of these online tools yet. The paper proposes an approach to using existing online education tools to support practical education on MOOCs by two enhancements: educational resources development and real-time collaborative learning. The key to the approach is operation reuse, which involves recording, optimizing and replaying user operations on existing online educations tools. Based on the approach, we develop an education platform called Smart Web Tutor. Case study is performed to analyze user experience, and the results show that our approach contributes to students' learning, especially for real-time collaborative learning. Also, some suggestions from students are inspiring to further increase the utility and user experience of our approach and education platform.

Web-based IDEs are more and more popular because developers can create or modify software artifacts in the browser without need to install any local development tool and spend valuable development time on system setup and maintenance. For those development tasks using a Web-based IDE, such as configuring programming context and batch test etc., some are frequent and repetitive because they are similar from project to project. Automating the repetitive tasks on Web-based IDEs, regardless of their complexity, would reduce the amount of work that developers must perform to complete the tasks, which would improve the development efficiency of Web applications. In this paper, we put forward a user-friendly approach to automating repetitive tasks on existing Web-based IDEs. The key to the approach is to extend the basic Web-based capture-replay technique with editable and reusable features, which are necessary for automation because some operations are redundant, as well as developers should recognize and define repetitive tasks. Moreover, we develop a supporting tool for the approach. In the case study, we introduce how the approach is used to support automating repetitive tasks on Web-based IDEs. Case studies verify that the approach can improve the development efficiency very well.