Data attribution methods quantify the influence of training data on model outputs and are becoming increasingly relevant for a wide range of LLM research and applications, including dataset curation, model interpretability, data valuation. However, there remain critical gaps in systematic LLM-centric evaluation of data attribution methods. To this end, we introduce DATE-LM (Data Attribution Evaluation in Language Models), a unified benchmark for evaluating data attribution methods through real-world LLM applications. DATE-LM measures attribution quality through three key tasks — training data selection, toxicity/bias filtering, and factual attribution. Our benchmark is designed for ease of use, enabling researchers to configure and run large-scale evaluations across diverse tasks and LLM architectures. Furthermore, we use DATE-LM to conduct a large-scale evaluation of existing data attribution methods. Our findings show that no single method dominates across all tasks, data attribution methods have trade-offs with simpler baselines, and method performance is sensitive to task-specific evaluation design. Finally, we release a public leaderboard for quick comparison of methods and to facilitate community engagement. We hope DATE-LM serves as a foundation for future data attribution research in LLMs.

Debugging is a crucial skill for programmers, yet it can be challenging for novices to learn. The introduction of large language models (LLMs) has opened up new possibilities for providing personalized debugging support to students. However, concerns have been raised about potential student over-reliance on LLM-based tools. This mixed-methods study investigates how a pedagogically-designed LLM-based chatbot supports students’ debugging efforts in an introductory programming course. We conducted interviews and debugging think-aloud tasks with 20 students at three points throughout the semester. We specifically focused on characterizing when students initiate help from the chatbot during debugging, how they engage with the chatbot’s responses, and how they describe their learning experiences with the chatbot. By analyzing data from the debugging tasks, we identified varying help-seeking behaviors and levels of engagement with the chatbot’s responses, depending on students’ familiarity with the suggested strategies. Interviews revealed that students appreciated the content and experiential knowledge provided by the chatbot, but did not view it as a primary source for learning debugging strategies. Additionally, students self-identified certain chatbot usage behaviors as negative, “non-ideal” engagement and others as positive, “learning-oriented” usage. Based on our findings, we discuss pedagogical implications and future directions for designing pedagogical chatbots to support debugging.