Understanding the impact of scientific publications is crucial for identifying breakthroughs and guiding future research.Traditional metrics based on citation counts often miss the nuanced ways a paper contributes to its field.In this work, we propose a new task: generating nuanced, expressive, and time-aware impact summaries that capture both praise (confirmation citations) and critique (correction citations) through the evolution of fine-grained citation intents.We introduce an evaluation framework tailored to this task, showing moderate to strong human correlation on subjective metrics such as insightfulness.Expert feedback from professors reveals a strong interest in these summaries and suggests future improvements.Data and code are made available.1

A hallmark of human innovation is recombination-the creation of novel ideas by integrating elements from existing concepts and mechanisms.In this work, we introduce CHIMERA, the first large-scale Knowledge Base (KB) of recombination examples automatically mined from the scientific literature.CHIMERA enables empirical analysis of how scientists recombine concepts and draw inspiration from different areas, and enables training models that propose cross-disciplinary research directions.To construct this KB, we define a new information extraction task: identifying recombination instances in papers.We curate an expertannotated dataset and use it to fine-tune an LLM-based extraction model, which we apply to a broad corpus of AI papers.We also demonstrate generalization to a biological domain.We showcase the utility of CHIMERA through two applications.First, we analyze patterns of recombination across AI subfields.Second, we train a scientific hypothesis generation model using the KB, showing that it can propose directions that researchers rate as inspiring.

We introduce Debate Speech Evaluation as a novel and challenging benchmark for assessing LLM judges.Evaluating debate speeches requires a deep understanding of the speech at multiple levels, including argument strength and relevance, the coherence and organization of the speech, the appropriateness of its style and tone, and so on.This task involves a unique set of cognitive abilities that previously received limited attention in systematic LLM benchmarking.To explore such skills, we leverage a dataset of over 600 meticulously annotated debate speeches and present the first in-depth analysis of how state-of-the-art LLMs compare to human judges on this task.Our findings reveal a nuanced picture: while larger models can approximate individual human judgments in some respects, they differ substantially in their overall judgment behavior.We also investigate the ability of frontier LLMs to generate persuasive, opinionated speeches, showing that models may perform at a human level on this task.