Automated Fact-Checking has largely focused on verifying general knowledge against static corpora, overlooking high-stakes domains like law where truth is evolving and technically complex.We introduce CaseFacts, a benchmark for verifying colloquial legal claims against U.S. Supreme Court precedents.Unlike existing resources that map formal texts to formal texts, CaseFacts challenges systems to bridge the semantic gap between layperson assertions and technical jurisprudence while accounting for temporal validity.The dataset consists of 6,294 claims categorized as SUP-PORTED, REFUTED, or OVERRULED.We construct this benchmark using a multi-stage pipeline that leverages Large Language Models (LLMs) to synthesize claims from expert case summaries, employing a novel semantic similarity heuristic to efficiently identify and verify complex legal overrulings.Experiments with state-of-the-art LLMs reveal that the task remains challenging; notably, augmenting models with unrestricted web search degrades performance compared to closed-book baselines due to the retrieval of noisy, non-authoritative precedents.We release CaseFacts 1 to spur research into legal fact verification systems. Case Input (Marbury v. Madison)Facts of the case: William Marbury had been appointed Justice of the Peace in the District of Columbia, but his commission was not delivered... Question: Do the plaintiffs have a right to receive their commissions?Can they sue for their

Frame-semantic parsing is a critical task in natural language understanding, yet the ability of large language models (LLMs) to extract frame-semantic arguments remains underexplored.This paper presents a comprehensive evaluation of LLMs on frame-semantic argument identification, analyzing the impact of input representation formats, model architectures, and generalization to unseen and out-ofdomain samples.Our experiments, spanning models from 0.5B to 72B parameters, reveal that JSON-based representations significantly enhance performance, and while larger models generally perform better, smaller models can achieve competitive results through fine-tuning.We also introduce a novel approach to frame identification leveraging predicted frame elements, achieving state-of-the-art performance on ambiguous targets.Despite strong generalization capabilities, our analysis finds that LLMs still struggle with out-of-domain data.

We present novel advancements in framesemantic parsing, specifically focusing on target identification and frame identification.Our target identification model employs a novel prefix tree modification to enable robust support for multi-word lexical units, resulting in a coverage of 99.4% of the targets in the FrameNet 1.7 fulltext annotations.It utilizes a RoBERTabased filter to achieve an F1 score of 0.775, surpassing the previous state-of-the-art solution by +0.012.For frame identification, we introduce a modification to the standard multiplechoice classification paradigm by incorporating additional negative frames for targets with limited candidate frames, resulting in a +0.014 accuracy improvement over the frame-only model of FIDO, the previous state-of-the-art system, and +0.002 over its full system.Our approach significantly enhances performance on rare frames, exhibiting an improvement of +0.044 over FIDO's accuracy on frames with 5 or fewer samples, and on under-utilized frames, with an improvement of +0.139 on targets with a single candidate frame.Overall, our contributions address critical challenges and advance the state-of-the-art in frame-semantic parsing. Message CognizerColumbus only thought that he had discovered Jamaica

This article presents the latest developments to ClaimBuster’s claim-spotting model, which tackles the critical task of identifying check-worthy claims from large streams of information. We introduce the first adversarially regularized, transformer-based claim-spotting model, which achieves state-of-the-art results on several benchmark datasets. In addition to analyzing model performance metrics, we also quantitatively and qualitatively analyze the impact of ClaimBuster’s real-world deployment. Moreover, to help facilitate reproducibility and community engagement, we publicly release our codebase, dataset, data curation platform, API, Google Colab notebooks, and various ClaimBuster-based demo systems, at claimbuster.org .