Large language models (LLMs) frequently prioritize parametric world knowledge over provided context -- a failure mode that is particularly catastrophic in enterprise or counterfactual settings where local facts contradict web-scale training data. While modern reasoning models improve general response quality, they fail to resolve these underlying prior knowledge biases even when generating a high volume of costly thinking tokens. We propose a context-editing framework that addresses this by performing selective abstraction over entities that appear in both the context and the question. Our approach replaces these overlapping entities with typed, indexed identifiers (e.g., Paris with City łangle1\rangle), suppressing spurious lexical associations while preserving the relational structure required for grounded reasoning. This mechanism is decoupled into an offline preprocessing stage and a lightweight inference-time substitution, requiring no model retraining. Experiments across counterfactual and multi-hop knowledge-conflict benchmarks demonstrate consistent accuracy gains across many model families, open- and closed-sourced, especially for small to medium sized LLMs (0.5B to 18B). Notably, our framework yields up to a 25% improvement over state-of-the-art instruct models and 13% over reasoning models, establishing symbolic abstraction as a highly cost-efficient solution for ensuring context fidelity in LLMs.

Retrieval-Augmented Generation (RAG) is often used with Large Language Models (LLMs) to infuse domain knowledge or user-specific information. In RAG, given a user query, a retriever extracts chunks of relevant text from a knowledge base. These chunks are sent to an LLM as part of the input prompt. Typically, any given chunk is repeatedly retrieved across user questions. However, currently, for every question, attention layers in LLMs fully compute the Keys and Values (KVs) repeatedly for the input chunks, as state-of-the-art methods cannot reuse KV-caches when chunks appear at arbitrary locations or with arbitrary contexts. Naive reuse leads to output quality degradation. This leads to potentially redundant computations on expensive GPUs and increases latency. In this work, we propose Cache-Craft , a system for managing and reusing precomputed KVs corresponding to the text chunks (which we call chunk-caches ) in RAG-based systems. We present how to identify chunk-caches that are reusable, how to efficiently perform a small fraction of recomputation to fix the cache and maintain output quality, and how to efficiently store and evict chunk-caches in the hardware for maximizing reuse while masking any overheads. With real production workloads as well as synthetic datasets, we show that Cache-Craft reduces redundant computation by 51% over SOTA prefix-caching and 75% over full recomputation. Additionally, with continuous batching on a real production workload, we get a 1.6× speed up in throughput for both the LLama-3-8B and 70B models and a 2.1× and 2× reduction in end-to-end response latency respectively, compared to prefix-caching, while maintaining generation quality.