Large language models (LLMs) frequently prioritize parametric world knowledge over provided context -- a failure mode that is particularly catastrophic in enterprise or counterfactual settings where local facts contradict web-scale training data. While modern reasoning models improve general response quality, they fail to resolve these underlying prior knowledge biases even when generating a high volume of costly thinking tokens. We propose a context-editing framework that addresses this by performing selective abstraction over entities that appear in both the context and the question. Our approach replaces these overlapping entities with typed, indexed identifiers (e.g., Paris with City łangle1\rangle), suppressing spurious lexical associations while preserving the relational structure required for grounded reasoning. This mechanism is decoupled into an offline preprocessing stage and a lightweight inference-time substitution, requiring no model retraining. Experiments across counterfactual and multi-hop knowledge-conflict benchmarks demonstrate consistent accuracy gains across many model families, open- and closed-sourced, especially for small to medium sized LLMs (0.5B to 18B). Notably, our framework yields up to a 25% improvement over state-of-the-art instruct models and 13% over reasoning models, establishing symbolic abstraction as a highly cost-efficient solution for ensuring context fidelity in LLMs.

Retrieval-Augmented Generation (RAG) is often used with Large Language Models (LLMs) to infuse domain knowledge or user-specific information. In RAG, given a user query, a retriever extracts chunks of relevant text from a knowledge base. These chunks are sent to an LLM as part of the input prompt. Typically, any given chunk is repeatedly retrieved across user questions. However, currently, for every question, attention layers in LLMs fully compute the Keys and Values (KVs) repeatedly for the input chunks, as state-of-the-art methods cannot reuse KV-caches when chunks appear at arbitrary locations or with arbitrary contexts. Naive reuse leads to output quality degradation. This leads to potentially redundant computations on expensive GPUs and increases latency. In this work, we propose Cache-Craft , a system for managing and reusing precomputed KVs corresponding to the text chunks (which we call chunk-caches ) in RAG-based systems. We present how to identify chunk-caches that are reusable, how to efficiently perform a small fraction of recomputation to fix the cache and maintain output quality, and how to efficiently store and evict chunk-caches in the hardware for maximizing reuse while masking any overheads. With real production workloads as well as synthetic datasets, we show that Cache-Craft reduces redundant computation by 51% over SOTA prefix-caching and 75% over full recomputation. Additionally, with continuous batching on a real production workload, we get a 1.6× speed up in throughput for both the LLama-3-8B and 70B models and a 2.1× and 2× reduction in end-to-end response latency respectively, compared to prefix-caching, while maintaining generation quality.

Learning to Rank (LTR) technique is ubiquitous in Information Retrieval systems, especially in search ranking applications. The relevance labels used to train ranking models are often noisy measurements of human behavior, such as product ratings in product searches. This results in non-unique ground truth rankings and ambiguity. To address this, Multi-Label LTR (MLLTR) is used to train models using multiple relevance criteria, capturing conflicting but important goals, such as product quality and purchase likelihood for improved revenue in product searches. This research leverages Multi-Objective Optimization (MOO) in MLLTR and employs modern MOO algorithms to solve the problem. A general framework is proposed to combine label information to characterize trade-offs among goals, and allows for the use of gradient-based MOO algorithms. We test the proposed framework on four publicly available LTR datasets and one E-commerce dataset to show its efficacy.

In Learning-to-Rank (LTR) problems, the task of delivering relevant search results and allocating fair exposure to items of a protected group can conflict. Previous works in Fair LTR have attempted to resolve this by combining the objectives of relevant ranking and fair ranking into a single linear combination, but this approach is limited by the nonconvexity of the objective functions and can result in suboptimal relevance in ranking outputs. To address this, we propose a solution using Multi-Objective Optimization (MOO) algorithms. We extend these algorithms to querywise MOO to reduce the exposure disparity, not only on average but also at the query level. Interestingly, for moderate fairness requirements, it improves the relevance of ranking instead of deteriorating. We attribute this improvement to the benefits of multi-task learning and study the effect of fair ranking on the relevant ranking task. Moreover, we significantly improve the computational efficiency compared to previous methods by using the Gumbel max trick to sample the Plackett-Luce distribution. We evaluate our proposed methods on three real-world datasets and show their improvement in relevance ranking over state-of-the-art solutions.

Multi-Task Learning (MTL) is a well established paradigm for jointly learning models for multiple correlated tasks. Often the tasks conflict, requiring trade-offs between them during optimization. In such cases, multi-objective optimization based MTL methods can be used to find one or more Pareto optimal solutions. A common requirement in MTL applications, that cannot be addressed by these methods, is to find a solution satisfying userspecified preferences with respect to task-specific losses. We advance the state-of-the-art by developing the first gradient-based multi-objective MTL algorithm to solve this problem. Our unique approach combines multiple gradient descent with carefully controlled ascent to traverse the Pareto front in a principled manner, which also makes it robust to initialization. The scalability of our algorithm enables its use in large-scale deep networks for MTL. Assuming only differentiability of the task-specific loss functions, we provide theoretical guarantees for convergence. Our experiments show that our algorithm outperforms the best competing methods on benchmark datasets.