Machine unlearning in neural information retrieval (IR) systems requires removing specific data while maintaining model performance. Applying existing machine unlearning methods to IR may compromise retrieval effectiveness or inadvertently expose unlearning actions due to the removal of particular items from the retrieved results presented to users. We formalise corrective unranking , which extends machine unlearning in the (neural) IR context by integrating substitute documents to preserve ranking integrity, and propose a novel teacher–student framework, Corrective unRanking Distillation (CuRD), for this task. CuRD (1) facilitates forgetting by adjusting the (trained) neural IR model such that its output relevance scores of to-be-forgotten samples mimic those of low-ranking, non-retrievable samples; (2) enables correction by fine-tuning the relevance scores for the substitute samples to match those of corresponding to-be-forgotten samples closely; (3) seeks to preserve performance on samples that are not targeted for forgetting. We evaluate CuRD on four neural IR models (BERTcat, BERTdot, ColBERT, PARADE) using MS MARCO and TREC CAR datasets. Experiments with forget set sizes from 1% to 20% of the training dataset demonstrate that CuRD outperforms seven state-of-the-art baselines in terms of forgetting and correction while maintaining model retention and generalisation capabilities.

Abstract Integrating diverse cues from metadata to make sense of retrieved data during relevance evaluation is a crucial yet challenging task for data searchers. However, this integrative task remains underexplored, impeding the development of effective strategies to address metadata's shortcomings in supporting this task. To address this issue, this study proposes the “Integrative Use of Metadata for Data Sense‐Making” (IUM‐DSM) model. This model provides an initial framework for understanding the integrative tasks performed by data searchers, focusing on their integration patterns and associated challenges. Experimental data were analyzed using an interpretable deep learning‐based prediction approach to validate this model. The findings offer preliminary support for the model, revealing that data searchers engage in integrative tasks to utilize metadata effectively for data sense‐making during relevance evaluation. They construct coherent mental representations of retrieved data by integrating systematic and heuristic cues from metadata through two distinct patterns: within‐category integration and across‐category integration. This study identifies key challenges: within‐category integration entails comparing, classifying, and connecting systematic or heuristic cues, while across‐category integration necessitates considerable effort to integrate cues from both categories. To support these integrative tasks, this study proposes strategies for mitigating these challenges by optimizing metadata layouts and developing intelligent data retrieval systems.