Data pruning, selecting small but impactful subsets, offers a promising way to efficiently scale NLP model training. However, existing methods often involve many different design choices, which have not been systematically studied. This limits future developments. In this work, we decompose data pruning into two key components: the data representation and the selection algorithm, and we systematically analyze their influence on the selection of instances. Our theoretical and empirical results highlight the crucial role of representations: better representations, e.g., training gradients, generally lead to a better selection of instances, regardless of the chosen selection algorithm. Furthermore, different selection algorithms excel in different settings, and none consistently outperforms the others. Moreover, the selection algorithms do not always align with their intended objectives: for example, algorithms designed for the same objective can select drastically different instances, highlighting the need for careful evaluation.

Language models excel in various tasks by making complex decisions, yet understanding the rationale behind these decisions remains a challenge.This paper investigates data-centric interpretability in language models, focusing on the next-word prediction task.Using representer theorem, we identify two types of support samples-those that either promote or deter specific predictions.Our findings reveal that being a support sample is an intrinsic property, predictable even before training begins.Additionally, while non-support samples are less influential in direct predictions, they play a critical role in preventing overfitting and shaping generalization and representation learning.Notably, the importance of non-support samples increases in deeper layers, suggesting their significant role in intermediate representation formation.These insights shed light on the interplay between data and model decisions, offering a new dimension to understanding language model behavior and interpretability. 1 * Equal contribution. 1 Our source code is publicly available at https://github.com/liyuqian44/ On-Support-Samples-of-Next-Word-Prediction.

Despite the massive success of fine-tuning Pre-trained Language Models (PLMs), they remain susceptible to out-of-distribution input. Dataset cartography is a simple yet effective dual-model approach that improves the robustness of fine-tuned PLMs. It involves fine-tuning a model on the original training set (i.e. reference model), selecting a subset of important training instances based on the training dynamics, % of the reference model, and fine-tuning again only on these selected examples (i.e. main model). However, this approach requires fine-tuning the same model twice, which is computationally expensive for large PLMs. In this paper, we show that 1) training dynamics are highly transferable across model sizes and pre-training methods, and that 2) fine-tuning main models using these selected training instances achieves higher training efficiency than empirical risk minimization (ERM). Building on these observations, we propose a novel fine-tuning approach: Fine-Tuning by transFerring Training dynamics (FTFT). Compared with dataset cartography, FTFT uses more efficient reference models and aggressive early stopping. FTFT achieves robustness improvements over ERM while lowering the training cost by up to ~50%

Large language models readily memorize arbitrary training instances, such as label noise, yet they perform strikingly well on reasoning tasks.In this work, we investigate how language models memorize label noise, and why such memorization in many cases does not heavily affect generalizable reasoning capabilities.Using two controllable synthetic reasoning datasets with noisy labels, four-digit addition (FDA) and two-hop relational reasoning (THR), we discover a reliance of memorization on generalizable reasoning mechanisms: models continue to compute intermediate reasoning outputs even when retrieving memorized noisy labels, and intervening reasoning adversely affects memorization.We further show that memorization operates through distributed encoding, i.e., aggregating various inputs and intermediate results, rather than building a look-up mechanism from inputs to noisy labels.Moreover, our FDA case study reveals memorization occurs via outlier heuristics, where existing neuron activation patterns are slightly shifted to fit noisy labels.Together, our findings suggest that memorization of label noise in language models builds on, rather than overrides, the underlying reasoning mechanisms, shedding lights on the intriguing phenomenon of benign memorization.1

Implicit reasoning is the ability of a language model to solve multi-hop reasoning tasks in a single forward pass, without chain of thought.We investigate this capability using GPT2-style language models trained from scratch on controlled k-hop reasoning datasets (k = 2, 3, 4).We show that while such models can indeed learn implicit k-hop reasoning, the required training data grows exponentially in k, and the required number of transformer layers grows linearly in k.We offer a theoretical explanation for why this depth growth is necessary.We further find that the data requirement can be mitigated, but not eliminated, through curriculum learning.

Fine-tuning pre-trained language models on downstream tasks with varying random seeds has been shown to be unstable, especially on small datasets.Many previous studies have investigated this instability and proposed methods to mitigate it.However, most studies only used the standard deviation of performance scores (SD) as their measure, which is a narrow characterization of instability.In this paper, we analyze SD and six other measures quantifying instability at different levels of granularity.Moreover, we propose a systematic framework to evaluate the validity of these measures.Finally, we analyze the consistency and difference between different measures by reassessing existing instability mitigation methods.We hope our results will inform the development of better measurements of fine-tuning instability. 1

Knowledge base (KB) embeddings have been shown to contain gender biases (Fisher et al., 2020b).In this paper, we study two questions regarding these biases: how to quantify them, and how to trace their origins in KB? Specifically, first, we develop two novel bias measures respectively for a group of person entities and an individual person entity.Evidence of their validity is observed by comparison with real-world census data.Second, we use influence function to inspect the contribution of each triple in KB to the overall group bias.To exemplify the potential applications of our study, we also present two strategies (by adding and removing KB triples) to mitigate gender biases in KB embeddings.

Various measures have been proposed to quantify human-like social biases in word embeddings.However, bias scores based on these measures can suffer from measurement error.One indication of measurement quality is reliability, concerning the extent to which a measure produces consistent results.In this paper, we assess three types of reliability of word embedding gender bias measures, namely testretest reliability, inter-rater consistency and internal consistency.Specifically, we investigate the consistency of bias scores across different choices of random seeds, scoring rules and words.Furthermore, we analyse the effects of various factors on these measures' reliability scores.Our findings inform better design of word embedding gender bias measures.Moreover, we urge researchers to be more critical about the application of such measures.1

Personalised response generation enables generating human-like responses by means of assigning the generator a social identity.However, pragmatics theory suggests that human beings adjust the way of speaking based on not only who they are but also whom they are talking to.In other words, when modelling personalised dialogues, it might be favourable if we also take the listener's social identity into consideration.To validate this idea, we use gender as a typical example of a social variable to investigate how the listener's identity influences the language used in Chinese dialogues on social media.Also, we build personalised generators.The experiment results demonstrate that the listener's identity indeed matters in the language use of responses and that the response generator can capture such differences in language use.More interestingly, by additionally modelling the listener's identity, the personalised response generator performs better in its own identity.

Yupei Du, Yuanbin Wu, Man Lan. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.