Abstract
Understanding the internal representations learned by machine learning models is critical for data auditing and model interpretability. We present RepMatch, a novel framework that quantifies the similarity between data subsets by comparing the "knowledge" encoded in models trained on them. By leveraging Low-Rank Adaptation (LoRA) to capture model updates and Grassmann similarity to measure subspace alignment, RepMatch provides a robust metric for identifying representative training instances and uncovering hidden dataset biases. Our experiments across diverse NLP tasks demonstrate that RepMatch-selected subsets significantly outperform random baselines, achieving up to 81.03% accuracy on SST-2 with only 100 instances.
Introduction
In modern machine learning, the quality and composition of training data are as important as model architecture. While existing techniques can rank individual instances by difficulty or noise, they often fail to provide a holistic view of how different data subsets—or even entirely different datasets—influence the model's internal representation space. Traditional methods, such as comparing [CLS] token embeddings, often capture surface-level distributional similarities rather than the actual learning dynamics.
RepMatch addresses this gap by asking: "Does the model learn the same concepts from Dataset A as it does from Dataset B?" By focusing on the changes in the model's representation space during fine-tuning, RepMatch identifies which examples best represent the core knowledge of a dataset.
Methodology
The RepMatch framework consists of three primary components designed for computational efficiency and robustness:
- LoRA as a Diagnostic Tool: Rather than comparing full weight matrices, RepMatch utilizes Low-Rank Adaptation (LoRA). When a model is fine-tuned on a specific data subset, LoRA captures the essential updates ($\Delta W$) into low-rank matrices. This acts as a "fingerprint" of the learning process for that specific data.
- Grassmann Similarity: To compare two models trained on different data, we measure the alignment of their respective LoRA update subspaces using Grassmann similarity. High similarity indicates that the two subsets teach the model similar features.
- RepMatch Score: The final score is an aggregation of similarity metrics across all model layers, providing a unified measure of representational similarity between data subsets.
Experimental Results
Identifying Representative Subsets
A primary application of RepMatch is the identification of highly informative data subsets. As shown in the scaling analysis, models trained on instances selected by RepMatch consistently outperform those trained on random subsets across various sample sizes.
Specifically, on the SST-2 dataset, a model fine-tuned on just 100 RepMatch-selected instances achieved an accuracy of 81.03%, a substantial improvement over the 69.53% achieved by a random baseline of the same size. These results are consistent across multiple datasets, as detailed in the performance table below.
Detecting Dataset Biases and Shortcuts
RepMatch proved highly effective at auditing dataset quality. In the context of Natural Language Inference (NLI), the framework identified that instances with high word-overlap between premise and hypothesis teach the model the same "heuristics" (shortcuts) found in the HANS challenge set. This suggests that RepMatch can be used to proactively identify and mitigate dataset biases that lead to poor generalization.
Cross-Dataset Sensitivity
The framework also demonstrates high sensitivity to task-level similarities. For instance, RepMatch correctly assigned a high similarity score (~0.45) between SST-2 and SST-5 (both sentiment tasks), while assigning a near-zero score (~0.01) between SST-2 and MNLI, despite both being classification tasks.
Discussion and Limitations
While RepMatch offers a powerful lens into model learning, some challenges remain. The computational cost of performing multiple small fine-tuning runs to calculate scores for individual instances can be significant for massive datasets. Additionally, the current implementation treats instances in isolation, potentially overlooking complex interactions within training batches. Future work will explore more sophisticated layer-wise aggregation strategies and extend the framework to computer vision and multi-modal domains.
Summary updated