RepMatch: Quantifying Cross-Instance Similarities in Representation Space

Mohammad Modarres, Sina Abbasi, Mohammad Taher PilehvarEMNLP 2024 · DOI

Abstract

Understanding the internal representations learned by machine learning models is critical for data auditing and model interpretability. We present RepMatch, a novel framework that quantifies the similarity between data subsets by comparing the "knowledge" encoded in models trained on them. By leveraging Low-Rank Adaptation (LoRA) to capture model updates and Grassmann similarity to measure subspace alignment, RepMatch provides a robust metric for identifying representative training instances and uncovering hidden dataset biases. Our experiments across diverse NLP tasks demonstrate that RepMatch-selected subsets significantly outperform random baselines, achieving up to 81.03% accuracy on SST-2 with only 100 instances.

Introduction

In modern machine learning, the quality and composition of training data are as important as model architecture. While existing techniques can rank individual instances by difficulty or noise, they often fail to provide a holistic view of how different data subsets—or even entirely different datasets—influence the model's internal representation space. Traditional methods, such as comparing [CLS] token embeddings, often capture surface-level distributional similarities rather than the actual learning dynamics.

RepMatch addresses this gap by asking: "Does the model learn the same concepts from Dataset A as it does from Dataset B?" By focusing on the changes in the model's representation space during fine-tuning, RepMatch identifies which examples best represent the core knowledge of a dataset.

Methodology

The RepMatch framework consists of three primary components designed for computational efficiency and robustness:

  1. LoRA as a Diagnostic Tool: Rather than comparing full weight matrices, RepMatch utilizes Low-Rank Adaptation (LoRA). When a model is fine-tuned on a specific data subset, LoRA captures the essential updates ($\Delta W$) into low-rank matrices. This acts as a "fingerprint" of the learning process for that specific data.
  2. Grassmann Similarity: To compare two models trained on different data, we measure the alignment of their respective LoRA update subspaces using Grassmann similarity. High similarity indicates that the two subsets teach the model similar features.
  3. RepMatch Score: The final score is an aggregation of similarity metrics across all model layers, providing a unified measure of representational similarity between data subsets.

Experimental Results

Identifying Representative Subsets

A primary application of RepMatch is the identification of highly informative data subsets. As shown in the scaling analysis, models trained on instances selected by RepMatch consistently outperform those trained on random subsets across various sample sizes.

Accuracy comparison between RepMatch-selected instances and a random baseline across varying sample sizes.

Specifically, on the SST-2 dataset, a model fine-tuned on just 100 RepMatch-selected instances achieved an accuracy of 81.03%, a substantial improvement over the 69.53% achieved by a random baseline of the same size. These results are consistent across multiple datasets, as detailed in the performance table below.

Performance comparison of RepMatch vs. Random selection across multiple NLP datasets.

Detecting Dataset Biases and Shortcuts

RepMatch proved highly effective at auditing dataset quality. In the context of Natural Language Inference (NLI), the framework identified that instances with high word-overlap between premise and hypothesis teach the model the same "heuristics" (shortcuts) found in the HANS challenge set. This suggests that RepMatch can be used to proactively identify and mitigate dataset biases that lead to poor generalization.

Cross-Dataset Sensitivity

The framework also demonstrates high sensitivity to task-level similarities. For instance, RepMatch correctly assigned a high similarity score (~0.45) between SST-2 and SST-5 (both sentiment tasks), while assigning a near-zero score (~0.01) between SST-2 and MNLI, despite both being classification tasks.

Discussion and Limitations

While RepMatch offers a powerful lens into model learning, some challenges remain. The computational cost of performing multiple small fine-tuning runs to calculate scores for individual instances can be significant for massive datasets. Additionally, the current implementation treats instances in isolation, potentially overlooking complex interactions within training batches. Future work will explore more sophisticated layer-wise aggregation strategies and extend the framework to computer vision and multi-modal domains.

Summary updated

BibTeX
@inproceedings{modarres2024repmatch,
  author = {Mohammad Modarres and Sina Abbasi and Mohammad Taher Pilehvar},
  title = {{RepMatch: Quantifying Cross-Instance Similarities in Representation Space}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.825},
}
Figure