Gradient Sketches for Training Data Attribution and Studying the Loss Landscape

Andrea Schioppa

Introduction

Understanding the behavior of Large Language Models (LLMs) often requires analyzing high-dimensional objects like gradients and Hessians. These analyses are crucial for Training Data Attribution (TDA)—identifying which training examples influenced a specific prediction—and for studying the loss landscape's geometry. However, the sheer size of LLM parameter spaces makes storing and manipulating these vectors computationally prohibitive.

The Challenge of High-Dimensional Analysis

Traditional methods for dimensionality reduction, such as the Fastfood Transform or dense random projections, face significant hurdles when scaled to billions of parameters:

  1. Memory Constraints: Dense projection matrices often exceed available RAM.
  2. Hardware Underutilization: Many existing algorithms rely on lookup tables and permutations that do not align well with the parallel processing architectures of modern GPUs and TPUs.
  3. Accuracy Degradation: A common shortcut is "layer selection"—analyzing only a subset of the model's layers. This paper demonstrates that such approximations significantly distort attribution results.

Proposed Methods: Hardware-Aware Sketching

To address these challenges, the authors introduce three "sketching" algorithms optimized for modern hardware accelerators. These methods project massive vectors into a manageable subspace without the overhead of traditional approaches.

Schematic of the proposed sketching algorithms: AFFD, AFJL, and QK.
  • AFFD (Augmented Fastfood): An improved version of the Fastfood Transform that eliminates lookup-based bottlenecks, making it significantly faster on TPUs.
  • AFJL (Augmented Fast Johnson-Lindenstrauss): A hardware-optimized variant of the FJL algorithm using diagonal Gaussian matrices to achieve high-speed projections.
  • QK (Kronecker Product Decomposition): A memory-efficient approach utilizing Kronecker products of orthogonal matrices. This allows for sketching without ever materializing the full projection matrix, drastically reducing the memory footprint.

Additionally, the authors propose a single-run search algorithm to determine the Intrinsic Dimension (ID)—the minimum number of parameters required to achieve a target performance level. Unlike previous multi-run approaches, this method progressively doubles the dimension during a single training session.

Key Insights

The Pitfalls of Layer Selection

A major finding of this research is that the common practice of performing data attribution on only the final layer or token embeddings is fundamentally flawed. The authors observed that influence scores derived from partial models have low correlation with the "ground truth" calculated from the full parameter set.

Correlation (R) and eigenvalue error across GPT-2 layers, demonstrating the unreliability of single-layer selection.

Generative Tasks and Intrinsic Dimension

The study reveals a stark contrast between classification and generative tasks. While classification tasks (e.g., sentiment analysis) often possess a low intrinsic dimension (ID $\approx 8,000$), generative tasks like summarization exhibit an ID that approaches the model's total parameter count. This suggests that generative modeling utilizes nearly the entire capacity of the network.

Experimental Results

The researchers evaluated these methods across models like GPT-2, BART, and RoBERTa on tasks including summarization (XSUM) and classification (SNLI).

  • Efficiency: AFJL and AFFD achieved up to 100x and 64x speedups on TPUs, respectively, compared to standard implementations.
  • Memory: The QK algorithm reduced memory usage to 1.1 GB on a TPU, compared to 4.3 GB for standard Fastfood transforms.
  • Accuracy: AFFD maintained over 95% correlation with ground-truth attribution scores even with a highly compressed sketch dimension ($D=1,024$).

Conclusion

By aligning sketching algorithms with the strengths of modern hardware, this work enables large-scale study of LLM training dynamics and loss landscapes. The discovery of high intrinsic dimensionality in generative tasks challenges previous assumptions and highlights the need for full-model analysis in future interpretability research.

Summary updated

BibTeX
@inproceedings{schioppa2024gradient,
  author = {Andrea Schioppa},
  title = {{Gradient Sketches for Training Data Attribution and Studying the Loss Landscape}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=8jyCRGXOr5},
}
Figure