TL;DR
DABGO introduces a bidirectional gradient optimization framework to trace which specific training data points most heavily influence an LLM's open-ended text generation. By perturbing a base model using both gradient ascent (unlearning) and gradient descent (finetuning) on a generated text sample, it accurately captures both factual and stylistic data provenance, significantly outperforming existing data attribution baselines.
What Problem Does This Paper Address?
When a Large Language Model (LLM) generates a fact or mimics a specific writing style, it is often nearly impossible to know exactly which parts of its massive training corpus triggered that specific output. This challenge, known as Training Data Attribution (TDA), is critical for AI governance, copyright accountability, and data valuation.
The gold standard for TDA is counterfactual retraining—removing a training sample and retraining the model from scratch to see how the output changes. However, doing this for millions or billions of documents is computationally impossible. Existing shortcuts, like traditional influence functions, require calculating and inverting a massive Hessian matrix, which is also intractable for modern LLMs. Newer "backward influence" methods attempt to "unlearn" the generated text via gradient ascent to see which training samples suffer, but they have historically been limited to single-word predictions or coarse, dataset-level granularity.
DABGO tackles the challenge of performing fine-grained data attribution for fully open-ended, auto-regressive text generation.
Methods
The authors introduce DABGO (Data Attribution via Bidirectional Gradient Optimization). Instead of relying on one-sided unlearning, DABGO uses a symmetric, bidirectional approach to perturb the base model based on a generated text sample ($\hat{x}$):
- Gradient Descent ($\pi_{\theta_-}$): Optimizes the model to make the generated text more likely (similar to standard finetuning).
- Gradient Ascent ($\pi_{\theta_+}$): Optimizes the model to make the generated text less likely (similar to machine unlearning).
To prevent the model from completely destroying its underlying weights during these steps, the authors apply Fisher-regularized gradient updates, a technique borrowed from continual learning known as Elastic Weight Consolidation (EWC).
Once these two perturbed models are created, DABGO evaluates the entire training dataset against both. It computes a Bidirectional Influence Score (BIS) for each training sample by measuring the absolute difference in its loss between the two models. If a training sample's loss swings drastically between the "ascent" and "descent" models, it indicates a strong causal coupling to the generated text. This framework can also be zoomed into the token level to highlight exactly which words within a document carried the most influence.
Datasets
The authors evaluated DABGO by training GPT-2 architecture models from scratch on two distinct corpora:
- Wikipedia Corpus (230M tokens): Derived from the WIT dataset, this corpus features a highly uniform writing style. It was chosen to isolate and evaluate factual attribution capabilities.
- Project Gutenberg Archive (2M tokens): Containing literary texts from various authors and eras, this dataset provides rich stylistic variation, making it ideal for evaluating stylistic attribution.
Key Insights and Ideas
- The Power of Symmetry: A key "aha moment" is that gradient ascent and descent capture complementary signals. Because LLMs are generative and probabilistic, looking at how a training sample responds to both the inflation and deflation of a target text's likelihood yields a much cleaner attribution signal than unlearning alone.
- Style is Distributed, Facts are Localized: The paper reveals a fundamental difference in data provenance: factual generation often traces back heavily to a single, highly overlapping source document. In contrast, stylistic generation draws minor influences from a wide array of documents across an author's portfolio.
- Empirical Validation via Retraining: Rather than just trusting proxy metrics, the authors undertook the expensive task of actually retraining models from scratch without the top-$k$ predicted influential documents to verify that the loss on the generated text truly spiked.
Key Results
The authors evaluated their method using the Tail-Patch Absolute (TPA) score, which measures how much the top-$k$ attributed samples affect the model's likelihood of the generated text.
- Factual Attribution (Wikipedia): DABGO consistently outperformed TRACKSTAR (a gradient-projection method) and GECKO (an embedding retrieval baseline). While BM25 performed well at $k=1$ due to direct lexical overlap, DABGO dominated as $k$ increased, capturing deeper semantic influences.
- Stylistic Attribution (Gutenberg): Style attribution proved much harder for traditional keyword methods like BM25. DABGO achieved a TPA score of 470.7 at $k=20$, vastly outperforming GECKO (258.5) and TRACKSTAR (60.6).
- Retraining Verification: As shown in Figure 2, when the top-$k$ documents identified by DABGO were completely removed from the training set and the model was retrained from scratch, the resulting model suffered a significantly higher loss on the target text compared to when baseline-selected documents were removed.
Limitations and Future Work
- Computational Overhead: DABGO requires two full forward/backward passes over the entire training corpus to compute the final influence scores. While faster than retraining, this is still too computationally expensive for industrial-scale LLMs trained on trillions of tokens.
- No Sample Interactions: The method calculates influence on a per-sample basis, meaning it cannot account for complex synergistic or adversarial interactions between groups of multiple training documents.
- Token-Level Causality: While token-level tracking provides beautiful qualitative highlights of influential phrases, the authors note that its exact causal mechanism remains unverified and warrants future study.
Summary updated