Data attribution aims to answer a fundamental question in machine learning: which specific training examples are responsible for a model's prediction? As models grow in complexity—from diffusion models to large language models (LLMs)—tracing outputs back to their sources becomes essential for debugging, ensuring fairness, and compensating creators.
However, the mathematical tools we use to measure this influence were largely designed for a simpler, "flatter" world. In the complex, non-convex landscapes of deep learning, these tools often break down.
The Linear Approximation Gap
The standard approach to data attribution, popularized by Koh et al., relies on Influence Functions (IF). The core idea is to "differentiate through the training process" using a first-order Taylor expansion. Instead of retraining a model thousands of times to see how removing a point $z$ changes the parameters, we use the gradient and the Hessian (the matrix of second-order derivatives) to estimate that change instantly.
This works beautifully in convex settings, like logistic regression, where the loss landscape has a single, well-behaved global minimum. But as Basu et al. demonstrate, influence functions in deep learning are "fragile." The non-convexity of neural networks means that the loss surface is highly curved and irregular. A linear approximation that holds at a specific local minimum may become completely inaccurate just a small distance away.
Why Influence Functions Fail in Deep Networks
The failure of traditional influence estimation in non-convex models stems from three primary issues:
- Hessian Instability: In deep networks, the Hessian matrix is often ill-conditioned or not positive semi-definite. Basu et al. found that this instability leads to a systematic underestimation of how training data affects predictions.
- Local vs. Global Minima: Influence functions assume that removing a training point causes an infinitesimal shift in parameters. In reality, removing a point might cause the optimization process to converge to an entirely different local minimum. As Ilyas et al. argue, these gradient-based approximations often fail to correlate with the actual impact of data removal.
- Model Depth and Width: The gap between theory and practice widens as models scale. Basu et al. show that increasing the depth and width of a network increases the curvature of the loss landscape, making first-order Taylor approximations increasingly unreliable.
Moving Beyond Convexity
To address these limitations, researchers are developing new frameworks that account for the non-linear and stochastic nature of deep learning.
Geometric and Ensemble Methods Rather than relying on a single gradient at a single point, some methods look at the broader geometry of the loss surface. Kim et al. proposed GEX (Geometric Ensemble for sample eXplanation), which estimates influence by measuring how sample loss fluctuates across an ensemble of model checkpoints. This captures non-linearities that a simple gradient-based linearization would miss.
Targeting Flat Minima The "sharpness" of the loss surface is a major source of error. Ye et al. introduced an estimator that targets Flat Validation Minima (FVM). By focusing on regions where the curvature is more regular, they improve the reliability of influence estimates for tasks like detecting mislabeled data.
Exact Influence via Metadifferentiation Instead of approximating the training process, some researchers seek to compute it more precisely. Ilyas et al. developed MAGIC, which uses metadifferentiation to calculate near-exact influence. This approach bypasses the coarse approximations of traditional influence functions, providing a much higher correlation with ground-truth retraining results.
The Distributional Perspective Perhaps the most radical shift is viewing influence not as a deterministic change in parameters, but as a shift in a distribution. Mlodozeniec et al. introduced Distributional Training Data Attribution (d-TDA), arguing that influence functions actually predict how the entire distribution of possible models shifts when data is removed. This framework helps explain why influence functions often work in practice even when their strict convex assumptions are violated.
Practical Implications
Despite the theoretical hurdles, non-convex influence estimation is becoming a practical tool for model auditing. It is being used to identify "harmful" data points that degrade accuracy Chhabra et al., detect noisy labels in contrastive learning Xie et al., and even "unlearn" specific graph elements without retraining Wu.
As libraries like dattri Deng et al. standardize these methods, the field is moving toward a more robust understanding of how data flows through the non-convex heart of modern AI.