Why the Local Linearity Assumption Fails

82 concepts65 papers1 proposal

In introductory calculus, we learn a foundational geometric intuition: if you zoom in close enough to any smooth curve, it eventually looks like a straight line. This concept of local linearity is the bedrock of modern machine learning interpretability, diagnostics, and optimization. If a deep neural network represents a highly complex, high-dimensional landscape, we should be able to understand its behavior at a specific point by looking at its tangent plane—its first-order Taylor expansion (the gradient).

This assumption underpins post-hoc explanation frameworks like LIME and SHAP, which perturb inputs locally and fit a sparse linear surrogate model to explain individual predictions. It also justifies gradient-based saliency maps, adversarial attack generation, and linearized data attribution methods.

The problem is that deep neural networks do not form smooth, gently sloping hills. When we assume local linearity, we assume the network's behavior remains stable within the neighborhood being analyzed. In reality, modern deep architectures produce local landscapes that are highly fractured, sharp, and oscillatory, causing first-order approximations to fail catastrophically.

The Reality of Deep Landscapes

The tension between linear approximations and non-linear realities is not new. As early as 1994, Hochreiter et al. noted in Simplifying Neural Nets by Discovering Flat Minima that algorithms relying on linear approximations of output changes fail when the underlying loss landscape is highly non-linear. Modern deep architectures have only exacerbated this issue. Zooming into a deep network's decision boundary rarely reveals a clean, flat plane; instead, it uncovers extreme local curvature and sharp transitions.

When a model exhibits high complexity, a simple linear surrogate model cannot consistently approximate the decision boundary across the data space. This mismatch in functional complexity leads to severe explanation errors, particularly in regions where the model's behavior is most intricate.

Why Local Surrogates Break Down

When post-hoc explainers attempt to demystify a black-box prediction, they sample points around a target instance and fit a linear model to those samples. However, as Ghalebikesabi et al. argue in On Locality of Local Explanation Models, if the target model exhibits extreme non-linearity or discontinuities within that small region, linear attribution methods fundamentally fail to capture the true decision logic.

This creates a severe fidelity gap. Chowdhury et al. demonstrated in Rank-LIME: Local Model-Agnostic Feature Attribution for Learning to Rank that while linear surrogates can faithfully approximate traditional probabilistic models, they struggle immensely with the highly non-linear local decision boundaries of transformer-based architectures.

This mismatch leads to a phenomenon known as low local concordance, detailed by Amparore et al. in To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods. When local linearity is violated, the explanation's predicted label can directly contradict the actual model output. If the local landscape fluctuates wildly, the surrogate's goodness-of-fit plummets, rendering the resulting explanations untrustworthy and misleading (Ahmad et al., Death vs. Data Science: Predicting End of Life).

The Mathematical Failure of Taylor Decompositions

The breakdown of local linearity is a mathematical certainty in piecewise linear networks (such as those utilizing ReLU activations). Many popular saliency methods rely on Taylor expansions to propagate relevance or importance from an input back to its features. These methods assume that the function remains within a single linear region between the input point and a chosen reference point.

In practice, this assumption is routinely violated. In A Rigorous Study Of The Deep Taylor Decomposition, Sixt demonstrates that selected reference points frequently fall outside the local linear region of the input. When the path between the input and the reference point crosses multiple activation boundaries, the decomposition becomes mathematically invalid, often collapsing into an uninformative gradient calculation.

Furthermore, by treating the local neighborhood as linear, first-order methods completely neglect the impact of the loss surface's curvature. As Singla et al. show in Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation, ignoring higher-order interactions (the Hessian) results in saliency maps that fail to reflect the true decision-making process in non-linear regions.

Downstream Consequences: Recourse and Robustness

When the local linearity assumption fails, it does not just produce inaccurate explanations—it breaks downstream applications like algorithmic recourse, data attribution, and adversarial defense.

Invalid Algorithmic Recourse

When a machine learning model denies a user a loan, algorithmic recourse frameworks use local linear approximations to suggest counterfactual actions (e.g., "if you increase your income by \$5,000, the model will approve you"). However, Nguyen-Hien et al. show in Robust Bayesian recourse that because these surrogates fail to capture non-linear relationships, the recommended actions often fail completely when applied to the actual model.

Closed-form guarantees for counterfactual plans collapse when the underlying decision boundary is highly irregular or non-linear (Bui et al., Counterfactual Plans under Distributional Ambiguity; Pawelczyk, Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse).

Unreliable Data Attribution

Linearizing a network to attribute its behavior back to specific training points (such as using the empirical Neural Tangent Kernel) depends heavily on the local smoothness of the loss landscape. As noted by Park in TRAK: Attributing Model Behavior at Scale, sharp transitions in model logic mean first-order approximations fail to accurately reflect which training points influenced a prediction.

Vulnerable Adversarial Defenses

First-order adversarial generation and defense methods assume that deep networks exhibit sufficient local linearity for Taylor approximations to remain valid. If a model possesses highly non-linear regions or high local curvature, these linear approximations fail to accurately represent the impact of adversarial perturbations, rendering first-order defenses ineffective (Balda et al., On Generation of Adversarial Examples using Convex Programming; Lyu, A Unified Gradient Regularization Family for Adversarial Examples).

Diagnosing and Moving Beyond Linearity

To mitigate these failures, researchers have begun developing diagnostic tools to quantify exactly when the local linearity assumption is being violated. For example, Löwe et al. introduced a non-linearity metric in fmeffects: An R Package for Forward Marginal Effects to evaluate the fidelity of a linear approximation over a discrete feature change. This allows practitioners to identify regions where a model's behavior is too complex for a simple linear summary to be accurate.

Recognizing the fragility of the local linearity assumption forces the field to move away from naive tangent-plane approximations. To achieve true fidelity, next-generation interpretability and optimization frameworks must incorporate higher-order curvature information, utilize non-linear local surrogates, or strictly bound evaluations to regions where linearity can be mathematically guaranteed.

Key Foundational Work

Simplifying Neural Nets by Discovering Flat Minima (1994) — contributed 1 concept elements to this direction.
A New Discriminative Kernel From Probabilistic Models (2001) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_local_linearity_assumption_17cd7cbd,
  title = {local linearity assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-linearity-assumption/rdr_17cd7cbd21b04c06837959183fef0d9c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_local_linearity_assumption_17cd7cbd_v1,
  title = {local linearity assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-linearity-assumption/rdr_17cd7cbd21b04c06837959183fef0d9c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake