For years, the extreme vulnerability of neural networks to adversarial examples—tiny, calculated perturbations that flip a model's prediction—was treated as a bug in model architecture or a symptom of poor training. However, a growing body of research suggests a more unsettling reality: adversarial examples may be a fundamental, inescapable consequence of high-dimensional geometry.
If you partition a high-dimensional space into different classes, the math of that space almost guarantees that every point will sit precariously close to a boundary.
The Concentration of Measure
The core of the geometric argument lies in the concentration of measure phenomenon. In low-dimensional spaces, we think of the "middle" of a shape as being far from its edges. But as the number of dimensions $n$ increases, the geometry of space changes intuitively.
In high-dimensional spheres or cubes, almost all the volume of the shape concentrates in a thin shell near the surface. As Saeed Mahloujifar and Mohammad Mahmoody demonstrate in The Curse of Concentration in Robust Learning, this means that if you have a classifier with even a tiny error rate, the "error region" is geometrically close to almost every correctly classified point.
Justin Gilmer and colleagues illustrated this using a simple dataset of concentric spheres in Adversarial Spheres. They proved that for any model with non-zero test error in high dimensions, a misclassified point is mathematically guaranteed to be lurking within a tiny distance of nearly every input. In this view, adversarial examples aren't "outliers"; they are the inevitable neighbors of every data point.
Isoperimetric Inequalities and the Boundary
To quantify this "nearness," researchers like Ali Shafahi and Tom Goldstein apply isoperimetric inequalities. These mathematical tools relate the volume of a set to the surface area of its boundary.
In Are adversarial examples inevitable?, the authors show that in high-dimensional distributions, a set with a certain volume must have a boundary that is "close" to everything else. If a class occupies a significant portion of the space, its boundary is so sprawling and intricate that you only need a small nudge to cross it. Zheng Dai and David K. Gifford further proved that for image classifiers, these perturbations can be as small as changing a few columns of pixels, regardless of the model's architecture.
The Manifold Gap and High Codimension
Another facet of this problem is the high codimension of data. Natural data (like photos of cats) does not fill the entire high-dimensional pixel space; it lives on a low-dimensional "manifold" embedded within that giant space.
Dylan Hadfield-Menell and Marc Khoury argue in On the Geometry of Adversarial Examples that models often fail to define decision boundaries correctly in the "empty" space off this manifold. Because the embedding space has so many more dimensions than the data manifold, there are countless "off-manifold" directions an attacker can use to push a point into a misclassified region. These directions are effectively invisible to the model during standard training because it never encounters data there.
The Paradox of Apparent Stability
If models are so fragile, why do they seem stable in the real world? This is the "paradox of apparent stability" explored by Oliver J. Sutton and Ivan Y. Tyukin in How adversarial attacks can disrupt seemingly stable accurate classifiers.
In high dimensions, the volume of a sphere concentrates near its "equator" relative to any specific direction. If you apply random noise, the perturbation is statistically likely to point along the equator—staying within the same class. However, an adversarial attack is a targeted search for the one specific direction that points toward the "pole" (the decision boundary).
This explains why a model can be robust to massive amounts of random noise while collapsing under a tiny, surgical perturbation. The geometry "hides" the vulnerability from random sampling but leaves it wide open to gradient-based search.
Is Robustness Possible?
While much of this research suggests that vulnerability is a mathematical certainty, there is a counter-perspective. Ambar Pal and Jeremias Sulam argue in Adversarial Examples Might be Avoidable that robustness depends on data concentration.
If natural data is localized in extremely small, well-separated volumes, a robust classifier can theoretically exist. The challenge, then, is not just building better models, but understanding the intrinsic geometry of the data itself. As Jack Prescott and David Evans suggest in their work on half-space estimation, our current inability to reach these theoretical limits may be a failure of our training algorithms to respect the underlying geometry of the task.