In traditional machine learning, we assume our test data comes from the same distribution as our training data. Adversarial attacks break this assumption by introducing small, intentional perturbations to individual data points. While this is often viewed as a "cat-and-mouse" game of specific pixel tweaks, a powerful theoretical perspective has emerged: adversarial robustness is a problem of Optimal Transport.
By reframing point-wise attacks as global distributional shifts, researchers have uncovered the fundamental limits of how robust any model can truly be.
The Equivalence of Points and Distributions
The core intuition behind this research direction is a mathematical "sleight of hand." As Po-Ling Loh and Justin Khim demonstrate in Adversarial Risk Bounds for Binary Classification via Function Transformation, we can transform a standard classifier into an "adversarially-aware" function. This transformation reveals that perturbing individual samples within a budget is mathematically equivalent to shifting the entire underlying data distribution within a specific distance.
This distance is measured using the Wasserstein metric (or Earth-Mover Distance). In this framework, an adversary isn't just changing a few pixels; they are choosing a new probability distribution $Q$ that is "close" to the original distribution $P$. Zhuozhuo Tu, Dacheng Tao, and Jingwei Zhang formalize this in Theoretical Analysis of Adversarial Learning: A Minimax Approach, showing that adversarial risk is simply the standard risk under the worst-case distribution shift within a Wasserstein ball.
The Geometry of the Attack Budget
If adversarial attacks are distribution shifts, what limits the attacker? The "budget" (e.g., how much a pixel can change) defines the radius of the Wasserstein ball.
Specifically, the infinity-Wasserstein metric ($W_\infty$) has emerged as the natural language for these budgets. As Natalie Frank explores in Adversarial consistency and the uniqueness of the adversarial bayes classifier, $W_\infty$ treats attacks as a coupling problem between probability measures. It represents the maximum allowable distance any single "unit of mass" (data point) can be moved.
This geometric view allows us to see the adversary's strategy as a transport map. In Optimal Transport Based Distributionally Robust Optimization, Karthyek Murthy and Fan Zhang show that the worst-case distribution often manifests as a simple shift of data points along straight-line paths toward regions of higher loss.
Fundamental Limits and the Cost of Transport
One of the most significant results of this Optimal Transport (OT) perspective is the ability to calculate universal lower bounds on robustness.
In Lower Bounds on Adversarial Robustness from Optimal Transport, Arjun Bhagoji, Daniel Cullina, and Prateek Mittal apply Kantorovich duality to show that the minimum possible adversarial error is determined by the transport cost between class distributions. If an adversary can move mass from the "Cat" distribution to the "Dog" distribution within their budget, no model—no matter how complex—can perfectly distinguish them.
Varun Jog further solidifies this in Adversarial Risk via Optimal Transport and Optimal Couplings, proving that the fundamental limit of robust risk is exactly the cost required to make two class distributions indistinguishable. This provides a "ceiling" for defense researchers: if the OT cost between classes is low, high robustness is mathematically impossible.
Adversarial Training as a Nash Equilibrium
If the attacker is trying to maximize risk by shifting the distribution, and the learner is trying to minimize it, we are in a zero-sum game.
Laurent Meunier, Meyer Scetbon, and Rafael Pinot prove in Mixed Nash Equilibria in the Adversarial Examples Game that a Mixed Nash Equilibrium always exists in this setting. This suggests that the most robust defense might not be a single deterministic model, but a "mixture" or ensemble of classifiers.
This game-theoretic framing extends beyond simple classification. In reinforcement learning, Mathieu Godbout and colleagues show in A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning that an adversary perturbing environment dynamics leads to policies that are naturally risk-sensitive, specifically optimizing for Conditional-Value-at-Risk (CVaR).
The Adversarial Bayes Classifier
Just as standard machine learning has the Bayes optimal classifier, adversarial learning has the Adversarial Bayes Classifier.
In a series of papers, Natalie Frank (e.g., A Notion of Uniqueness for the Adversarial Bayes Classifier) has developed the theory for these optimal robust models. By leveraging the $W_\infty$ metric, she transforms the risk minimization problem into a dual form that allows for pointwise analysis of the decision boundary.
A surprising insight from this work is that as the attack budget increases, the optimal robust classifiers actually become simpler and more regular. This suggests that robustness isn't just about "defending" but about finding a more stable, generalized representation of the data that is inherently less sensitive to the transport of probability mass.