Robust Deep Prototype Classifiers via Lipschitz Training

Adia Khalid Khan, Lukas Gehring, Benjamin PaaßenarXiv 2026 · arXiv

TL;DR

This paper introduces a novel approach to building deep learning models that are both robust to adversarial attacks and inherently interpretable. By combining prototype-based classifiers with a new "Lipschitz training" scheme, the authors provide a way to transfer robustness guarantees from a low-dimensional embedding space back to the original high-dimensional input space (like images).

What Problem Does This Paper Address?

Deep neural networks are notoriously fragile: a tiny, imperceptible change to an image (an "adversarial attack") can completely flip the model's prediction. This isn't just a security flaw; it's an interpretability problem. If a model changes its mind based on features a human can't even see, we can't truly "understand" its decision-making process.

While some "shallow" models like prototype-based classifiers are interpretable by design and have built-in robustness certificates, they struggle with complex data like images. To make them work, researchers often use a deep "embedding network" to simplify the data first. However, an unconstrained deep network can "stretch" space: it might map two very similar images to two very different points in the embedding space, effectively breaking any robustness guarantees the classifier had.

Methods

The authors propose a two-part solution to bridge this gap:

  1. A Composition Theorem: They prove that if an embedding network is locally $L$-Lipschitz (meaning it doesn't amplify distances by more than a factor of $L$) and the classifier is $\epsilon$-robust in the embedding space, then the entire combined model is guaranteed to be $\epsilon/L$-robust in the original input space.
  2. Lipschitz Training: To actually achieve this $L$-Lipschitz property, they introduce a new training loss. During training, for every data point, they sample a nearby "neighbor" by adding a small amount of Gaussian noise. They then penalize the network if the distance between these two points in the embedding space is too large relative to their distance in the input space.

This approach is computationally much cheaper than standard "adversarial training" because it only requires sampling random noise rather than solving a complex optimization problem to find the "perfect" attack for every training step.

Figure 1: Top Left: An adversarial attack of size d from point x. Top right: In embedding space, the attack is enlarged to size D, violating Lipschitz continuity. Bottom: With Lipschitz training, the classification boundary shifts such that the attack fails because D is now upper- bounded by L · d.

Datasets

The researchers evaluated their method on three standard image classification benchmarks of increasing complexity:

  • MNIST: 28x28 grayscale handwritten digits.
  • CIFAR-10: 32x32 color images across 10 categories.
  • Imagenette-160: A 10-class subset of ImageNet, resized to 64x64 pixels.

Key Insights and Ideas

  • Robustness and Interpretability are Linked: The authors argue that a truly interpretable model should only change its prediction when the input has changed enough for a human to notice a difference.
  • The "Stretching" Problem: The core contribution is identifying and fixing the "gap" where deep embedding networks invalidate the formal robustness of shallow classifiers.
  • Efficiency Matters: Lipschitz training offers a "middle ground" defense. It's faster than adversarial training and less restrictive than global Lipschitz bounds (which often ruin a model's accuracy).
  • Prototype Power: Prototype-based classifiers (like GLVQ) are naturally more robust than standard linear layers because their decisions are based on distances to "typical" examples, providing a natural "margin" of safety.

Key Results

The combination of a Lipschitz-constrained embedding and a GLVQ (prototype) head consistently provided the strongest robustness across different types of attacks (FGSM, PGD, and C&W).

  • MNIST Resilience: On the MNIST dataset, the Lipschitz-GLVQ model retained roughly 52.9% accuracy under a strong PGD attack ($\epsilon = 0.20$), while all standard, unconstrained models collapsed to near-zero accuracy.
  • Increased Distortion: The mean $\ell_2$ distortion (the amount of "noise" needed to fool the model) nearly doubled for the GLVQ models on MNIST when Lipschitz training was applied (rising from 1.57 to 2.93).
  • The Accuracy Trade-off: While robustness improved significantly, there was a cost to "clean" accuracy (accuracy on un-attacked images), particularly on more complex datasets like Imagenette, where accuracy dropped from ~73% to ~65% to achieve the robustness gains.

Limitations and Future Work

  • Empirical vs. Certified: Because the Lipschitz constraint is enforced by random sampling during training rather than by the mathematical structure of the network itself, the resulting robustness is "empirical" (observed in practice) rather than "certified" (mathematically guaranteed for all possible cases).
  • Complexity Scaling: While the method works exceptionally well on MNIST, the gains on CIFAR-10 and Imagenette are more modest against the strongest multi-step attacks.
  • Hyperparameter Sensitivity: The model's performance is sensitive to the choice of the Lipschitz constant $L$ and the noise level $\sigma$, requiring careful tuning for different datasets.
  • Future Directions: The authors suggest exploring more effective sampling schemes and testing the approach on non-image data modalities.

Summary updated

Abstract

Robustness and interpretability are interrelated and desirable properties of machine learning models. A model that changes its prediction under imperceptible perturbations behaves inconsistently with human-recognizable features and, hence, is not interpretable. Prototype-based classifiers such as Generalized Learning Vector Quantization address both at once. They are interpretable by construction because they make the decision by comparing the input to the prototypical example; and their hypothesis margin yields a closed form robustness certificate. However, on high-dimensional data, these shallow models need deep embedding networks to achieve high accuracy, and this voids the robustness certificate. The margin is a statement about distances in the embedding space, and an unconstrained network can map an arbitrarily small input perturbation to arbitrarily large attacks in the embedding space. We close this gap with a theorem: if the embedding network is locally L-Lipschitz around a point, and the classifier is ε-robust around its embedding, then the full model is ε/L-robust in the input space. Based on this, we propose a Lipschitz training scheme to achieve L-Lipschitz properties for embedding networks. We evaluate three classification heads (linear, linear Support Vector Machine, and GLVQ) crossed with two embedding networks (standard and Lipschitz-constrained) on MNIST, CIFAR-10, and Imagenette-160, under FGS, PGD and C\&W attacks. The evaluation results show that combining a prototype-based head with a Lipschitz-constrained embedding provides the strongest robustness. This suggests that the Lipschitz constraint provides an additional robustness benefit in addition to the advantage provided by the prototype-based head. Since the Lipschitz constraint is enforced by sampling rather than by construction, the reported robustness is empirical rather than certified.
BibTeX
@misc{khan2026robust,
  author = {Adia Khalid Khan and Lukas Gehring and Benjamin Paaßen},
  title = {{Robust Deep Prototype Classifiers via Lipschitz Training}},
  year = {2026},
  eprint = {2611.09327},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09327},
}
Figure