LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

Arthur Chiron, Franck Mamalet, Thomas Massena, Thomas Deltort, Mathieu SerrurierarXiv 2026 · arXiv

TL;DR

LiST (Lipschitz Scaling Training) bridges the gap between adversarial robustness and model calibration by revealing a fundamental duality between Lipschitz constraints and Temperature Scaling. By dynamically adjusting a network's global Lipschitz constant during training based on validation calibration feedback, LiST automatically discovers an optimal, certifiably robust model that is perfectly calibrated out-of-the-box without manual hyperparameter tuning.

What Problem Does This Paper Address?

For deep learning models deployed in safety-critical domains like autonomous driving or medical diagnostics, three pillars of trustworthiness are non-negotiable: accuracy, robustness to adversarial attacks, and calibration (ensuring a model's confidence scores match its actual empirical accuracy). Unfortunately, these three properties are usually studied in isolation, and standard neural networks remain notoriously brittle and overconfident.

Lipschitz-constrained Neural Networks (LNNs) offer a mathematically rigorous way to guarantee adversarial robustness by limiting how wildly a model's outputs can change in response to small input perturbations. However, choosing the right global Lipschitz constraint ($L$) is a painful, manual process of grid-searching across an accuracy-robustness Pareto front. Furthermore, the calibration properties of these robust networks have remained largely unexplored—until now.

Methods

The authors introduce Lipschitz Scaling Training (LiST), a two-phase training paradigm that treats the global Lipschitz constant not as a static hyperparameter, but as a dynamic control variable guided by calibration.

  • Phase I: Dynamic Search: The algorithm exploits a structural duality between Temperature Scaling (a popular post-hoc calibration method that divides logits by a scalar $T$) and the Lipschitz constraint $L$. At each epoch, LiST computes the optimal temperature $T_e^*$ on a held-out validation set and updates the Lipschitz constraint using a negative feedback loop: $$L_{e+1} \leftarrow L_e / T_e^*$$ If the model is overconfident ($T^* > 1$), the constraint tightens; if it is underconfident ($T^* < 1$), the constraint relaxes. The system naturally converges to an intrinsic equilibrium ($L^*$) where $T^* = 1$.

  • Phase II: Data Re-integration: Once $L^*$ is found and frozen, the validation data is no longer needed for feedback. Because a fixed Lipschitz bound strictly limits the model's Rademacher complexity, the authors prove it is theoretically safe to merge the validation data back into the training set for a final fine-tuning phase, boosting sample efficiency without risking miscalibration.

Additionally, by introducing a margin offset parameter ($\xi$) into the cross-entropy loss, users can safely slide along the accuracy-robustness Pareto front while LiST automatically keeps the model calibrated.

Datasets

The method was validated on three standard image classification benchmarks:

  • CIFAR-10
  • CIFAR-100
  • Tiny-ImageNet

The authors used a stratified 90/10 train/calibration split for Phase I before re-integrating the 10% calibration data in Phase II.

Key Insights and Ideas

  • The Lipschitz-Temperature Duality: Training a network with a specific Lipschitz constraint $L$ yields an Expected Calibration Error (ECE) profile remarkably similar to applying post-hoc Temperature Scaling to a fixed 1-Lipschitz network.
Figure 1: ECE as a function of the logit scaling mechanism. (a) Networks trained with varying fixed Lipschitz constants L. (b) Post-hoc Temperature Scaling on a fixed 1-Lipschitz network (plotted against 1/T). Both profiles are qualitatively similar.
  • Calibration as an Unsupervised Guide: Instead of arbitrarily picking a point on the accuracy-robustness trade-off, calibration serves as a natural, unsupervised criterion to find a well-defined operating equilibrium between under-fitting (over-constrained) and instability (under-constrained).
  • Robustness to Initialization: No matter if you initialize $L$ to be tiny (0.01) or large (10.0), the dynamic feedback loop reliably converges to the exact same intrinsic $L^*$ for a given task and architecture, effectively eliminating the need for expensive grid searches.
Figure 2: Training dynamics of LiST (Phase I). (a) Evolution of the Lipschitz constraint L for four initializations L0. Regardless of L0, the algorithm first relaxes the constraint to gain expressivity (relaxation regime), then progressively tightens it as the model becomes over-confident (contraction regime), converging to the same intrinsic value L∗. (b) Corresponding evolution of the calibration temperature T. The inset highlights the asymptotic convergence towards the target equilibrium T = 1.

Key Results

  • Superior Out-of-the-Box Calibration: On CIFAR-10, LiST achieved an ECE of 0.79%, significantly outperforming unconstrained baselines like Focal Loss (1.68%) and the best fixed-L baseline (1.60%)—all without any post-hoc calibration adjustments.
  • Decoupled Trade-offs: By tuning the loss offset $\xi$ from 0 to 3, the authors successfully constructed a fully calibrated Pareto front. They moved the model's Certified Robust Accuracy (CRA) from 34.52% to 63.06% while keeping the ECE exceptionally low (~1.17%).
  • Data Re-integration Gains: Phase II consistently improved both clean accuracy and robustness across all datasets (e.g., providing up to a +0.52% accuracy boost on Tiny-ImageNet) by safely utilizing 100% of the available data.

Limitations and Future Work

While the authors provide a formal proof for the existence of the optimal calibrated constant $L^*$, they note that its uniqueness remains an open theoretical question. Additionally, the discovered $L^*$ is highly dependent on the specific training setup (architecture, optimizer, data augmentation) and cannot be directly transferred to different configurations.

Future work includes extending the LiST framework to other neural network architectures (like Vision Transformers), different modalities beyond vision, and exploring its utility in out-of-distribution detection tasks.

Summary updated

Abstract

While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge. Lipschitz-constrained models guarantee robustness by design, yet the manual selection of the Lipschitz constraint L governs the resulting accuracy-robustness trade-off, and their calibration properties remain largely underexplored. In this work, we highlight a theoretical and empirical link between the enforced Lipschitz constraint and Temperature Scaling, a state-of-the-art calibration method. Specifically, we find that for a given training scheme, there exists a non-trivial value L* that yields an out-of-the-box calibrated network, and that calibration acts as a principled criterion to select a well-defined operating point on the accuracy-robustness Pareto front. Leveraging these insights, we introduce Lipschitz Scaling Training (LiST), a novel training paradigm that iteratively adjusts the global Lipschitz constant to reach this operating point. Through a margin parameter in the training loss, LiST further enables the construction of a fully calibrated Pareto front, allowing users to navigate the accuracy-robustness trade-off while remaining calibrated throughout. At convergence, LiST also enables the reintegration of calibration data into training, improving sample efficiency without sacrificing calibration. We validate LiST on CIFAR-10/100 and Tiny-ImageNet, demonstrating competitive accuracy and robustness against constrained and unconstrained baselines, while remaining calibrated out of the box. Code is available at GitHub.
BibTeX
@misc{chiron2026list,
  author = {Arthur Chiron and Franck Mamalet and Thomas Massena and Thomas Deltort and Mathieu Serrurier},
  title = {{LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks}},
  year = {2026},
  eprint = {2607.07745},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.07745},
}
Figure