Abstract
Neural Ordinary Differential Equations (ODEs) have gained traction in many applications. While recent studies have focused on empirically increasing the robustness of neural ODEs against natural or adversarial attacks, certified robustness is still lacking. In this letter, we propose a framework for training a neural ODE using barrier functions and demonstrate improved robustness for classification problems. We further provide the first generalization guarantee of robustness against adversarial attacks using a wait-and-judge scenario approach.
Introduction
Deep learning models are notoriously fragile; small, intentional "adversarial" perturbations to an input can cause a model to confidently misclassify it. Neural ODEs—which treat the layers of a network as a continuous dynamical system—offer some inherent resistance to these attacks due to their mathematical structure, but they are still far from secure.
The challenge is two-fold: empirical robustness often fails against novel attacks, and there is a lack of generalization guarantees proving that robustness on training data translates to unseen samples. This paper introduces B-NODE, a framework that makes Neural ODEs "certifiably robust" by integrating control-barrier functions into the training process.
Methodology: Learning-based Barrier Functions
The authors bridge the gap between deep learning and classical control theory by introducing the Learning-based Barrier Function. In control theory, a barrier function defines a "safe" region of a system's state space that the system is guaranteed never to leave. In B-NODE, the "safe" region is the set of hidden states that lead to a correct classification.
Key components include:
- The Robustness Certificate: A proof that if a specific differential inequality holds for the ODE's trajectory, the model is guaranteed to return to a correct classification, even if perturbed.
- B-NODE Training: The loss function includes "barrier" terms (using a Lagrangian relaxation) to penalize internal dynamics that move toward decision boundaries.
- Wait-and-Judge Scenario Approach: A statistical framework used to calculate a "robustness probability" for unseen data with high confidence after training.
The authors realized that the final affine transformation of the network can serve as the barrier function itself, simplifying the math and making training computationally feasible.
Experimental Results
B-NODE was evaluated against standard benchmarks including MNIST, FashionMNIST, and CIFAR10. It consistently outperformed vanilla NODEs and other state-of-the-art robust ODE methods like TisODE and ODE-TRADES across various attacks.
Against PGD attacks on MNIST at a perturbation magnitude of 0.05, B-NODE maintained 83.18% accuracy, while the vanilla NODE plummeted to 47.40%. On the gradient-free "Square Attack," B-NODE achieved 85.49% accuracy on MNIST. For CIFAR10, B-NODE achieved a robust accuracy of 61.81% at PGD-6/255, compared to 58.56% for standard adversarial training.
Furthermore, the generalization proof for a training set of 2,000 samples mathematically guaranteed a robust accuracy of at least ~53-60% on unseen data with 99% confidence.
Conclusion
B-NODE represents a significant step forward in securing Neural ODEs. By borrowing from control theory, it provides both empirical performance gains and the first mathematical generalization guarantees for robustness. Future work will explore more complex barrier functions and apply this framework to regression and time-series prediction tasks.
Summary updated