Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning

Mathias Lechner, Alexander Amini, Daniela Rus, Thomas A. Henzinger

Abstract

Adversarial training (i.e., training on adversarially perturbed input data) is a well-studied method for making neural networks robust to potential adversarial attacks during inference. However, the improved robustness does not come for free but rather is accompanied by a decrease in overall model accuracy and performance. Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off but inflict a net loss when measured in holistic robot performance. This work revisits the robustness-accuracy trade-off in robot learning by systematically analyzing if recent advances in robust training methods and theory in conjunction with adversarial robot learning, are capable of making adversarial training suitable for real-world robot applications. We evaluate three different robot learning tasks ranging from autonomous driving in a high-fidelity environment amenable to sim-to-real deployment to mobile robot navigation and gesture recognition. Our results demonstrate that, while these techniques make incremental improvements on the trade-off on a relative scale, the negative impact on the nominal accuracy caused by adversarial training still outweighs the improved robustness by an order of magnitude. We conclude that although progress is happening, further advances in robust learning methods are necessary before they can benefit robot learning tasks in practice.

The Robustness-Accuracy Dilemma in Robotics

In deep learning, adversarial training (AT) is the gold standard for making networks resilient to malicious, near-invisible perturbations. However, AT is famous for a "no free lunch" rule: as you make a model more robust, its performance on clean, unperturbed data (nominal accuracy) typically drops.

In robotics, this tradeoff is often a dealbreaker. A robot that is robust to adversarial noise but crashes 10% more often during normal operation is a net liability. A previous influential study (Lechner et al., 2021) concluded that AT wasn't ready for robotics because the performance hit was too high. This paper revisits that conclusion to see if the latest tools in the AI shed—specifically larger models and Transformer architectures—can change the verdict.

Evaluating Modern Robustness Boosters

The authors systematically evaluate five modern "robustness boosters" across three robotic tasks:

  1. Overparameterization: Moving from small CNNs to much larger ones (e.g., ResNet152) to see if increased capacity allows the model to learn both accuracy and robustness.
  2. Vision Transformers (ViTs): Testing if the self-attention mechanism is inherently more robust than the local filters of CNNs.
  3. Smooth Activations: Replacing standard ReLU with SiLU or ELU to provide smoother gradients during training.
  4. Advanced AT (AWP): Using Adversarial Weight Perturbation, which injects noise into the network's weights to smooth the loss landscape.
  5. Bag of Tricks: Tuning hyperparameters like weight decay and learning rate schedules specifically for robust objectives.
A visual summary of how model architecture and training techniques influence the tradeoff between accuracy and robustness.

Experimental Benchmarks

The study uses three distinct benchmarks to represent real-world robotic challenges:

  • End-to-End Driving (VISTA Simulator): A high-fidelity "sim-to-real" environment where models predict road curvature. It includes "out-of-distribution" (OOD) tests like driving at night or in autumn after training on summer/winter data.
  • Visual Gesture Recognition: A 3-class image classification task (idle, enable, disable) used to control a mobile robot. The authors introduced a new test set for this task to better estimate real-world performance.
  • LiDAR-based Navigation: A 1D-CNN task where a controller maps laser scans to movement categories, evaluated using Certified Safety-Domain Training to ensure the robot never chooses a "forward" command when an obstacle is detected.

Key Results and Insights

  • The "Order of Magnitude" Problem: While modern methods improve the tradeoff on a relative scale, the negative impact on nominal accuracy still outweighs the robustness gains by an order of magnitude. In many cases, a standard model is still safer for everyday use.
  • ViTs > CNNs: Vision Transformers consistently outperformed CNNs in robust scenarios. The global context provided by self-attention makes ViTs naturally more resilient to the local pixel-level noise typical of adversarial attacks.
  • The Power of Combination: No single trick solved the problem. The most promising results came from "stacking" methods—specifically, using a Large ViT combined with AWP.
  • Robustness vs. Generalization: A surprising finding was that adversarial training often hurt the model's ability to handle distribution shifts. In the driving task, adversarially trained models struggled significantly more with night-time driving than their standard counterparts.
  • Performance Metrics: In gesture recognition, a standard ResNet50 achieved 93.5% accuracy but only 2.3% robustness. Switching to a ViT-Large with AWP achieved a better balance: 88.9% accuracy and 89.7% robustness.

Conclusion

The primary limitation remains that even with state-of-the-art architectures, the "robustness tax" remains high. The authors conclude that we are still missing a fundamental piece of the puzzle. While progress is happening through the integration of ViTs and advanced weight perturbation, adversarial training is not yet ready for prime-time robot deployment where nominal safety is paramount. Future work must focus on developing architectures that do not rely on the same gradient-based learning that makes current models vulnerable.

Summary updated

Abstract

Adversarial training (i.e., training on adversarially perturbed input data) is a well-studied method for making neural networks robust to potential adversarial attacks during inference. However, the improved robustness does not come for free but rather is accompanied by a decrease in overall model accuracy and performance. Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off but inflict a net loss when measured in holistic robot performance. This work revisits the robustness-accuracy trade-off in robot learning by systematically analyzing if recent advances in robust training methods and theory in conjunction with adversarial robot learning, are capable of making adversarial training suitable for real-world robot applications. We evaluate three different robot learning tasks ranging from autonomous driving in a high-fidelity environment amenable to sim-to-real deployment to mobile robot navigation and gesture recognition. Our results demonstrate that, while these techniques make incremental improvements on the trade-off on a relative scale, the negative impact on the nominal accuracy caused by adversarial training still outweighs the improved robustness by an order of magnitude. We conclude that although progress is happening, further advances in robust learning methods are necessary before they can benefit robot learning tasks in practice.
BibTeX
@article{lechner2023revisiting,
  author = {Mathias Lechner and Alexander Amini and Daniela Rus and Thomas A. Henzinger},
  title = {{Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2023},
  url = {https://openreview.net/forum?id=A5U2OCM2Sk},
}
Figure