FLOAT: Fast Learnable Once-for-All Adversarial Training for Tunable Trade-off between Accuracy and Robustness

Souvik Kundu, Sairam Sundaresan, Massoud Pedram, Peter A. Beerel

Introduction

Deep Neural Networks (DNNs) are highly susceptible to adversarial attacks, where small, calculated perturbations to input data lead to incorrect classifications. While Adversarial Training (AT) provides a robust defense, it typically forces a trade-off: increasing robustness often degrades accuracy on clean, unperturbed images. Traditional AT models are static, fixed at a single point on this trade-off curve. Adapting to varying environmental conditions (e.g., switching from high accuracy in clear weather to high robustness in stormy conditions) usually requires deploying multiple models, which is impractical for resource-constrained edge devices.

The FLOAT Framework

To address these challenges, the authors propose FLOAT (Fast Learnable Once-for-All Adversarial Training). FLOAT enables a single neural network to dynamically adjust its trade-off between clean-image accuracy and adversarial robustness at inference time without the need for additional conditioning layers or significant parameter overhead.

Methodology

FLOAT introduces three core architectural and training innovations:

  1. Weight-Conditioned Learning: Unlike previous "Once-for-All" (OAT) methods that use Feature-wise Linear Modulation (FiLM) layers to adjust activations, FLOAT modulates the network's weight tensors directly. It adds a learnable noise tensor ($\alpha' * \eta^I$) to the original weights ($\theta'$), controlled by a switch ($\lambda$). This allows the model to capture distinct feature distributions for clean and adversarial inputs within the same weight space.
  2. Dual Batch Normalization (BN): Clean and adversarial images exhibit different statistical properties. FLOAT employs two independent BN branches ($BN_C$ for clean, $BN_A$ for adversarial) to track these distributions separately, switching between them based on the input type.
  3. Inference-Time Tuning: At test time, users can provide a continuous hyperparameter $\lambda_n \in [0, 1]$. This parameter scales the weight modulation, allowing the model to smoothly transition between its "clean" and "robust" operational modes.
Comparison of OAT and FLOAT architectures (a) and the resulting accuracy-robustness trade-off curve (b).

Experimental Evaluation

The researchers evaluated FLOAT and its sparse variant, FLOATS, across several benchmarks, including CIFAR-10/100, SVHN, STL10, and a subset of ImageNet. They tested these on popular architectures like ResNet34 and WideResNet (WRN16-8, WRN40-2).

Key Results

FLOAT demonstrates significant improvements over existing state-of-the-art (SOTA) OAT methods in terms of performance, efficiency, and speed:

  • Performance Gains: FLOAT achieved up to a ~6% increase in clean accuracy and a ~10% increase in robust accuracy compared to previous methods.
  • Efficiency: The framework uses up to 1.47× fewer parameters than FiLM-based OAT. The FLOATS slim variant further improves parameter efficiency by up to 2.69×.
  • Speed: Training time is reduced by up to 1.43× because FLOAT does not require training on a dense grid of intermediate trade-off values. Furthermore, the absence of extra layers results in up to 1.66× faster inference latency on specialized hardware.
Performance and efficiency comparison of FLOAT and FLOATS against OAT baselines across different datasets and models.

Conclusion

FLOAT provides a hardware-friendly, efficient solution for deploying tunable robust models on edge devices. By shifting the conditioning mechanism from activations to weights and utilizing dual batch normalization, FLOAT achieves superior accuracy-robustness trade-offs with lower computational overhead. Future work may explore automated selection of the tuning parameter $\lambda_n$ based on real-time environmental context or model confidence.

Summary updated

Abstract

Existing models that achieve state-of-the-art (SOTA) performance on both clean and adversarially-perturbed images rely on convolution operations conditioned with feature-wise linear modulation (FiLM) layers. These layers require additional parameters and are hyperparameter sensitive. They significantly increase training time, memory cost, and potential latency which can be costly for resource-limited or real-time applications. In this paper, we present a fast learnable once-for-all adversarial training (FLOAT) algorithm, which instead of the existing FiLM-based conditioning, presents a unique weight conditioned learning that requires no additional layer, thereby incurring no significant increase in parameter count, training time, or network latency compared to standard adversarial training. In particular, we add configurable scaled noise to the weight tensors that enables a trade-off between clean and adversarial performance. Extensive experiments show that FLOAT can yield SOTA performance improving both clean and perturbed image classification by up to ~6% and ~10%, respectively. Moreover, real hardware measurement shows that FLOAT can reduce the training time by up to 1.43× with fewer model parameters of up to 1.47× on iso-hyperparameter settings compared to the FiLM-based alternatives. Additionally, to further improve memory effi ciency we introduce FLOAT sparse (FLOATS), a form of non-iterative model pruning and provide detailed empirical analysis in yielding a three-way accuracy-robustness-complexity trade-off for these new class of pruned conditionally trained models.
BibTeX
@inproceedings{kundu2023float,
  author = {Souvik Kundu and Sairam Sundaresan and Massoud Pedram and Peter A. Beerel},
  title = {{FLOAT: Fast Learnable Once-for-All Adversarial Training for Tunable Trade-off between Accuracy and Robustness}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00238},
}
Figure