Enhancing Robustness of Neural Networks through Fourier Stabilization

Netanel Raviv, Aidan Kelley, Minzhe Guo, Yevgeniy Vorobeychik

Abstract

Deep neural networks (DNNs) are highly susceptible to adversarial evasion attacks, where small, intentional perturbations to inputs cause misclassification. This vulnerability is particularly critical in security domains like malware detection. While adversarial training (AT) is the standard defense, its high computational cost and negative impact on clean accuracy limit its utility. This paper presents Fourier stabilization, a novel post-training defense specifically designed for models with binary inputs. By utilizing Fourier analysis to identify and implement more stable neuron weights, the method provides a fine-grained, efficient way to harden pre-trained models. Experimental results across multiple security-centric datasets show that Fourier stabilization can dramatically increase robust accuracy with negligible loss in standard performance.

Introduction

In security-critical applications, the "binary feature" representation is ubiquitous. For instance, malware detection often relies on the presence or absence of specific API calls or file structure elements. Despite their importance, DNNs operating on these features remain fragile. Adversaries can perform "evasion attacks" by flipping a few binary bits to bypass detection.

Existing defenses like Adversarial Training require retraining the entire model on perturbed data, which is often impractical due to resource constraints or lack of access to original training data. This research addresses these challenges by proposing a post-training optimization that targets individual neurons, allowing for a controlled trade-off between robustness and accuracy.

Methodology: Fourier Stabilization

The core innovation lies in treating individual neurons as linear classifiers over binary inputs $\{-1, 1\}^n$. The authors leverage the mathematical framework of Fourier analysis of Boolean functions.

1. Robust Neuron Optimization

Any function mapping binary inputs to real numbers can be represented as a linear combination of Fourier basis functions. The authors define a "robust" neuron as one that maximizes the average distance of all possible inputs from its decision boundary. They prove that the most robust weights for such a neuron are simply the signs of its first-order Fourier coefficients (Chow parameters).

2. Selective Stabilization and Greedy Algorithms

Replacing all neuron weights with their Fourier-stabilized versions can lead to significant accuracy drops. To mitigate this, the authors frame a combinatorial optimization problem: selecting the optimal subset of neurons to stabilize to maximize robustness while maintaining a user-defined accuracy threshold ($\beta$). They propose two greedy heuristics:

  • GMB (Greedy Marginal Benefit): Selects neurons based on their individual contribution to robustness.
  • GMBC (Greedy Marginal Benefit per Unit Cost): Factors in the accuracy "cost" of stabilizing each neuron.

Experimental Setup

The approach was evaluated on three datasets where binary features are standard:

  • PDFRate: Malware detection based on 135 PDF metadata and content features.
  • Hidost: A larger PDF malware dataset extracting 961 features from file logical structures.
  • Hate Speech: A dataset of Gab posts using a 200-feature bag-of-words representation.

Key Results and Discussion

Fourier stabilization demonstrated a superior robustness-accuracy trade-off compared to baseline models.

Figure 2: Accuracy vs. perturbation magnitude (ε) for various datasets and attacks (Black Box and JSMA) using the GMB algorithm. Different colored lines represent different clean accuracy thresholds (β).

As shown in the figures above, the defense significantly shifts the accuracy curves to the right, meaning the models maintain high accuracy even under larger adversarial perturbations (higher $\epsilon$ values).

  • Significant Robustness Gains: On the Hidost dataset, robust accuracy increased by 60% while maintaining a clean accuracy of 99%. On PDFRate, robust accuracy jumped from near 0% to 70% with minimal clean accuracy loss.
  • Additive Defense: The method is complementary to existing techniques. Applying Fourier stabilization to a model that had already undergone adversarial training increased its robust accuracy from 20% to 80% at high perturbation levels.
  • Binarization Insights: The stabilized neurons result in weights of $\{-1, 1\}$. This provides a theoretical link to Binarized Neural Networks (BNNs), suggesting that their inherent robustness can be explained and optimized through Fourier analysis.

Conclusion and Future Work

Fourier stabilization offers a powerful, efficient, and additive defense for binary-input neural networks. It allows practitioners to "harden" existing models after training, which is vital for real-world security deployments.

Future research will focus on extending this framework beyond the first hidden layer and adapting the mathematical tools to handle continuous data distributions. Additionally, exploring the impact of non-uniform input distributions could further refine the theoretical bounds and empirical performance of the stabilization process.

Summary updated

BibTeX
@inproceedings{raviv2021enhancing,
  author = {Netanel Raviv and Aidan Kelley and Minzhe Guo and Yevgeniy Vorobeychik},
  title = {{Enhancing Robustness of Neural Networks through Fourier Stabilization}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
}
Figure