Abstract
Deep Neural Networks (DNNs) are vulnerable to adversarial perturbations—small, calculated noises that cause misclassification. Generative defense methods like PixelDefend attempt to "purify" these images using generative models like PixelCNN. However, standard PixelCNNs use a raster scan order that fails to preserve local spatial dependencies. This paper proposes Hilbert-based PixelDefend (HPD), which utilizes space-filling Hilbert curves to maintain spatial proximity during the purification process. Experimental results on CIFAR-10 demonstrate that HPD significantly improves robustness against sophisticated attacks, including Projected Gradient Descent (PGD) and Obfuscated Gradients.
Introduction
Adversarial examples pose a significant threat to the reliability of DNNs in security-critical applications. A common defense strategy is input purification, where a generative model is used to project a potentially adversarial input back onto the manifold of natural images before classification. PixelDefend is a prominent example of this approach, leveraging the density estimation capabilities of PixelCNN.
The effectiveness of PixelCNN-based purification is heavily dependent on the model's ability to capture local pixel correlations. Traditionally, PixelCNN processes images in a raster scan order (left-to-right, top-to-bottom). This 1D serialization often separates pixels that are physically adjacent in 2D space, hindering the model's ability to reconstruct fine-grained local features necessary for effective purification.
Methodology: Hilbert-Based Purification
The core innovation of this work is the replacement of the raster scan with a Hilbert curve scan.
Hilbert Curve Scan
Unlike the raster scan, the Hilbert curve is a space-filling curve that exhibits high spatial proximity. Pixels that are close together in the 2D image plane remain close in the 1D Hilbert sequence. This property allows the autoregressive PixelCNN to better model local dependencies and structural features of the image.
Ensemble Hilbert-based PixelDefend (EHPD)
To further enhance robustness, the authors introduce an ensemble approach. By exploiting the self-similarity and rotational symmetry of Hilbert curves, they generate eight different orientations (rotations and reflections). The final classification is determined by a consensus across these eight purified versions, providing a diverse defense that is harder for attackers to bypass.
Experimental Evaluation
The proposed methods were evaluated on the CIFAR-10 dataset using ResNet50 and VGG architectures. The defense was tested against several adversarial attacks:
- FGSM & PGD: Standard gradient-based attacks.
- C&W: A high-confidence optimization-based attack.
- Obfuscated Gradients: Attacks specifically designed to circumvent defenses that rely on gradient masking or shattering.
Key Results
HPD consistently outperforms the baseline PixelDefend across various attack scenarios. On CIFAR-10 with ResNet50, HPD achieved a robust accuracy of 64% under a strong PGD attack ($\epsilon=16$), compared to 56% for the original PixelDefend.
The most significant improvement was observed against Obfuscated Gradient attacks. While the original PixelDefend's accuracy plummeted to 15%, the ensemble version (EHPD) maintained a robust 52% accuracy. Qualitative analysis via heatmaps confirms that Hilbert-based purification preserves object features more accurately than raster-based methods.
Discussion and Limitations
While HPD offers superior robustness, it inherits the computational overhead of PixelCNN. The autoregressive nature of the generation process (pixel-by-pixel) makes real-time inference challenging. Additionally, Hilbert curves are naturally defined for $2^n \times 2^n$ grids, necessitating padding or resizing for non-standard image dimensions.
Conclusion
Hilbert-based PixelDefend demonstrates that the scanning order in generative purification is a critical factor for adversarial robustness. By aligning the model's processing sequence with the spatial structure of natural images, HPD provides a more effective defense against both standard and specialized adversarial attacks. Future work will focus on integrating more efficient generative architectures to reduce the computational cost of purification.
Summary updated