Abstract
Existing neural networks for computer vision tasks are vulnerable to adversarial attacks: adding imperceptible perturbations to input images can fool these methods into making false predictions. While various defense methods propose image-to-image mapping or adversarial training, they often ignore the fact that natural RGB images are recovered from RAW color filter array captures. In this work, we exploit the RAW data distribution as an empirical prior for adversarial defense. We propose a model-agnostic defensive method that maps input RGB images to Bayer RAW space and back to output RGB using a learned camera image signal processing (ISP) pipeline to eliminate adversarial patterns. Our method acts as an off-the-shelf preprocessing module that generalizes to unseen tasks without additional retraining. Experiments on ImageNet, COCO, and Pascal VOC validate that our method significantly outperforms existing input-transformation defenses.
Introduction
Deep Neural Networks (DNNs) are notoriously fragile when faced with adversarial attacks—small, carefully crafted perturbations that are invisible to humans but catastrophic for model accuracy. Most existing defenses fall into two categories: Adversarial Training (AT), which is computationally expensive and often overfits to specific attacks, and Input Transformation (IT), which attempts to "denoise" images before they reach the model.
However, most IT methods operate entirely in the RGB space, ignoring the fundamental truth that natural images are recovered from RAW sensor data through a complex ISP pipeline. Adversarial noise typically does not follow the statistical properties of real sensor data, such as photon shot noise or optical aberrations. This makes the RAW domain a powerful space for identifying and eliminating artificial patterns.
Methodology
We propose a "Raw-Defense" preprocessing module that acts as a bridge between the adversarial RGB input and the clean RAW-like distribution. The pipeline consists of three main components:
- F Operator (Image-to-RAW Mapping): A learned encoder-decoder network that maps an input RGB image ($I'$) back to its estimated intermediate RAW measurements ($I_W$). It is trained to map both clean and noisy images to the same RAW distribution.
- G Operator (Learned ISP): A pyramidal convolutional neural network (based on PyNet) that reconstructs a high-quality RGB image ($I_G$) from the estimated RAW data. It uses a coarse-to-fine structure to handle different scales of image features.
- S Operator (Conventional ISP): A traditional, non-differentiable software ISP (performing demosaicing, white balance, etc.) to produce a second RGB reconstruction ($I_S$).
The final "defended" image is a weighted sum of the outputs from $G$ and $S$. The inclusion of the non-differentiable S operator is a critical security measure; it breaks the gradient flow, making it significantly harder for an attacker to generate "white-box" adversarial examples that can bypass the defense.
Experimental Results
Our method was validated on standard benchmarks including ImageNet (classification), MS COCO (semantic segmentation), and Pascal VOC (object detection). The $F$ and $G$ operators were trained on the Zurich-Raw-to-RGB dataset.
Key Findings
- Superior Robustness: Against strong PGD attacks on ImageNet, our method maintained ~66% Top-1 accuracy on ResNet-101, while competitors like JPEG compression and Total Variation Minimization (TVM) often dropped below 50%.
- The "Rawer" the Better: Defense performance is highest when the intermediate mapping space is closest to the actual sensor output. Mapping to a "demosaiced" stage instead of "raw capture" leads to a drop in defensive efficacy.
- Real Statistics Matter: The defense fails if trained on synthetic RAW data (e.g., simple downsampling). This proves the method exploits unique statistical cues found in real physical sensors rather than just performing simple blurring.
- Task Generalizability: As a model-agnostic module, Raw-Defense requires no knowledge of the downstream model. A pipeline trained on street scenes successfully defended models for ImageNet classification and Pascal VOC detection without retraining.
Conclusion
By leveraging the natural distribution of RAW sensor data as an empirical prior, Raw-Defense effectively "washes away" adversarial perturbations. This approach provides a robust, task-agnostic, and off-the-shelf defense mechanism that outperforms traditional RGB-based input transformations. Future work will explore using RAW natural image statistics as an unsupervised prior to potentially remove the need for paired training data.
Summary updated