TL;DR
This paper introduces a novel defense strategy to make Under-Display Camera (UDC) image restoration models more resilient to adversarial attacks. By combining diffusion-based purification with efficient fine-tuning, the authors significantly improve the robustness of these models, demonstrating substantial PSNR gains even under strong attacks.
What Problem Does This Paper Address?
Modern smartphones and other devices increasingly feature Under-Display Cameras (UDCs) to achieve seamless, notch-less displays. However, placing cameras behind display panels introduces significant image quality issues like blur, flare, and color distortion. Deep learning models have shown great promise in restoring these UDC images. Yet, a critical vulnerability has emerged: these models are susceptible to adversarial attacks. These attacks, which involve subtly altering input images with imperceptible perturbations, can drastically degrade the quality of restored UDC images, posing a risk to device functionality and user experience. This paper addresses the underexplored problem of adversarial robustness in UDC image restoration, aiming to fortify these models against such attacks.
Methods
The authors propose a two-stage defense framework:
- Adversarial Purification: This stage employs diffusion models (specifically, inspired by DDPM) to "purify" adversarial perturbations from UDC images. The process involves a forward diffusion step to add noise, bringing both clean and adversarial images closer in distribution, followed by a reverse denoising process to reconstruct a cleaner image. This aims to neutralize the adversarial noise without significantly altering the original image content.
- Efficient Fine-Tuning: After purification, the UDC image restoration model is fine-tuned. This step is crucial because the purification process might introduce slight domain shifts. The fine-tuning uses a supervised loss function (a combination of L2 reconstruction and perceptual loss) to adapt the model to the purified images, enhancing its resilience without requiring adversarial training data. This fine-tuning is designed to be efficient and preserve performance on clean images.
The paper also includes a comprehensive evaluation of existing UDC image restoration models against various white-box (PGD, C&W) and black-box (SimBA, Square Attack) adversarial attacks to establish a baseline for their defense.
Datasets
The study utilizes a synthesized dataset for training and evaluation. This dataset is generated using:
- Nine existing ZTE Axon 20 phone datasets.
- Synthesized Point Spread Functions (PSFs).
- One real-scene PSF.
- Data from the 2016 training images and 360 testing images collected from the HDRI Haven dataset.
The training set for the fine-tuning stage is derived from this synthetic dataset, using purified images generated via the DDPM process with varying diffusion steps.
Key Insights and Ideas
- UDC Models are Vulnerable: The initial robustness evaluation reveals that even state-of-the-art UDC image restoration models suffer severe performance degradation under various adversarial attacks, with PSNR dropping significantly (e.g., DISCNet's PSNR from 35.24 to 15.16 under C&W attack).
- Purification as a Preprocessing Step: Diffusion models, known for their generative capabilities, can effectively denoise adversarial perturbations, acting as a powerful preprocessing step before image restoration.
- Fine-tuning is Key for Adaptation: Simply purifying images isn't enough. A subsequent fine-tuning step is necessary to adapt the restoration model to the purified data, ensuring it can leverage the denoised input effectively without compromising clean image performance.
- Efficiency Matters: The proposed fine-tuning approach avoids traditional, computationally expensive adversarial training, offering a more efficient path to robustness.
- Trade-offs Exist: While adversarial training can improve robustness, it often leads to a performance drop on clean images. The proposed method aims to balance robustness with clean image quality.
Key Results
- Significant Robustness Improvement: The proposed defense framework demonstrates marked improvements in resilience. For instance, after purification and fine-tuning, DISCNet's PSNR rebounds to 32.17 under PGD attack (compared to 30.17 without defense), and UFormer achieves 19.71 PSNR under LPIPS-guided attacks (vs. 17.38 baseline).
- Model-Specific Performance: DISCNet shows excellent performance on clean images but is sensitive to C&W attacks. UFormer, while having lower clean image performance, exhibits better overall robustness across various attacks.
- Attack Objective Impact: The study found that MSE objectives tend to cause more degradation than LPIPS objectives in PGD attacks for most models.
- Visual Improvements: Qualitative results show that the proposed defense strategy significantly reduces artifacts, glare, and color blocks introduced by adversarial attacks, leading to visually superior restored images.
Limitations and Future Work
The authors acknowledge several limitations:
- Computational Efficiency: The multi-step iteration of the diffusion model for purification leads to low computational efficiency, making it difficult to meet real-time requirements.
- Extreme Disturbances: The defense effectiveness decreases in extreme disturbance or dynamic attack scenarios.
- Generalization: Validation on synthetic data has limited generalization to real-world complex scenes, especially those with dynamic lighting.
Future research directions include:
- Designing dynamic detection modules for adaptive defense.
- Utilizing self-supervised learning to reduce reliance on paired data.
- Integrating multimodal sensor data to enhance system robustness.
- Improving computational efficiency for real-time applications.
Summary updated