Non-uniform Illumination Attack for Fooling Convolutional Neural Networks

Akshay Jain, Shiv Ram Dubey, Satish Kumar Singh, Kc Santosh, Bidyut Baran Chaudhuri

TL;DR

Researchers have developed a novel "Non-Uniform Illumination" (NUI) attack that fools state-of-the-art Convolutional Neural Networks (CNNs) by subtly altering image brightness using spatial masks. While these changes are nearly invisible or easily ignored by humans, they cause a massive drop in AI accuracy—a vulnerability the authors successfully mitigate by using these same attacks as a form of "defensive" data augmentation.

What Problem Does This Paper Address?

Convolutional Neural Networks (CNNs) are the workhorses of modern computer vision, powering everything from medical imaging to self-driving cars. However, they are notoriously brittle. Most "adversarial attacks"—techniques designed to trick AI—rely on calculating complex mathematical gradients or require deep knowledge of the specific model being targeted.

This paper tackles a more practical, real-world challenge: illumination variance. In the real world, light isn't uniform; shadows fall across faces, and sunlight glares off car hoods. The authors ask: Can we fool a CNN using only simple, data-independent changes to lighting? If so, how do we make these models robust enough to handle the messy lighting of the physical world?

Methods

The authors introduce the Non-Uniform Illumination (NUI) attack. Unlike traditional attacks that might add "noise" to every pixel, the NUI attack uses 12 specific spatial masks to vary brightness and darkness across an image.

  • The Attack Formula: The attacked image is created by taking the original image ($I$) and adding a weighted mask: $I_{attacked} = I + (Mask \times k)$. The variable $k$ controls the intensity of the effect.
  • The Masks: The 12 masks target different spatial regions. Some focus on corners or sides, while others use non-linear transformations to create circular patterns (Masks 6-8) or vertical/horizontal stripes of light and shadow (Masks 9-10).
  • Data-Independence: Crucially, this attack is "black-box." It doesn't need to know the model's architecture or its weights to generate a deceptive image.
  • The Defense: To counter this, the researchers propose NUI Augmentation. They retrain the models by including NUI-perturbed images in 80% of the training set, teaching the network to recognize the underlying object regardless of the lighting mask applied.

Datasets

The study evaluated the attack and defense across three widely recognized benchmarks:

  1. CIFAR10: 60,000 small images across 10 categories.
  2. TinyImageNet: A subset of the massive ImageNet dataset, featuring 200 categories.
  3. CalTech256: A complex dataset with 257 object categories and high inter-class similarity.

They tested these against popular architectures including VGG16/19, ResNet18, MobileNetV3-small, and InceptionV3.

Key Insights and Ideas

  • Spatial Complexity Matters: Simple corner-based lighting changes (Masks 1-4) caused some errors, but the real "killers" were patterned masks. Masks 9 and 10 (vertical and horizontal lines) and Mask 6 (severe circular perturbation) were the most effective at tanking model performance.
  • The "Human-AI Gap": The t-SNE visualizations (which map how the AI "sees" data clusters) showed that NUI attacks cause different classes to overlap in the model's feature space. To a human, a dog in a shadow is still a dog; to the AI, the shadow changes the mathematical features so much that it begins to look like a cat or a ship.
  • Generalizability: Because the attack is data-independent, a mask that fools a ResNet model on CIFAR10 is likely to also fool a VGG model on TinyImageNet. This suggests that CNNs share a fundamental weakness in how they process spatial light distribution.

Key Results

The results highlight just how vulnerable standard models are:

  • Accuracy Plunge: On the CIFAR10 dataset, VGG16's accuracy dropped by 72.29% when subjected to Mask 6. Even the robust InceptionV3 saw a 68.02% decline under the same mask.
  • Successful Defense: After applying the proposed NUI-augmented training, the models didn't just recover; they often became more resilient than the originals. For example, VGG16 on TinyImageNet showed a staggering 675.8% increase in accuracy on NUI-attacked test sets compared to the undefended version.
  • Preserving Clean Accuracy: Importantly, training on these "attacked" images did not degrade the models' performance on standard, clean images.

Limitations and Future Work

While effective, the authors note that these masks are currently fixed and static. A more sophisticated version of this research might involve dynamic masks that adapt to the specific geometry of an object in an image.

The authors also briefly explored the effect of NUI attacks on specific color channels (Red, Green, and Blue), finding that perturbing combinations of channels (like Red and Blue) had a higher impact than single channels. They suggest that future work should dive deeper into how multi-channel illumination attacks can be used to further stress-test computer vision systems.

Summary updated

Abstract

Convolutional neural networks (CNNs) have made remarkable strides; however, they remain susceptible to vulnerabilities, particularly to image perturbations that humans can easily recognize. This weakness, often termed as “attacks,” underscores the limited robustness of CNNs and the need for research into fortifying their resistance against such manipulations. This study introduces a novel nonuniform illumination (NUI) attack technique, where images are subtly altered using varying NUI masks. Extensive experiments are conducted on widely accepted datasets including CIFAR10, TinyImageNet, CalTech256, and NWPU-RESISC45 focusing on image classification with 12 different NUI masks. The resilience of VGG, ResNet, MobilenetV3-small, InceptionV3, and EfficientNet_b0 models against NUI attacks are evaluated. Our results show a substantial decline in the CNN models’ classification accuracy when subjected to NUI attacks, due to changes in the image pixel value distribution, indicating their vulnerability under NUI. To mitigate this, a defense strategy is proposed, including NUI-attacked images, generated through the new NUI transformation, into the training set. The results demonstrate a significant enhancement in CNN model performance when confronted with perturbed images affected by NUI attacks. This strategy seeks to bolster CNN models’ resilience against NUI attacks. A comparative study with other attack techniques shows the effectiveness of the NUI attack and defense technique.11The code is available at https://github.com/Akshayjain97/Non-Uniform_Illumination
BibTeX
@article{jain2025non,
  author = {Akshay Jain and Shiv Ram Dubey and Satish Kumar Singh and KC Santosh and Bidyut Baran Chaudhuri},
  title = {{Non-uniform Illumination Attack for Fooling Convolutional Neural Networks}},
  journal = {IEEE Transactions on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=uVcLhYfuf2},
}
Figure