BRI3L: A Brightness Illusion Image Dataset for Identification and Localization of Regions of Illusory Perception

Aniket Roy, Anirban Roy, Soma Mitra, Kuntal GhoshICIP 2024 · DOI · OpenReview

TL;DR

This paper introduces BRI3L, the first large-scale dataset for identifying and localizing brightness illusions, addressing the limitations of existing methods that rely on a small number of illusions. They demonstrate that deep learning models can effectively learn to detect and segment these illusions, even generalizing to unseen types.

What Problem Does This Paper Address?

Visual perception is a complex field, and understanding how our brains interpret visual information, especially illusions, is crucial. Brightness illusions, where perceived brightness differs from actual luminance, are a fascinating subset of these phenomena. Current research on these illusions often relies on deterministic filtering methods applied to a limited set of examples, making it difficult to generalize findings. This lack of comprehensive datasets and data-driven approaches hinders progress in understanding the underlying mechanisms of visual perception and developing robust computational models. The authors aim to bridge this gap by creating a large, diverse dataset and evaluating deep learning models for both classifying and localizing these illusory regions.

Methods

The core of this work is the creation of the BRI3L dataset, containing 22,366 images across five types of brightness illusions (Hermann grid, Simultaneous Brightness Contrast (SBC), White illusion, Grid illusion, and Induced Grating illusion). Each image is annotated with a binary mask indicating the illusory region.

For illusion identification (classification), they employed standard Convolutional Neural Networks (CNNs) like ResNet-18, AlexNet, and SqueezeNet.

For illusion localization (segmentation), they utilized the U-Net architecture, a popular choice for image segmentation tasks. A key innovation here is the use of a perception-inspired loss function that combines Mean Squared Error (MSE) with Structural Similarity Index Metric (SSIM) to better capture perceptual similarities.

Additionally, they explored the use of diffusion models for generating novel brightness illusions, a novel application of these generative models.

Datasets

  • BRI3L Dataset: A newly introduced dataset comprising 22,366 images of five types of brightness illusions (Hermann grid, SBC, White illusion, Grid illusion, and Induced Grating illusion). It includes binary segmentation masks for illusory regions.
  • Natural Images: Used for comparative network analysis (e.g., ImageNet dataset).

Key Insights and Ideas

  • The Power of Scale: A large, diverse dataset is crucial for training robust models that can generalize beyond a few well-known illusions.
  • Deep Learning for Perception: CNNs and U-Nets are not only capable of classifying and segmenting illusions but can also learn to generalize to unseen illusion types, suggesting they capture fundamental aspects of visual processing.
  • Perceptual Loss Matters: Incorporating SSIM into the loss function significantly improves illusion localization, highlighting the importance of perceptual similarity metrics in computer vision tasks related to human perception.
  • Diffusion Models for Illusions: Generative models like diffusion models can be used to create novel visual illusions, opening up new avenues for research and creative applications.
  • Low-Level Features are Key: Network analysis (using Grad-CAM) suggests that identifying brightness illusions is primarily a low- and mid-level vision task, with lower layers focusing on boundaries and higher layers on more abstract features.

Key Results

  • Illusion Identification: Achieved 99.56% accuracy using ResNet-18.
  • Illusion Localization: Achieved 84.37% accuracy and 0.75 mIoU using U-Net with the combined MSE+SSIM loss.
  • Generalization: The models demonstrated generalization to unseen illusion types, with varying degrees of success (e.g., performing well on most but struggling with the Hermann grid, which lacks physically present illusory regions).
  • Diffusion Model Performance: Image-to-image diffusion models generated better illusions than text-to-image models, with qualitative rankings of SBC > White illusion > Hermann grid.

Limitations and Future Work

  • Hermann Grid Generalization: The models struggled to generalize to the Hermann grid illusion, likely because the illusory regions are not physically present in the image, making segmentation challenging.
  • Non-Illusion Data Scarcity: The dataset has a significantly smaller number of non-illusion images compared to illusion images, which was addressed through data augmentation.
  • Diffusion Model Output: While diffusion models can generate illusions, the initial output was sometimes misleading, suggesting room for improvement in fine-tuning or training strategies.

Future work could involve exploring more advanced generative models, investigating the role of different network architectures, and further analyzing the learned features to gain deeper insights into the computational basis of visual perception.

Summary updated

Abstract

Visual illusions play a significant role in understanding visual perception. Current methods in understanding and evaluating visual illusions are mostly deterministic filtering based approach and they evaluate on a handful of visual illusions, and the conclusions therefore, are not generic. To this end, we generate a large-scale dataset of 22,366 images (BRI3L: BRightness Illusion Image dataset for Identification and Localization of illusory perception) of the five types of brightness illusions and benchmark the dataset using data-driven neural network based approaches. The dataset contains label information - (1) whether a particular image is illusory/nonillusory, (2) the segmentation mask of the illusory region of the image. Hence, both the classification and segmentation task can be evaluated using this dataset. We follow the standard psychophysical experiments involving human subjects to validate the dataset. To the best of our knowledge, this is the first attempt to develop a dataset of visual illusions and benchmark using data-driven approach for illusion classification and localization. We consider five well-studied types of brightness illusions: 1) Hermann grid, 2) Simultaneous Brightness Contrast, 3) White illusion, 4) Grid illusion, and 5) Induced Grating illusion. Benchmarking on the dataset achieves 99.56% accuracy in illusion identification and 84.37% pixel accuracy in illusion localization. The application of deep learning model, it is shown, also generalizes over unseen brightness illusions like brightness assimilation to contrast transitions. We also test the ability of state-of-theart diffusion models to generate brightness illusions. We have provided all the code, dataset, instructions etc in the github repo: https://github.com/aniket004/BRI3L
BibTeX
@inproceedings{roy2024bri,
  author = {Aniket Roy and Anirban Roy and Soma Mitra and Kuntal Ghosh},
  title = {{BRI3L: A Brightness Illusion Image Dataset for Identification and Localization of Regions of Illusory Perception}},
  booktitle = {ICIP},
  year = {2024},
  doi = {10.1109/icip51287.2024.10647946},
}
Figure