Perceptual Dominance in Brief Presentations of Mixed Images: Human Perception vs. Deep Neural Networks

Liron Z. Gruber, Aia Haruvi, Ronen Basri, Michal IraniFrontiers Comput. Neurosci 2018 · DOI · OpenReview

Abstract

When presented with a mixture of two images, both humans and Deep Neural Networks (DNNs) tend to "choose" one dominant image rather than perceiving a blend. This study identifies specific image features, such as high-frequency gradients, that predict which image will win this perceptual competition. Furthermore, it demonstrates that injecting stochastic noise into DNNs makes their behavior significantly more human-like, suggesting that internal noise is a critical component of biological visual processing.

Introduction

The human visual system is constantly bombarded with more information than it can process. To manage this overflow, the brain performs a "winner-take-all" operation known as perceptual dominance, where prominent inputs are selected and others are suppressed. While Deep Neural Networks (DNNs) have become the standard for modeling the ventral stream of the human visual system, it remains unclear whether they resolve visual ambiguity in the same way biological systems do. This study investigates whether DNNs share this perceptual dominance and identifies the specific image parameters—such as contrast, colorfulness, or texture—that drive these decisions.

Methodology

The researchers compared human performance against two popular pre-trained DNN architectures: VGG19 and ResNet. The study utilized the ImageNet validation set, selecting 180 images to create 90 unique pairs for mixing experiments.

Image Morphing

Two types of "mixed" images were generated:

  1. 50/50 Mix: A simple pixel-wise average of two images.
  2. Phase/Magnitude (phs/mag) Mix: A complex mixture where the phase information of one image is combined with the magnitude information of another in the frequency domain.
Overview of image mixing techniques: 50/50 pixel-wise averaging and phase-magnitude frequency domain swaps.

Experimental Design

  • Human Trials: 600 participants viewed these mixtures for 100ms—a duration brief enough to prevent eye movement and re-sampling—and provided free-text reports of their perception.
  • DNN Classification: To determine a DNN's "perception," the authors analyzed the top $N$ categories in the output layer. If the network's top guesses matched one original image but not the other, that image was considered the "winner."
  • Stochasticity: To better replicate human inconsistency, researchers experimented with injecting Gaussian noise into the DNN's final layers.
  • Predictive Modeling: A Generalized Linear Model (GLM) was employed to test which of 12 image parameters (e.g., luminance, colorfulness, gradient) best predicted the dominant image.

Key Results and Insights

The "One Image" Tendency

Both humans and DNNs are naturally inclined to suppress one signal in favor of another. In 50/50 mixtures, humans reported seeing only one image in approximately 70% of trials. DNNs showed a similar preference for a single "winner," rather than identifying both components of the blend.

Phase Dominance and Stochasticity

In phase/magnitude mixtures, humans almost exclusively perceived the image that provided the phase information. While DNNs were sensitive to phase, they were significantly less so than humans, often failing to recognize either image in these specialized mixtures. However, by adding a specific level of noise to the DNN's decision-making layer, the researchers aligned the DNN's recognition and failure rates almost perfectly with human data.

Shared Predictive Features

The image gradient—a measure of high-frequency edge information—was the strongest predictor of dominance for both systems. If one image possessed sharper, more distinct edges, it was highly likely to be the perceived winner. Using the GLM, researchers predicted the winning image with 77.2% accuracy for DNNs and 74.0% for humans.

Beyond gradients, low frequencies served as a strong secondary predictor for both. Interestingly, while colorfulness and saturation influenced DNN choices, they had almost no predictive power for human choices in this brief-exposure context.

Discussion and Future Work

Analysis of correlations throughout the DNN layers revealed that the decision of which image wins is not instantaneous. While a slight preference exists in early layers, the dominance of the winning image increases dramatically in the final three fully-connected layers.

A primary limitation of this study is the use of feed-forward networks. The human brain relies heavily on recurrent (feedback) connections to resolve visual competition over time. Even a 100ms exposure allows for some internal recurrence that a single pass through a network like VGG19 cannot capture. Future work should focus on incorporating stochasticity as a fundamental feature of DNNs and training networks specifically on human-perceived winners to better align artificial intuition with human visual priorities.

Summary updated

Abstract

Visual perception involves continuously choosing the most prominent inputs while suppressing others. Neuroscientists induce visual competitions in various ways to study why and how the brain makes choices of what to perceive. Recently deep neural networks (DNNs) have been used as models of the ventral stream of the visual system, due to similarities in both accuracy and hierarchy of feature representation. In this study we created non-dynamic visual competitions for humans by briefly presenting mixtures of two images. We then tested feed-forward DNNs with similar mixtures and examined their behavior. We found that both humans and DNNs tend to perceive only one image when presented with a mixture of two. We revealed image parameters which predict this perceptual dominance and compared their predictability for the two visual systems. Our findings can be used to both improve DNNs as models, as well as potentially improve their performance by imitating biological behaviors.
BibTeX
@article{gruber2018perceptual,
  author = {Liron Z. Gruber and Aia Haruvi and Ronen Basri and Michal Irani},
  title = {{Perceptual Dominance in Brief Presentations of Mixed Images: Human Perception vs. Deep Neural Networks}},
  journal = {Frontiers Comput. Neurosci.},
  year = {2018},
  doi = {10.3389/fncom.2018.00057},
}
Figure