On the role of spatial phase and phase correlation in vision, illusion, and cognition

Evgeny Gladilin, Roland EilsFrontiers Comput. Neurosci 2015 · DOI · OpenReview

TL;DR

This paper proposes that the visual system, through evolution, has developed mechanisms based on spatial phase information and phase correlation to efficiently process images, detect motion, and recognize patterns. The authors demonstrate how this phase-based approach can explain various visual illusions and suggest it forms a fundamental computational principle for visual cognition.

What Problem Does This Paper Address?

The human visual system is remarkably adept at interpreting complex scenes, recognizing objects, and tracking motion. While amplitude (intensity) carries information about image brightness, the role of spatial phase (information about edges and structures) has been a subject of debate. Despite strong phenomenological evidence suggesting phase is crucial for recognizing image features and even generating illusions, its precise computational and neural underpinnings remain unclear. This paper aims to bridge this gap by analyzing the role of spatial phase in computational vision, exploring its contribution to visual illusions, and proposing a unified, phase-based mechanism for pattern recognition and motion detection that could be implemented in biological vision.

Methods

The core of the paper's methodology lies in extending and applying the Fourier Phase Correlation (PC) technique. This method, originally developed for image registration, works by analyzing the phase difference between the Fourier transforms of two images.

  • Fourier Analysis: Images are transformed into the frequency domain, where they are represented by amplitude and phase components.
  • Phase Correlation: The paper leverages the property that a spatial shift in an image results in a linear phase shift in its Fourier transform. Phase correlation essentially measures the similarity of phase information between two images, making it robust to changes in illumination and contrast.
  • Amplitude-Normalization Phase-Only Reconstruction: To highlight the importance of phase, the authors demonstrate reconstructing images using only their phase information. This process acts as an edge-enhancer, similar to Marr's "Primal Sketch."
  • Saccades-Enhanced Phase Correlation: To address challenges in noisy or complex scenes, the authors propose an iterative approach where multiple saccadic eye movements (simulated by sampling different parts of the image) are used to build up a more robust phase correlation. This method aims to isolate the target pattern from background noise.

Datasets

The paper does not introduce new datasets. Instead, it uses:

  • Standard Images: Examples like the "Lenna" image are used to illustrate the effects of amplitude and phase distortion.
  • Illustrative Stimuli: Various visual illusions (stereograms, rotating snakes, anomalous motion, virtual tilt, completion illusions) and synthetic patterns (smiley faces, geometric shapes) are used as examples to demonstrate the principles of phase correlation and its explanatory power for perceptual phenomena.

Key Insights and Ideas

  • Phase is King for Structure: The paper strongly argues that spatial phase, not amplitude, is the primary carrier of structural information (like edges) in images. Reconstructing an image from phase alone yields a recognizable sketch, while phase-less reconstruction is largely unintelligible.
  • Unified Mechanism for Vision: The authors propose that phase correlation, a computational technique, could be the underlying neural mechanism for core visual functions like motion detection and pattern recognition. This suggests a parsimonious evolutionary path for visual processing.
  • Illusions as Phase Artifacts: Many visual illusions, particularly those involving apparent motion and depth, can be explained by the brain misinterpreting coherently phase-shifted patterns. The way our eyes move (saccades) also plays a crucial role in how these phase shifts are perceived.
  • Saccades as a Computational Strategy: The proposed "saccades-enhanced phase correlation" offers a computational explanation for how the visual system can overcome noise and ambiguity by iteratively sampling the scene, mimicking eye movements.
  • Evolutionary Advantage of Phase: The paper posits that the importance of phase for invariant feature detection (independent of position, scale, etc.) provided a significant evolutionary advantage, driving the development of phase-based neural processing.

Key Results

  • Phase-Only Reconstruction: Demonstrates that phase information alone is sufficient to generate a "Primal Sketch" of an image, highlighting edges and structures.
  • Phase Correlation for Motion and Recognition: Shows that phase correlation can accurately detect image translations and is robust to noise and structural distortions, making it a powerful tool for pattern recognition.
  • Explanation of Illusions: The framework provides plausible explanations for various visual illusions, linking them to specific phase relationships in stimuli and the role of saccadic eye movements. For instance, the "Rotating Snake" illusion is explained by phase advances at high contrast and saccadic sampling.
  • Saccadic Strategy Effectiveness: The "saccades-enhanced phase correlation" method shows improved recognition scores in noisy or ambiguous scenarios, suggesting a biologically plausible strategy for dealing with complex visual input.

Limitations and Future Work

  • Theoretical Framework: While the paper presents a strong theoretical argument and computational demonstrations, it acknowledges that direct neurophysiological evidence for phase correlation as a neural mechanism is still developing.
  • Amplitude's Role: The paper focuses heavily on phase but notes that amplitude also plays a role, particularly in peripheral vision and for textural images. Future work should explore how phase and amplitude are integrated.
  • Experimental Validation: The authors explicitly call for experimental validation of their predictions, such as the dependency of saccade strategies on image properties and the singular neural activity patterns in the association cortex for recognizing similar stimuli.
  • Generalization to Other Cognitive Functions: The paper speculates that phase correlation might extend beyond vision to other cognitive functions, a hypothesis that warrants further investigation.

Summary updated

Abstract

Numerous findings indicate that spatial phase bears an important cognitive information. Distortion of phase affects topology of edge structures and makes images unrecognizable. In turn, appropriately phase-structured patterns give rise to various illusions of virtual image content and apparent motion. Despite a large body of phenomenological evidence not much is known yet about the role of phase information in neural mechanisms of visual perception and cognition. Here, we are concerned with analysis of the role of spatial phase in computational and biological vision, emergence of visual illusions and pattern recognition. We hypothesize that fundamental importance of phase information for invariant retrieval of structural image features and motion detection promoted development of phase-based mechanisms of neural image processing in course of evolution of biological vision. Using an extension of Fourier phase correlation technique, we show that the core functions of visual system such as motion detection and pattern recognition can be facilitated by the same basic mechanism. Our analysis suggests that emergence of visual illusions can be attributed to presence of coherently phase-shifted repetitive patterns as well as the effects of acuity compensation by saccadic eye movements. We speculate that biological vision relies on perceptual mechanisms effectively similar to phase correlation, and predict neural features of visual pattern (dis)similarity that can be used for exper...
BibTeX
@article{gladilin2015role,
  author = {Evgeny Gladilin and Roland Eils},
  title = {{On the role of spatial phase and phase correlation in vision, illusion, and cognition}},
  journal = {Frontiers Comput. Neurosci.},
  year = {2015},
  doi = {10.3389/fncom.2015.00045},
}
Figure