Optimal integration of visual speed across different spatiotemporal frequency channels

Matjaz Jogan, Alan A. StockerNeurIPS 2013 · OpenReview

Human speed perception is surprisingly susceptible to bias. For decades, vision researchers have known that low-contrast objects appear to move slower than high-contrast ones. While Bayesian models—which assume the brain has a "prior" expectation that objects usually move slowly—have successfully explained this contrast effect for simple patterns, they have struggled to account for more complex, real-world stimuli.

Real-world objects contain many different spatial frequencies (textures) and temporal frequencies. Previous models often treated an entire visual stimulus as a single "cue." This paper investigates a more sophisticated question: How does the brain integrate speed information when a stimulus contains multiple different frequency components? Does it simply rely on the strongest signal, or does it mathematically combine them to reach a more reliable estimate?

The Proposed Bayesian Observer Model

To answer this, the authors developed an extended Bayesian observer model that mimics the physiological structure of the visual system. The model is built on three key components:

  1. Multiple Channels: The model decomposes a moving stimulus into multiple independent spatiotemporal frequency channels, similar to how the primary visual cortex processes visual input.
  2. Optimal Integration: Each channel provides its own "likelihood"—a noisy estimate of the object's speed. The model multiplies these independent likelihoods together, along with a "slow-speed prior," to calculate a posterior probability distribution. The peak of this distribution represents the final perceived speed.
  3. Divisive Normalization: The authors also tested a version of the model where these channels interact. In this biologically plausible setup, the response of one channel is divided by the sum of all channel responses, a standard mechanism for how neurons regulate gain and contrast.
Block diagram of the proposed Bayesian observer model for speed perception.

Experimental Design and Stimuli

To validate the model, the researchers conducted a two-alternative forced choice (2AFC) speed discrimination experiment with seven human subjects. Participants were asked to compare the speeds of "simple" gratings (single spatial frequencies at 0.5 c/deg or 1.5 c/deg) and "combined" gratings (two frequencies superimposed) across various contrast levels.

Crucially, the combined stimuli were presented in two specific phase configurations: "peaks-add" and "peaks-subtract."

Illustration of the 'peaks-add' and 'peaks-subtract' combined grating stimuli.

This design was a clever way to test the core hypothesis. If the brain perceives speed based on the overall global contrast of an image, the "peaks-add" stimulus (which has higher peak contrast) would be perceived differently than the "peaks-subtract" stimulus. However, if the brain decomposes the image into independent frequency channels, the relative phase shouldn't matter.

Key Findings

  • Phase Independence: The experiment revealed that perceived speed was entirely independent of the relative phase of the grating components. This is a critical finding because it proves that the visual system decomposes complex motion into separate frequency channels before estimating speed, rather than just reacting to the raw, global image contrast.
  • The "Optimal" Brain: When two frequency components were combined, subjects' discrimination thresholds (their sensitivity to subtle speed changes) decreased. This aligns perfectly with the mathematical predictions of "optimal cue combination." It suggests the brain is statistically efficient, pooling information across different textures to create a more reliable estimate than any single texture could provide alone.
  • Normalization is Necessary: While the standard Bayesian model fit the data well, the version incorporating divisive normalization was significantly better. The normalization model outperformed the non-interaction model with a high degree of statistical significance ($p = 10^{-28}$). This indicates that the "likelihoods" the brain uses are dynamically scaled by the total activity in the visual system, making channel interaction a fundamental part of speed processing.
  • Predictable Perceptual Biases: As predicted by the slow-speed prior, lower contrast led to slower perceived speeds. Interestingly, combined stimuli were perceived as faster than their individual low-contrast components because the combined evidence was stronger, thereby reducing the "pull" of the slow-speed prior.

Limitations and Future Directions

While the study provides a robust framework for understanding complex speed perception, it has limitations. The experiment only utilized two specific spatial frequencies. Real-world "natural" motion contains a much broader, continuous spectrum of frequencies. Additionally, the authors used a local linear approximation for the log-prior, whereas a more complex, non-linear prior might be necessary to explain perception across a wider, naturalistic range of speeds.

Ultimately, this research provides an important stepping stone. By proving that the brain optimally integrates speed across frequency channels using divisive normalization, the authors have laid the groundwork for future models capable of predicting perceived speeds for the messy, multi-textured motion of the real world.

Summary updated

Abstract

How does the human visual system compute the speed of a coherent motion stimulus that contains motion energy in different spatiotemporal frequency bands? Here we propose that perceived speed is the result of optimal integration of speed information from independent spatiotemporal frequency tuned channels. We formalize this hypothesis with a Bayesian observer model that treats the channel activity as independent cues, which are optimally combined with a prior expectation for slow speeds. We test the model against behavioral data from a 2AFC speed discrimination task with which we measured subjects' perceived speed of drifting sinusoidal gratings with different contrasts and spatial frequencies, and of various combinations of these single gratings. We find that perceived speed of the combined stimuli is independent of the relative phase of the underlying grating components, and that the perceptual biases and discrimination thresholds are always smaller for the combined stimuli, supporting the cue combination hypothesis. The proposed Bayesian model fits the data well, accounting for perceptual biases and thresholds of both simple and combined stimuli. Fits are improved if we assume that the channel responses are subject to divisive normalization, which is in line with physiological evidence. Our results provide an important step toward a more complete model of visual motion perception that can predict perceived speeds for stimuli of arbitrary spatial structure.
BibTeX
@inproceedings{jogan2013optimal,
  author = {Matjaz Jogan and Alan A. Stocker},
  title = {{Optimal integration of visual speed across different spatiotemporal frequency channels}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2013},
  url = {https://openreview.net/forum?id=SJbnhvb_Zr},
}
Figure