Abstract
Motion can only be defined relative to a reference frame; yet it remains unclear which reference frame guides perception. A century of psychophysical studies has produced conflicting evidence: retinotopic, egocentric, world-centric, or even object-centric. We introduce a hierarchical Bayesian model mapping retinal velocities to perceived velocities. Our model mirrors the structure in the world, in which visual elements move within causally connected reference frames. Friction renders velocities in these reference frames mostly stationary, formalized by an additional delta component (at zero) in the prior. Inverting this model automatically segments visual inputs into groups, groups into supergroups, progressively inferring structured reference frames and "perceives" motion in the appropriate reference frame. Critical model predictions are supported by two experiments, and fitting our model to the data allows us to infer the subjective set of reference frames used by individual observers. Our model provides a quantitative normative justification for key Gestalt principles providing inspiration for building better models of visual processing in general.
Introduction
Motion is never absolute; it is always relative to a reference frame. When you watch a passenger walk down the aisle of a moving train, do you perceive their speed relative to the carriage, the tracks, or your own eyes? For over a century, psychophysicists have struggled to pin down which reference frame the brain uses. Evidence is conflicting: sometimes perception seems tied to the retina (retinotopic), sometimes to the body (egocentric), and sometimes to other objects in the scene (object-centric).
The famous "Johansson illusion"—where the perceived path of a dot is distorted by the motion of nearby dots—highlights that our brain doesn't just use one frame; it somehow combines them. This paper addresses the lack of a unified, normative framework that explains how the brain chooses these frames and how it "chunks" visual elements into coherent groups, providing a mathematical basis for Gestalt principles like "common fate."
Methods: Hierarchical Causal Inference
The researchers developed a hierarchical Bayesian model based on causal inference to explain how the brain navigates the complex geometry of moving scenes.
- The Generative Model: The world is modeled as a hierarchy of nested reference frames (e.g., dots to groups to supergroups to world). Each element's velocity is the sum of its parent's velocity and its own relative velocity.
- The "Friction" Prior: A key innovation is the mixture prior for relative velocity. It combines a standard Gaussian (the "slow-speed" prior) with a Dirac delta function at zero. This delta component formalizes the physical intuition of friction: objects are most likely to be exactly stationary relative to their immediate reference frame.
- Inference: To "perceive" motion, the model inverts this process. It calculates the probability of different hierarchical structures (who belongs to which group) and the velocities within them.
Key Insights: Segmentation and Stationarity
The model's ability to explain human perception stems from several core principles:
- Stationarity is the Default: The "delta at zero" in the prior is essential. Without it, the model cannot explain why we tend to see objects as rigid groups. This provides a mathematical justification for the Gestalt principle of "common fate."
- Automatic Segmentation: The model doesn't need to be told which dots form an object. By performing causal inference, it automatically decides whether to integrate (treat dots as one group) or segment (treat them as independent) based on how similar their velocities are.
- Perception as Sampling: Human responses are best explained by posterior sampling—the brain effectively "picks" a sample from its internal probability distribution rather than just reporting the average. This explains the high variability and occasional bimodal responses seen in observers.
Key Results
The authors validated the model through two psychophysical experiments involving 14 naive human observers. Participants reported the perceived direction of a "center" patch of dots surrounded by other moving patches across thousands of trials.
- High Predictive Power: The model explained 92–96% of the variance in human reports across different observers.
- Explaining Illusions: The model successfully replicated the Johansson illusion, showing that the bias in perceived motion is a rational consequence of the brain's preference for stationary relative motion.
- Structure Discovery: Fitting the model allowed the researchers to "read out" the subjective hierarchy each observer used. They found that most observers rely on a mixture of different causal structures rather than a single fixed one.
- Local vs. Global Frames: Experiment 2 showed that once the brain segments a target from a global frame, that global frame no longer influences the target's perceived motion. Perception becomes "local" within the inferred hierarchy.
Discussion and Future Work
While the model provides a powerful normative framework, several challenges remain. The number of possible causal structures grows factorially with the number of objects, suggesting the brain must use heuristics for complex real-world scenes. Furthermore, while the authors suggest that "surround-suppressed" neurons in visual area MT might represent these relative velocities, the specific neural architecture for performing hierarchical causal inference remains unknown. Future work will need to incorporate speed perception and active vision signals (eye movements and vestibular inputs) to create a truly comprehensive world-centric model.
Summary updated