The Violation of Expectation Paradigm for Intuitive Physics

This direction is split into other directions: surprise-based evaluation. Its last version stays readable and citable here.

Version 1 of 144 papers58 conceptsfrozen 2026-10-04

Humans are born with a "common sense" understanding of the physical world. We don't need a degree in classical mechanics to know that a ball shouldn't pass through a solid wall or that an object continues to exist even when it rolls behind a couch. This internal "physics engine" is what researchers call intuitive physics.

While modern AI can generate photorealistic videos, it often fails at these basic rules. To bridge this gap, researchers have turned to developmental psychology, specifically the Violation of Expectation (VoE) paradigm, to measure and build models that truly understand the constraints of reality.

The Infant as a Scientist

In developmental psychology, researchers measure what infants understand by showing them something "impossible"—like a toy disappearing into thin air—and measuring how long they stare at it. If the infant looks longer at the impossible event than a possible one, it suggests they have an internal expectation that was violated.

Researchers like Ronan Riochet and Emmanuel Dupoux adapted this into the IntPhys benchmark. Instead of gaze duration, they measure a model's "surprise." A model demonstrates physical understanding if it assigns a significantly higher error or lower probability to physically impossible videos compared to plausible ones.

Surprise as a Mathematical Metric

How do you turn "surprise" into a number? In the VoE framework, surprise is typically quantified as the discrepancy between a model's internal prediction and the actual observation.

  1. Prediction Error: Models like PLATO, developed by Luis S. Piloto and Peter W. Battaglia, predict the next state of a scene. If an object teleports, the difference between the predicted position and the observed position creates a spike in prediction error.
  2. Negative Log-Likelihood: Kevin A. Smith and colleagues in the ADEPT model use probabilistic physics engines. Surprise is the negative log-likelihood of the visual data given the model's physical predictions. If the data is highly improbable (like a vanishing object), the surprise value sky-rockets.
  3. Latent Divergence: More recent work, such as V-JEPA by Quentin Garrido and Nicolas Ballas, measures surprise in an abstract "latent" space rather than at the pixel level. This allows the model to ignore irrelevant pixel noise and focus on conceptual violations like shape constancy.

As Andrew Ortony and Derek Partridge noted in their foundational 1987 paper, surprise isn't just about failing to predict the future; it's about the violation of passive background assumptions that we only notice when they are broken.

From Pixels to Objects

A recurring theme across this research is that pixel-level prediction is insufficient for physical reasoning. Models that treat the world as a collection of discrete, persisting entities—object-centric models—perform much better on VoE tasks.

For instance, POD-Net, developed by Yilun Du and Kevin A. Smith, learns to discover 3D objects from video without supervision. By enforcing that objects must move consistently and persist through time, the model naturally develops a "surprise" response when those constraints are violated. Similarly, X-VoE by Bo Dai suggests that when a model encounters a violation, it should try to "explain" it by inferring hidden factors (like an invisible barrier) to resolve the surprise.

The Gap in Current Models

Despite the success of specialized models like PLATO, general-purpose Large Language Models (LLMs) and video generators still struggle. Benchmarks like GRASP and PAI-Bench show that even state-of-the-art models like GPT-4o often perform near chance on simple VoE tests.

To fix this, researchers are using VoE signals as a "reward" to align models with reality. Jianhao Yuan and Melissa Hall demonstrated that using a latent world model's surprise score can act as a filter, discarding physically implausible video generations and forcing the model to adhere to the laws of physics.

Extending to Social Intuition

The VoE paradigm isn't limited to physics. It is also being used to evaluate intuitive psychology—the understanding of agents and goals. Benchmarks like BIB (Baby Intuitions Benchmark) and AGENT by Tianmin Shu use the same logic: familiarizing a model with an agent's goal and then measuring surprise when the agent acts irrationally or inefficiently.

By treating "surprise" as a fundamental signal for learning, researchers are moving away from simple pattern matching toward AI that possesses a robust, internal model of how the world—and the agents within it—should behave.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_the_violation_of_expectation_paradigm_fo_b2a56b98,
  title = {The Violation of Expectation Paradigm for Intuitive Physics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-violation-of-expectation-paradigm-for-intuitive-physics/rdr_b2a56b98752a426680bc140a0763a016},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_the_violation_of_expectation_paradigm_fo_b2a56b98_v1,
  title = {The Violation of Expectation Paradigm for Intuitive Physics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-violation-of-expectation-paradigm-for-intuitive-physics/rdr_b2a56b98752a426680bc140a0763a016/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}