Causal Asymmetry in Human Motion and Interaction

This direction is merged into another direction: asymmetric relationship modeling (asymmetric social influence). Its last version stays readable and citable here.

Version 1 of 113 papers23 conceptsfrozen 2026-10-04

In traditional computer vision, human motion is often treated as a "bag of features"—a collection of moving joints and pixels that happen to correlate in time. If a leg moves and an arm swings, the model labels the action as "walking." However, this approach ignores the fundamental causal structure of the physical world. Movement is rarely a collection of independent events; it is a directed symphony where one entity initiates and another responds.

A growing body of research suggests that to truly understand and generate realistic motion, we must move beyond symmetric correlations and embrace causal asymmetry. This perspective treats interactions not as two things happening at once, but as a relationship between a "driver" and a "responder."

The Failure of Symmetry

When models treat all moving parts as equal, they often produce "telekinetic" artifacts—objects that begin to float before a hand touches them, or people who react to a punch before it is thrown. These errors occur because the model sees a statistical correlation between two movements but doesn't understand which one causes the other.

Sanath Narayan and Kalpathi R. Ramakrishnan addressed this in A Cause and Effect Analysis of Motion Trajectories for Modeling Actions by applying Granger causality to body parts. They demonstrated that motion is characterized by lead-follow relationships; the movement of a torso might "cause" the subsequent movement of an arm. By modeling these asymmetric dependencies, they could distinguish between actions that look similar but have different mechanical drivers.

Driver and Responder Dynamics

In human-object interaction (HOI), the causal direction is usually clear: the human is the active agent, and the object is the passive recipient. Yet, many predictive frameworks still treat human and object trajectories as a joint, symmetric distribution.

To fix this, Lin Wu and colleagues introduced HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion, which explicitly frames HOI as a driver-responder system. By ensuring that objects only move in response to human contact, they eliminated physically impossible "pre-emptive" object motion. Similarly, Xiaotong Lin and the team behind CoopDiff used this asymmetry to decouple motion modeling. They injected human motion features into the object's prediction path, treating the human as a conditional input that dictates the object's physical trajectory.

Social Actor Reactor Relationships

This causal framework extends beyond physics into social dynamics. In human-human interaction (HHI), one person often initiates an action (the actor) while the other responds (the reactor).

Liang Xu and collaborators have championed this "actor-reactor" framing through the Inter-X dataset and the ReGenNet framework. By labeling participants as initiators or respondents, they allow generative models to focus on synthesizing reactive behaviors that are semantically contingent on the initiator's movements. This moves AI away from generating two independent people and toward generating a coherent social exchange.

Why Asymmetry Works

Why is this directional modeling so much more effective than simple joint modeling? The answer lies in how humans generalize.

Bonan Zhao and Neil R. Bramley found in Building Object-based Causal Programs for Human-like Generalization that humans have a strong inductive bias toward the features of the agent rather than the recipient. We associate causal laws with the entity initiating the action. When a model mirrors this bias, it doesn't just memorize trajectories; it learns the underlying "rules" of the interaction.

This logic also applies to the environment. In Context-aware Human Motion Prediction, Albert Pumarola and Enric Corona used semantic graphs to model how static objects (like chairs or walls) constrain human movement. This interaction is the ultimate asymmetry: the environment dictates the person's path, but the person (usually) does not move the wall. By acknowledging that influence only flows one way, models become significantly more stable and accurate.

By shifting from "what is moving together" to "what is driving what," researchers are creating AI that understands the invisible threads of influence that govern the physical and social world.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_causal_asymmetry_in_human_motion_and_int_e99478c7,
  title = {Causal Asymmetry in Human Motion and Interaction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-asymmetry-in-human-motion-and-interaction/rdr_e99478c7a813488b9f73eb6e6eab9e7d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_causal_asymmetry_in_human_motion_and_int_e99478c7_v1,
  title = {Causal Asymmetry in Human Motion and Interaction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-asymmetry-in-human-motion-and-interaction/rdr_e99478c7a813488b9f73eb6e6eab9e7d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}