In computer vision, identifying an object without manual labels often relies on a simple biological intuition: things that move together belong together. This principle, known as Common Fate, allows models to "discover" objects by grouping pixels that share a similar trajectory. However, this reliance on movement creates a fundamental "blind spot." When an object stops moving, it effectively vanishes from the perspective of a motion-based system, merging into the background.
Recent research has focused on bridging this gap, moving from purely motion-driven models to hybrid systems that use motion to discover objects and appearance to maintain them.
The Power and Brittleness of Common Fate
The Gestalt principle of Common Fate is a powerful supervisor for unsupervised learning. As Matthias Tangemann and colleagues demonstrate in Unsupervised Object Learning via Common Fate, we can decompose a scene into individual objects by looking for correlated movement. This allows models to learn object-centric representations without a single human-provided label.
Most of these systems rely on optical flow—the pattern of apparent motion of objects between consecutive frames. By analyzing flow, models like MoTok, developed by Pavel Tokmakov and others in Object Discovery from Motion-Guided Tokens, can tokenize a scene into moving foreground and static background.
The problem, as noted by Charig Yang in Self-supervised Video Object Segmentation by Motion Grouping, is that this creates a "predator-prey" dynamic. Just as a predator may lose sight of a camouflaged animal the moment it stops moving, a motion-based segmentation model loses the object identity when its velocity relative to the background hits zero.
Why Motion Fails for Static Objects
The failure of motion-based systems in static scenarios isn't just a minor error; it is a structural limitation of the signal.
- Camouflage: In Video Camouflaged Object Segmentation, Yuli Zhou shows that even advanced models like SAM2 struggle when movement stops, as the visual boundaries between the object and the environment are intentionally indistinct.
- Slow Camera Movement: Shihao Shen points out in DytanVO that when a camera is stationary or moving slowly, the "residual flow" (the difference between expected and actual motion) becomes too noisy to distinguish a moving car from a parked one.
- Temporal Gaps: If an object is occluded or stops moving mid-sequence, the temporal link is broken. Traditional methods fail to "re-identify" the object when it reappears or starts moving again because they lack a persistent memory of what the object looks like.
Appearance as a Temporal Bridge
To solve this, researchers are developing methods that use motion as a "starter" signal to train appearance-based models. The intuition is: use motion to find the object, then use appearance to keep it.
In Appearance-Based Refinement for Object-Centric Motion Segmentation, Junyu Xie and Weidi Xie propose a refinement stage. Once motion cues identify a high-confidence "anchor frame" where an object is moving clearly, the system extracts an appearance profile. This profile is then propagated through the rest of the video, allowing the model to track the object even when it becomes stationary.
Similarly, Lin Xi and the team behind IMCNet move away from explicit optical flow entirely. Instead, they perform implicit motion compensation at the feature level. By learning an "objectness prior" that persists across frames, the model maintains a consistent mask even when the primary motion cue disappears.
Beyond 2D Pixels: 3D and Submodular Logic
Other researchers are looking beyond 2D image planes to solve the stationary problem. In A Continuous Occlusion Model for Road Scene Understanding, Vikas Dhiman suggests treating objects as 3D ellipsoids. By modeling the physical space, the system can understand that a parked car is still an independent object, even if its velocity is zero, because it occupies a distinct volume that occludes the background.
Finally, some approaches use global optimization to ensure consistency. Tinghuai Wang in Submodular video object proposal selection treats the problem as a facility location problem. Instead of making a frame-by-frame decision based on motion, the system looks at the entire video and selects the most consistent object proposals across time, effectively "filling in" the segments where the object was stationary by looking at the frames where it was moving.
By combining the "free" labels provided by motion with the persistence of appearance and 3D geometry, these models are moving toward a more robust, human-like understanding of objects that doesn't require them to be constantly in motion to be seen.