Trajectory Reconstruction for Tracking Through Occlusion

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2368 papers480 conceptsfrozen 2026-10-04

Standard computer vision systems often follow a "tracking-by-detection" paradigm: find the object in each frame and link the detections. This approach works until the object disappears. When a target passes behind a wall, enters a crowd, or is obscured by motion blur, the visual evidence vanishes, and the tracker typically "breaks," resulting in a lost identity or a fragmented trajectory.

To solve this, researchers are moving toward predictive trajectory reconstruction. Instead of relying solely on what is visible, these systems treat motion as a continuous physical process that can be inferred even when pixels are missing. By modeling the underlying dynamics of objects and their relationships to the scene, these models "inpaint" the missing segments of a path, maintaining identity through total invisibility.

Bridging Gaps with Motion Priors

The most direct way to handle a missing object is to predict where it should be based on where it was. Early robust systems like BeesBook, developed by Franziska Boenisch and colleagues, use forward and backward linear extrapolation to bridge gaps. By calculating the error between where a tracklet ended and where a new one began, the system can merge fragmented trajectories with high precision.

Modern deep learning approaches have evolved this into a "reconstruct-to-embed" strategy. The LGM Tracker by Gaoang Wang and others uses temporal gated convolutions to explicitly inpaint missing trajectory segments. By predicting the full path of an object during an occlusion, the system can compare reconstructed trajectories rather than isolated, fragmented boxes. This allows the tracker to ignore visual appearance entirely if necessary, relying instead on the "signature" of the object's movement.

Tracking Together: The Power of Context

Objects rarely move in total isolation. In crowded scenes or group settings, the motion of visible neighbors provides a powerful proxy for the motion of a hidden target. This "social context" allows a tracker to maintain a lock on an object it cannot see.

The GSM (Graph Similarity Model), proposed by Qiankun Liu and Qi Chu, represents objects and their neighbors as a graph. When an object is occluded, the system estimates its position by looking at the collective movement of its neighbors. Similarly, CoTracker, introduced by Ignacio Rocco and Andrea Vedaldi, tracks thousands of points simultaneously. By exploiting the statistical dependencies between points—"tracking together"—the model can estimate the coordinates of points that have moved outside the field of view or behind obstacles by analyzing the global motion of the visible background.

This principle extends to containment. Wei Liang and Yixin Zhu demonstrated that if an object is "contained" (e.g., a ball in a cup or a person in a car), its trajectory can be perfectly recovered by inheriting the movement patterns of the container, even during long-term occlusions.

Amodal Persistence and Point Propagation

In autonomous driving, knowing that an object exists even when hidden is a safety requirement. This has led to the development of amodal video instance segmentation.

The S-AModal framework by Jasmin Breitenstein and colleagues maintains object identity by tracking discrete points through occluders. When a car passes behind a truck, the system propagates the car's last known spatial coordinates using a point-based motion vector. This shifts the "amodal mask"—the representation of the full object shape—along the predicted path, ensuring the vehicle remains a persistent entity in the world model.

Similarly, TrackIME by Seong Hyeon Park uses instance motion estimation to re-sample new points from a predicted mask when existing points become occluded. This creates a continuous flow of motion information, replacing lost features with visible ones from the same semantic entity to ensure the trajectory never breaks.

Motion as a Supervision Signal

Beyond active tracking, trajectory reconstruction is being used to improve the detectors themselves. In TrajSSL, Philip Jacobson and Mingyu Ding use motion forecasting to "double-check" object detections. If a detector misses a car in a single frame due to sparsity or blur, the motion model generates a "virtual label" at the predicted location.

This highlights the core intuition of the field: temporal consistency is a form of truth. By treating object positions as continuous mathematical functions—as explored by Xiaobai Liu—researchers can enforce physical constraints that bridge the gaps left by imperfect sensors and complex environments.

Papers in this version

Showing 60 of 368 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 500 papers
  2. Version 1 · 2026-10-04 · 368 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_occlusion_robust_tracking_predictive_sta_354383ce,
  title = {occlusion-robust tracking (predictive state estimation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/occlusion-robust-tracking-predictive-state-estimation/rdr_354383ce5ef04a8e8a5152053a4e3c39},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_trajectory_reconstruction_for_tracking_t_354383ce_v1,
  title = {Trajectory Reconstruction for Tracking Through Occlusion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/trajectory-reconstruction-for-tracking-through-occlusion/rdr_354383ce5ef04a8e8a5152053a4e3c39/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}