GateTrace: Where Do Crowd Crossings Go Missing Under Occlusion?

Ao He, Qianchen Yu, Vivian W. H. WongarXiv 2026 · arXiv

TL;DR

GateTrace is a diagnostic framework that pinpoints exactly why and where crowd-counting systems fail under occlusion. By tracing individual crossing events from ground truth to final report, it reveals that most misses stem from tracking failures rather than overly strict counting rules.

What Problem Does This Paper Address?

In crowd monitoring, we often use video-based tracking to count people passing through "gates" (like stadium entrances). While we get a final tally, these totals are "black boxes"—they don't tell us which individuals were missed or where in the pipeline the failure occurred.

Occlusion is the primary challenge: it can break a person's trajectory or cause the system to swap IDs between people. Current evaluation metrics (like HOTA or MOTA) provide a global sense of tracking quality but don't specifically diagnose why a discrete event, like a line crossing, disappears from the final count.

Methods

The authors introduce GateTrace, a four-step framework designed to audit the "life cycle" of a crossing event:

  1. Build Crossing References: Using ground truth annotations (like those in MOT20), it identifies every time a person actually crosses a virtual gate.
  2. Find Crossings in Tracks: It applies the same counting rules to the output of various trackers (e.g., ByteTrack, OC-SORT).
  3. Match Crossings: It attempts to match tracker reports to ground truth references based on person ID, direction, time, and spatial position.
  4. Trace Missed Crossings: For every unmatched reference, GateTrace "drills down" to find the failure point. Did the detector fail? Did the tracker lose the person? Or did a post-processing filter (like a "depth check" requiring a person to move a certain distance past the line) reject a valid crossing?
The GateTrace framework: from building references to tracing misses.

Datasets

  • MOT20: A public dataset of crowded scenes. It is crucial here because it provides consistent person IDs across frames and, importantly, visibility values for each person.
  • Stadium Study: The work was originally motivated by a study of pedestrian flow in a football stadium, though the lack of per-frame annotations there led the researchers to MOT20 for deeper analysis.

Key Insights and Ideas

  • The Visibility Cliff: There is a massive performance gap based on visibility. While trackers recover nearly 99% of highly visible crossings, they miss more than half of those where the person is mostly obscured.
  • Tracking, Not Rules, is the Bottleneck: A common intuition might be that counting rules are too strict. However, GateTrace shows that 47–79% of low-visibility misses happen because there is simply no tracker output linked to that person near the gate.
  • The Precision Cost of Leniency: Relaxing counting rules (e.g., removing "depth checks") only marginally improves the recovery of missed crossings but significantly degrades precision (dropping from ~96% to ~77%), leading to many "ghost" counts.
  • Human-in-the-Loop Validation: Human annotators confirmed that many crossings missed by AI are clearly visible and valid to the human eye, highlighting a significant remaining gap for state-of-the-art trackers in dense crowds.

Key Results

  • Tracker Performance: Popular trackers like ByteTrack, OC-SORT, and BoT-SORT recovered only 38.9–46.7% of low-visibility crossings on development sequences, compared to 98.1–99.0% for high-visibility ones.
  • BASE Tracker: A more heavily tuned system (BASE) performed better, recovering 63–78% at low visibility, but the visibility gap remained persistent.
  • Failure Modes: As shown in the figure below, the two primary failure modes are (a) the track never actually crosses the gate regions in the system's eyes, or (b) a "possible crossing" is found but rejected by a depth check because the track didn't penetrate far enough into the destination zone.
Schematic of failure modes: no crossing found vs. rejected by depth check.

Limitations and Future Work

  • Scene Diversity: The study is limited to four sequences from the MOT20 dataset. Results might vary in different environments or lighting conditions.
  • Point-based Counting: The system relies on the bottom-center of bounding boxes. In very dense crowds, feet are often the first things to be occluded, which can lead to "noisy" entry/exit events.
  • Human Timing: While humans are good at seeing if a crossing happened, they struggle to provide the exact frame-level timing needed for perfect alignment with automated systems.
  • Future Work: The authors suggest that GateTrace can be used to develop more robust "repair" strategies that selectively recover missed crossings without sacrificing precision.

Summary updated

Abstract

At large events, pedestrian tracking supports entrance and passageway counts, but totals hide which crossings were missed. In a stadium study, the system recorded about four in five clearly visible crossings but fewer than one in three mostly obscured crossings. The videos lacked trajectories annotated across frames, so we turned to MOT20, which provides pedestrian trajectories and visibility. We introduce GateTrace to locate missed crossings and assess changes to counting rules. It derives reference crossings from annotations under fixed rules, matches reports by person, direction, time and position, and traces misses through saved boxes and counting decisions. With a shared detector, ByteTrack, OC-SORT and BoT-SORT recover 38.9-46.7% of low-visibility reference crossings on development sequences, versus 98.1-99.0% at high visibility. The gap recurs on two further sequences under a protocol fixed in advance. Published outputs of MOT20-tuned BASE recover 63-78% at low visibility across both sequence groups. Losses stop at different stages across systems. Across four systems and both groups, 47-79% of low-visibility misses have no nearby output linked to the person or no possible crossing in the saved tracks. Removing later checks from BASE on development sequences leaves most misses unresolved and lowers precision against the fixed reference list from about 96% to 77%. Misses also remain among sampled reference crossings that two human annotators supported after checking the videos. GateTrace shows where crossings are lost and measures the recovery and precision costs of changing later checks. We will release the code and per-crossing records.
BibTeX
@misc{he2026gatetrace,
  author = {Ao He and Qianchen Yu and Vivian W. H. Wong},
  title = {{GateTrace: Where Do Crowd Crossings Go Missing Under Occlusion?}},
  year = {2026},
  eprint = {2611.26388},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26388},
}
Figure