Abstract
Multi-Object Tracking (MOT) is a challenging task in complex scenes such as surveillance and autonomous driving. In this paper, we propose a novel tracklet processing method to cleave and re-connect tracklets on crowd or long-term occlusion by Siamese Bi-Gated Recurrent Unit (GRU). The tracklet generation utilizes object features extracted by CNN and RNN to create high-confidence tracklet candidates in sparse scenarios. Due to mis-tracking in the generation process, tracklets from different objects are split into several sub-tracklets by a bidirectional GRU. After that, a Siamese GRU based tracklet re-connection method is applied to link the sub-tracklets which belong to the same object to form a whole trajectory. In addition, we extract the tracklet images from existing MOT datasets and propose a novel dataset to train our networks. Experimental results demonstrate the advantages of our model over state-of-the-art methods on MOT16.
Introduction
Modern Multi-Object Tracking (MOT) systems typically follow the "tracking-by-detection" paradigm. While effective, this approach struggles in crowded environments or during long-term occlusions, leading to two primary failure modes: Identity Switches, where a tracker merges two different individuals into a single tracklet, and Fragmentation, where a single person's path is broken into multiple disconnected segments.
Instead of solely focusing on improving initial association, the "Trajectory Factory" framework acknowledges that errors are inevitable and proposes a post-processing pipeline to identify and rectify these mistakes.
The Trajectory Factory Pipeline
The proposed framework treats tracklets as repairable objects through a three-stage process: generation, cleaving, and re-connection.
1. Tracklet Generation
Initial high-confidence tracklets are generated using appearance features from a ResNet-50 CNN and motion predictions from an LSTM. This stage focuses on creating reliable candidates in sparse scenarios.
2. Tracklet Cleaving
To resolve identity switches, the system employs a Bidirectional Gated Recurrent Unit (Bi-GRU). By analyzing the temporal sequence of features from both forward and backward directions, the model calculates a "feature distance" at every point. A sudden peak in this distance indicates a likely switch between different individuals, triggering a "cleave" at that temporal location.
3. Tracklet Re-connection
Fragmented sub-tracklets are linked using a Siamese Bi-GRU architecture. This network determines if two fragments belong to the same identity by optimizing a multi-task loss that combines verification and identification. Polynomial curve fitting is then used to interpolate smooth trajectories across the gaps between re-connected segments.
Experimental Evaluation
The networks were trained using a combination of Market1501, PathTrack, and MARS datasets. Additionally, the authors introduced a specialized dataset containing over 95,160 pedestrian images (793 unique identities) extracted from existing MOT datasets to specifically train the cleaving and re-connection modules.
On the MOT16 benchmark, the "Trajectory Factory" (GCRA) achieved a Multi-Object Tracking Accuracy (MOTA) of 48.2, outperforming established methods like QuadMOT16 (44.1) and EDMT (45.3). The system demonstrated particularly strong performance in static camera sequences, where it frequently ranked first in accuracy.
Conclusion
The "Trajectory Factory" demonstrates that treating tracklets as temporal sequences and explicitly modeling the "cleave then link" process significantly improves tracking robustness. While the system excels in static scenes, performance in moving camera scenarios remains a challenge due to the noise introduced by camera ego-motion. Future research will focus on developing motion models and appearance descriptors that are more invariant to rapid perspective changes.
Summary updated