Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking

Gaoang Wang, Renshu Gu, Zuozhu Liu, Weijie Hu, Mingli Song, Jenq-Neng Hwang

Abstract

Multi-object tracking (MOT) traditionally relies on visual appearance features, which are computationally expensive and prone to failure under occlusion. This paper presents the Local-Global Motion (LGM) tracker, a novel framework that performs vehicle tracking using only motion data. By leveraging Graph Convolutional Networks (GCNs) in a two-stage embedding process—local box embedding and global tracklet embedding—the LGM tracker achieves state-of-the-art performance on vehicle tracking benchmarks, rivaling complex appearance-based models.

Introduction

In Multi-Object Tracking (MOT), maintaining object identity across video frames typically depends on "appearance features"—extracting pixel-level data to distinguish between targets. However, this approach is computationally intensive and often fails when objects are occluded.

Motion-based tracking offers a faster alternative by utilizing bounding box coordinates over time. While vehicles follow predictable, physics-bound paths, motion data is sparse. Pure motion trackers frequently lose targets due to false detections or long-term occlusions. The LGM tracker addresses these challenges by pushing the boundaries of what can be achieved using only motion data, effectively bridging the gap between sparse trajectory information and robust identity preservation.

Methodology

The LGM tracker models the tracking problem as a hierarchical two-stage embedding process using deep Graph Convolutional Networks (GCNs).

1. Box Motion Embedding (Local)

The system first processes raw bounding box detections across adjacent frames. These are connected to form short, continuous trajectories known as tracklets. An attention-guided GCN is employed to ensure the network aggregates information specifically from the same object, effectively filtering out noise from nearby vehicles.

2. Tracklet Motion Embedding (Global)

The core challenge in motion-only tracking is linking these fragmented tracklets across long temporal gaps, such as when a vehicle is hidden behind an obstacle. The LGM tracker introduces a "reconstruct-to-embed" strategy to solve this.

The LGM tracker pipeline: from raw detections to box embeddings, tracklets, and final tracks via a two-stage GCN architecture.

The Reconstruct-to-Embed Strategy

Directly matching fragmented tracklets is mathematically difficult due to varying lengths and missing data. Instead of direct matching, the LGM tracker uses Temporal Gated Convolutions (TGC) to extrapolate and "fill in" missing trajectory data. By forcing the network to reconstruct the missing path, the fragmented tracklets are transformed into complete representations. These are then embedded into a latent space where their similarities can be easily measured, making it clear whether a vehicle emerging from an occlusion is the same one that entered it.

Comparison between direct embedding and the proposed reconstruct-to-embed strategy, illustrating how reconstruction simplifies the embedding space for easier matching.

Experimental Results

The model was evaluated on several standard benchmarks, demonstrating that motion is a powerful cue for rigid object tracking:

  • KITTI-car tracking benchmark: The LGM tracker achieved a Higher Order Tracking Accuracy (HOTA) of 73.1% and a Multiple Object Tracking Accuracy (MOTA) of 87.6%. This performance surpasses other motion-only trackers and even slightly outperforms leading appearance-based models like CenterTrack (73.0% HOTA).
  • UA-Detrac: On this large-scale traffic dataset, the model achieved a PR-MOTA of 22.5%, outperforming state-of-the-art methods that rely on heavy appearance data, such as FAMNet (19.8%).
  • Ablation Studies: The authors noted that the reconstruction loss is critical; without it, the tracklet embedding module failed to converge, highlighting the importance of the "reconstruct-to-embed" approach.

Conclusion and Future Work

The LGM tracker demonstrates that for rigid objects like vehicles, motion patterns are sufficiently distinct for high-accuracy tracking without visual cues. The self-attention mechanism in the GCN is vital for preventing identity switches when vehicles pass closely.

While highly effective for vehicles, the pure-motion approach faces limitations with non-rigid or erratic subjects like pedestrians. Future research aims to integrate appearance information back into this robust motion-consistency framework to further enhance tracking performance across diverse object categories.

Summary updated

Abstract

Vehicle tracking is an essential task in the multi-object tracking (MOT) field. A distinct characteristic in vehicle tracking is that the trajectories of vehicles are fairly smooth in both the world coordinate and the image coordinate. Hence, models that capture motion consistencies are of high necessity. However, tracking with the standalone motion-based trackers is quite challenging because targets could get lost easily due to limited information, detection error and occlusion. Leveraging appearance information to assist object re-identification could resolve this challenge to some extent. However, doing so requires extra computation while appearance information is sensitive to occlusion as well. In this paper, we try to explore the significance of motion patterns for vehicle tracking without appearance information. We propose a novel approach that tackles the association issue for long-term tracking with the exclusive fully-exploited motion information. We address the tracklet embedding issue with the proposed reconstruct-to-embed strategy based on deep graph convolutional neural networks (GCN). Comprehensive experiments on the KITTI-car tracking dataset and UA-Detrac dataset show that the proposed method, though without appearance information, could achieve competitive performance with the state-of-the-art (SOTA) trackers. The source code will be available at https://github.com/GaoangW/LGMTracker.
BibTeX
@inproceedings{wang2021track,
  author = {Gaoang Wang and Renshu Gu and Zuozhu Liu and Weijie Hu and Mingli Song and Jenq-Neng Hwang},
  title = {{Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00973},
}
Figure