Unsupervised Embedding and Association Network for Multi-Object Tracking

Yu-Lei LiIJCAI 2022 · DOI

TL;DR

This paper introduces UEANet, a novel unsupervised framework for multi-object tracking (MOT) that bypasses the need for costly manual identity annotations. By leveraging pseudo identity labels and a discriminative feature enhancer, UEANet achieves competitive performance with state-of-the-art supervised methods, offering a more accessible approach to MOT.

What Problem Does This Paper Address?

Multi-object tracking (MOT) is crucial for applications like autonomous driving and robot vision, aiming to accurately track multiple objects across video sequences. The dominant "tracking-by-detection" paradigm involves detecting objects and then associating these detections into coherent trajectories. While recent advancements have focused on improving detection and using identity embeddings for association, a significant bottleneck remains: the reliance on extensive manual identity annotations. These annotations are expensive to create and limit the scalability of MOT systems. Existing unsupervised methods often struggle with confusing embedding features due to the lack of supervision, leading to degraded performance. This paper tackles the challenge of achieving high-performance MOT without manual identity labels, aiming to reduce the annotation cost while maintaining tracking accuracy.

Methods

The core of UEANet is its unsupervised embedding and association network. It comprises two main branches:

  • Discriminative Feature Enhancer (DFE): This novel module is integrated into both the detection and identity embedding branches. It uses a combination of attention mechanisms (e.g., SGE-attention) and MLP components (e.g., MLPMixer) to suppress competition between the two branches. The DFE aims to generate more reliable detection predictions and, crucially, more discriminative embedding features that are less susceptible to the noise introduced by pseudo labels.
  • MLP-based Data Association Branch: This branch takes the enhanced embedding features and performs end-to-end data association. Unlike traditional hand-crafted methods or simpler deep association networks, this MLP-based approach directly learns to associate candidate trajectories, contributing to more robust tracking.

The training process is unsupervised, relying on pseudo identity labels generated by a Kalman filter tracker (specifically, ByteTrack). These pseudo labels are used to train the identity embedding and data association branches, while the detection branch is trained with original manual detection labels. This strategy allows the network to learn discriminative features without direct supervision on identities.

Datasets

The paper evaluates UEANet on three standard multi-object tracking datasets:

  • MOT2016
  • MOT2017
  • MOT2020

These datasets are commonly used for benchmarking MOT algorithms and represent varying levels of scene complexity, object density, and occlusion.

Key Insights and Ideas

  • Pseudo Labels as a Viable Alternative: The paper demonstrates that pseudo identity labels, generated by a robust tracker like ByteTrack, can be effectively used to train identity embedding and association modules, significantly reducing the need for manual annotations.
  • DFE for Branch Competition Suppression: The Discriminative Feature Enhancer (DFE) is a key innovation. It successfully mitigates the negative impact of noisy pseudo labels by enhancing the distinctiveness of features for both detection and identity embedding, leading to more robust tracking.
  • End-to-End Unsupervised Association: The MLP-based data association branch, trained jointly with the embedding branch, provides a more integrated and effective approach to tracking compared to separate, hand-crafted association steps.
  • Unsupervised Performance Closing the Gap: UEANet achieves performance comparable to supervised methods, suggesting that unsupervised MOT is a promising direction for practical applications where annotation costs are prohibitive.

Key Results

  • UEANet achieves comparable performance to state-of-the-art supervised methods on MOT2016 and MOT2017, ranking top-1 or top-2 on these datasets.
  • On MOT2016, UEANet shows an improvement of 1.3%/3.3%/2.2% on MOTA/IDF1/HOTA compared to a strong supervised baseline.
  • UEANet significantly decreases IDS (Identity Switches) by 30.2% on MOT2017 compared to ByteTrack, from 2196 to 1533.
  • The proposed method outperforms another unsupervised method, UTrack, on MOTA, IDF1, and IDS.
  • Ablation studies confirm the effectiveness of the attention variants and MLP components within the DFE module, with SGE-Attention and MLPMixer showing particularly strong results.

Limitations and Future Work

  • Performance on MOT2020: While UEANet performs well, its performance on MOT2020 is relatively lower compared to MOT2017. The authors attribute this to the heavier occlusion cases in MOT2020, which can still adversely affect pseudo identity labels.
  • Failure Examples: The paper acknowledges that even with improvements, there are still false negatives (FN) in predicted trajectories, indicating room for further enhancement, particularly in challenging scenarios.
  • Future Work: The authors suggest that further improvements could be made by addressing the challenges posed by heavy occlusion and improving the robustness of pseudo-label generation. The current work lays a strong foundation for future research in unsupervised MOT.

Summary updated

BibTeX
@inproceedings{li2022unsupervised,
  author = {Yu-Lei Li},
  title = {{Unsupervised Embedding and Association Network for Multi-Object Tracking}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/157},
}
Figure