Trans2k: Unlocking the Power of Deep Models for Transparent Object Tracking

Alan Lukezic, Ziga Trojer, Jiri Matas, Matej Kristan

Abstract

Transparent object tracking presents a unique challenge in computer vision as target appearance is heavily influenced by dynamic backgrounds. Previous benchmarks suggested that deep learning models struggled with transparency compared to simpler methods. This paper introduces Trans2k, the first large-scale synthetic dataset specifically for transparent object tracking. By providing over 2,000 high-quality sequences with realistic physics-based rendering, we demonstrate that deep models can significantly outperform traditional methods when trained on appropriate data, achieving performance gains of up to 16%.

Introduction

Visual object tracking has traditionally focused on opaque targets with stable textures. Transparent objects, such as glassware or bubbles, lack consistent internal features, making them difficult for standard trackers that often inadvertently lock onto background textures. Early evaluations on the Transparent Object Tracking Benchmark (TOTB) led to the hypothesis that deep features were inherently unsuitable for this task.

We argue that this performance gap stems from a lack of specialized training data rather than architectural limitations. To address this, we present Trans2k, a dataset designed to bridge the domain gap between opaque and transparent object tracking.

Qualitative comparison of trackers pre-trained on opaque objects versus those fine-tuned with the Trans2k dataset.

The Trans2k Dataset

Trans2k consists of 2,039 sequences (104,343 frames) generated using the BlenderProc rendering engine. The dataset is meticulously designed to cover the specific challenges of transparency through three core strategies:

  1. Attribute Parameterization: We varied six critical attributes: transparency level, motion blur, partial occlusion, distractors, target motion, and 3D rotation.
  2. Difficulty-Aware Selection: Rather than random generation, we identified "breaking points" for existing trackers and focused data generation on these challenging scenarios.
  3. Realistic Background Integration: To improve generalization, we used real-world video frames from the GoT10k dataset as backgrounds for our synthetic renders, forcing models to learn the subtle boundaries of transparent objects.

Results and Discussion

Our experiments debunk the myth that deep backbones are ill-suited for transparent tracking. When trained on Trans2k, models with deeper backbones (e.g., ResNet50) consistently outperformed shallower counterparts.

Quantitative Performance

Training with Trans2k yielded substantial improvements across various architectures:

  • DiMP showed an absolute improvement of 9.9% in AUC.
  • STARK, a transformer-based tracker, achieved state-of-the-art results, benefiting from its global attention mechanism to discern transparent boundaries.
  • Generalization: Models trained on Trans2k maintained high performance on standard opaque datasets, indicating minimal catastrophic forgetting.
Success and precision plots showing the consistent performance boost across different tracking architectures when utilizing the Trans2k dataset.

Conclusion

The Trans2k dataset demonstrates that the "invisible" can be tracked effectively by deep learning models given sufficient, high-quality training data. By leveraging synthetic rendering that respects the laws of physics, we have unlocked significant performance gains in transparent object tracking. Future work will explore extending this approach to 6-DoF pose estimation to aid robotic manipulation of transparent items.

Summary updated

Abstract

Abstract
BibTeX
@inproceedings{lukezic2022trans,
  author = {Alan Lukezic and Ziga Trojer and Jiri Matas and Matej Kristan},
  title = {{Trans2k: Unlocking the Power of Deep Models for Transparent Object Tracking}},
  booktitle = {BMVC},
  year = {2022},
  url = {https://openreview.net/forum?id=Rr1V2m5JZR},
}

Loading related content...

Figure