WEA-DINO: An Improved DINO With Word Embedding Alignment for Remote Scene Zero-Shot Object Detection

Guangbiao Wang, Hongbo Zhao, Qing Chang, Shuchang Lyu, Guangliang Cheng, Huojin ChenIEEE GRSL 2024 · OpenReview

Abstract

Remote sensing scene zero-shot object detection (ZSD) aims to detect and recognize both seen and unseen categories of landscape elements with the guidance of the word embeddings. In this task, two primary challenges are identified. First, there exists considerable variability within categories of landscape elements, causing a misalignment between visual features and word embeddings, particularly noticeable for unseen categories. Second, the existing detection models struggle to provide accurate localization predictions, greatly impacting overall performance. To address these two issues, we propose word embedding alignment-DINO (WEA-DINO). Based on the original DINO structure, our WEA-DINO-Head is specifically designed to align the hidden features of “matching queries” with word embedding features, effectively addressing the misalignment issue between visual features and word embeddings. Furthermore, aligning the hidden features of “denoising queries” with word embedding features enables the translation of localization capabilities from known categories to previously unseen ones. Through extensive experimentation on the DIOR benchmark dataset, our method demonstrates state-of-the-art (SOTA) performance. The code is available at https://github.com/cv516Buaa/WEA-DINO .
BibTeX
@article{wang2024wea,
  author = {Guangbiao Wang and Hongbo Zhao and Qing Chang and Shuchang Lyu and Guangliang Cheng and Huojin Chen},
  title = {{WEA-DINO: An Improved DINO With Word Embedding Alignment for Remote Scene Zero-Shot Object Detection}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2024},
  url = {https://openreview.net/forum?id=DJ1hQ8D9L4},
}

Loading related content...

Figure