Attention-guided Contrastive Hashing for Long-tailed Image Retrieval

Xuan Kou, Chenghao Xu, Xu Yang, Cheng DengIJCAI 2022 · OpenReview

Introduction

In practical computer vision applications, datasets often exhibit a "long-tailed" distribution where a few "head" categories contain the majority of samples, while numerous "tail" categories have very few. Standard image hashing techniques—which compress high-dimensional feature vectors into compact binary codes for efficient retrieval—frequently struggle with such skewed data. This failure typically stems from two primary issues: significant information loss during extreme dimensionality reduction where rare class features are discarded in favor of dominant ones, and the breakdown of instance-level contrastive learning in unbalanced batches where tail categories lack sufficient positive pairs to learn from.

Methodology: ACHNet

To address these challenges, we propose ACHNet (Attention-guided Contrastive Hashing Network). The framework introduces two core innovations designed to preserve discriminative information and stabilize learning under extreme imbalance.

Overview of the ACHNet architecture, featuring the Cross Attention Feature Enhancement (CAFE) module and the Category-centered Contrastive Hashing (CCH) mechanism.

Cross Attention Feature Enhancement (CAFE)

Since the final output of a hashing network is a binary code (±1), the polarity (sign) of intermediate features is more critical than their precise magnitude. The CAFE module utilizes parallel layers to predict feature importance based on these signs. By generating an attention mask that highlights discriminative components before binarization, CAFE effectively mitigates the loss of unique features essential for identifying rare classes. This "triage" of information ensures that even when space is limited to 32 bits, the most vital class-specific signals are preserved.

Category-centered Contrastive Hashing (CCH)

Traditional contrastive learning relies on instance-to-instance comparisons, which often fail in long-tailed scenarios because a training batch may contain only a single sample from a tail category. CCH circumvents this by comparing samples to pseudo hash code centers for every category, which are recalculated at the start of each epoch. This ensures that every sample, regardless of its category's frequency, has a stable "positive center" to approach and multiple "negative centers" to avoid, providing consistent gradients even for rare classes.

Experimental Setup

ACHNet was evaluated on two benchmark datasets, Cifar-100 and ImageNet-100, modified to simulate real-world imbalance using Zipf’s Law. The experiments utilized various Imbalance Factors (IF), ranging from perfectly balanced (IF1) to highly skewed (IF100), where the most frequent class is 100 times larger than the least frequent.

Dataset statistics for Cifar-100 and ImageNet-100 across different Imbalance Factors (IF).

Results and Discussion

ACHNet demonstrates superior performance over state-of-the-art methods like LTHNet and CSQ, particularly on highly unbalanced data and at lower bit lengths.

  • Performance Gains: On Cifar-100 (IF=100, 32 bits), ACHNet achieved a 22.46% mAP (mean Average Precision), a significant jump over LTHNet’s 18.19%. On ImageNet-100 (IF=100, 32 bits), it reached 79.65% mAP, compared to LTHNet’s 71.46%.
  • Robustness: Unlike many models that degrade sharply as imbalance increases, ACHNet remains robust across all IF settings. It does not sacrifice "head" category performance to improve "tail" category retrieval.
  • Global vs. Local Contrast: The success of CCH highlights that global category centers are more effective than local batch-wise pairs for long-tailed learning, as they provide a reliable target for every sample in every batch, regardless of batch composition.

Conclusion

ACHNet provides a robust solution for image retrieval in "messy," real-world datasets. While it is slightly less specialized for perfectly balanced data, its ability to handle extreme skew makes it a valuable tool for practical applications. Future work may focus on optimizing the computation of category centers for massive-scale datasets and extending the attention-guided hashing approach to other long-tailed tasks such as object detection or medical imaging.

Summary updated

BibTeX
@inproceedings{kou2022attention,
  author = {Xuan Kou and Chenghao Xu and Xu Yang and Cheng Deng},
  title = {{Attention-guided Contrastive Hashing for Long-tailed Image Retrieval}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=Z7R9wJuDQl},
}
Figure