Abstract
Unsupervised image hashing is essential for large-scale image retrieval but often fails when query images are degraded. We present Contrastive Transformer Masked Image Hashing (CTMIH), a framework that leverages Vision Transformers (ViT) and masked patch reconstruction to learn transform-invariant hash codes. By integrating debiased contrastive learning and semantic token reconstruction, CTMIH bridges the gap between degraded queries and clean database images. Our experiments on MS COCO, NUS-WIDE, and ImageNet demonstrate that CTMIH significantly outperforms existing state-of-the-art methods, particularly in maintaining robustness against various image deteriorations.
Introduction
Image hashing facilitates rapid retrieval by mapping high-dimensional images to compact binary codes. While effective for clean data, real-world queries—such as those from surveillance or social media—are frequently degraded by noise, blur, or compression. This "appearance gap" shifts hash codes, leading to retrieval failures. Most existing unsupervised methods lack the robustness to handle such variations without extensive manual labeling.
Methodology
CTMIH employs a Siamese Vision Transformer (ViT) architecture trained through three primary objectives:
- Cross-view Debiased Contrastive Learning: The model processes two differently degraded views of an image, minimizing the distance between their hash codes while maximizing the distance from other images. A debiasing mechanism is used to prevent the accidental penalization of semantically similar "negative" samples.
- Semantic Mask Reconstruction: Inspired by Masked Autoencoders, the model predicts latent patch tokens of masked regions rather than raw pixels. This encourages the network to capture high-level semantic structures rather than low-level noise.
- Quantization Loss: To bridge the gap between continuous features and binary bits, a Gaussian distribution estimator is used to optimize the quantization process.
Experimental Evaluation
The framework was evaluated on three standard benchmarks: MS COCO, NUS-WIDE, and a subset of ImageNet. To simulate real-world conditions, queries were subjected to synthetic degradations including flipping, color jittering, grayscale conversion, and blurring.
Key Insights
- ViT Advantages: Unlike CNNs, ViTs utilize self-attention to capture long-range dependencies, which is crucial for understanding images where significant portions are obscured or degraded.
- Token-based Reconstruction: Reconstructing semantic tokens allows the model to ignore irrelevant visual artifacts, focusing instead on the core conceptual content of the image.
Results
CTMIH demonstrates superior performance across all benchmarks:
- Accuracy: On ImageNet (64 bits), CTMIH achieved a mean Average Precision (mAP) of 0.869, surpassing the previous state-of-the-art (0.814).
- Robustness: While traditional methods saw performance drops of up to 20% under heavy degradation, CTMIH remained stable with only a 1.3% decrease.
- Generality: The model also outperformed existing methods on standard (non-degraded) retrieval tasks, indicating higher quality feature representation.
Conclusion and Future Work
CTMIH provides a robust solution for unsupervised image hashing in the presence of query degradation. Future research will focus on optimizing the computational efficiency of the ViT backbone and validating the model against complex, "in-the-wild" degradations such as extreme low-light noise and motion blur.
Summary updated