Deep Position-Aware Hashing for Semantic Continuous Image Retrieval

Ruikui Wang, Ruiping Wang, Shishi Qiao, Shiguang Shan, Xilin Chen

Introduction

Image hashing is a critical technique for large-scale image retrieval, mapping high-dimensional visual data into compact binary codes. This allows for rapid search and minimal memory usage. However, traditional deep hashing methods often struggle with preserving semantic similarity across an entire dataset, particularly when dealing with complex multi-label scenarios. This paper introduces Deep Position-Aware Hashing (DPAH), a framework designed to provide a global coordinate system in Hamming space and minimize information loss during the quantization process.

Addressing the Global Perspective

A significant limitation in current deep hashing is the reliance on "local" perspectives. Most models utilize pair-wise or triplet-wise losses that only compare images within a small training batch. This leads to "intermittent similarity," where the model learns to distinguish images it sees together but lacks a consistent global understanding of category relationships.

Comparison of traditional hashing strategies (a-c) versus the proposed Position Awareness (d), which utilizes global class centers to establish a global coordinate system.

To solve this, DPAH introduces Hamming Position Awareness (PA). Instead of purely relative comparisons, the model learns a set of class centers that serve as global proxies for each category. These centers act as the "origin" of a coordinate system in Hamming space. Every image is constrained to be near its relevant class centers and separated from irrelevant ones by a specific margin $\alpha$.

The Position Awareness learning process transforms scattered feature representations into structured clusters around learnable class centers, ensuring global semantic consistency.

For multi-label images, the model targets the average of the relevant class centers. This approach naturally handles continuous semantic similarity; for example, an image containing both a "dog" and a "park" will be positioned between the respective centers for those categories.

Bridging the Quantization Gap

The second major challenge is the "quantization gap." Neural networks naturally output continuous real values, but hashing requires discrete bits (0 or 1). Standard methods often "relax" these bits during training, leading to significant information loss when the final thresholding is applied.

DPAH introduces a novel Kurtosis Loss (KT Loss) to shape the distribution of neural network outputs. By optimizing the statistical kurtosis (the "peaky-ness") of the feature distribution, the KT Loss penalizes values that are far from the binary targets. This forces the internal representations into a bimodal, "double-peak" shape before the final quantization step, significantly reducing error and preserving retrieval accuracy.

Experimental Evaluation

The authors evaluated DPAH on three standard benchmarks: ImageNet (single-label), MS COCO (multi-label), and NUS-WIDE (multi-label).

Key results include:

  • Superior Accuracy: On the NUS-WIDE dataset with 48-bit codes, DPAH achieved a Mean Average Precision (mAP) of 0.8346, outperforming previous state-of-the-art methods like HashNet (0.8053).
  • Robustness: The performance gains were consistent across different backbone architectures, including AlexNet and ResNet-101.
  • Clarity in Hamming Space: t-SNE visualizations confirm that DPAH creates tighter, more distinct clusters compared to traditional methods, where categories often overlap due to the lack of global positioning.

Conclusion

Deep Position-Aware Hashing effectively addresses the core weaknesses of local-batch training and quantization error in image retrieval. By establishing a global coordinate system through learnable class centers and shaping feature distributions with Kurtosis Loss, DPAH provides a robust solution for both single-label and complex multi-label retrieval tasks. Future work may explore extending this supervised framework to unsupervised or self-supervised settings where category labels are not readily available.

Summary updated

Abstract

Preserving the semantic similarity is one of the most important goals of hashing. Most existing deep hashing methods employ pairs or triplets of samples in training stage, which only consider the semantic similarity within a minibatch and depict the local positional relationship in Hamming space, leading to intermittent semantic similarity preservation. In this paper, we propose Deep Position-Aware Hashing (DPAH) to ensure continuous semantic similarity in Hamming space by modeling global positional relationship. Specifically, we introduce a set of learnable class centers as the global proxies to represent the global information and generate discriminative binary codes by constraining the distance between data points and class centers. In addition, in order to reduce the information loss caused by relaxing the binary codes to real-values in optimization, we propose kurtosis loss (KT loss) to handle the distribution of real-valued features before thresholding to be double-peak, and then enable the real-valued features to be more binarylike. Comprehensive experiments on three datasets show that our DPAH outperforms state-of-the-art methods.
BibTeX
@inproceedings{wang2020deep,
  author = {Ruikui Wang and Ruiping Wang and Shishi Qiao and Shiguang Shan and Xilin Chen},
  title = {{Deep Position-Aware Hashing for Semantic Continuous Image Retrieval}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  doi = {10.1109/wacv45572.2020.9093468},
}
Figure