Accelerate Learning of Deep Hashing With Gradient Attention

Long-Kai Huang, Jianda Chen, Sinno Jialin Pan

Abstract

Deep hashing is a pivotal technique for efficient large-scale image retrieval, yet standard training methods often suffer from slow convergence. This paper identifies a "gradient dilemma" where simultaneous updates to paired hash codes cause them to overshoot their targets, stalling the optimization process. To address this, we propose a Gradient Attention Mechanism (GAH)—a lightweight auxiliary network that learns to dynamically weight gradients. By selectively modulating updates, GAH prevents sign-switching overshoots and significantly accelerates the training of high-performance hashing models.

Introduction: The Gradient Dilemma

Deep hashing maps high-dimensional images into compact binary codes to enable fast retrieval via Hamming distance. Most state-of-the-art models utilize pairwise similarity loss: similar images should have close hash codes, while dissimilar ones should be far apart.

However, standard gradient descent introduces a fundamental inefficiency. When correcting the distance between two codes, the model updates both simultaneously. If the update scale is too large, the codes may "switch" directions—passing each other and maintaining the same relative distance. This "dilemma" results in zero net progress for the loss function, leading to slow and unstable training.

The gradient dilemma in deep hashing: (a) simultaneous sign switching, (b) standard update, and (c) the proposed asymmetric update strategy.

Proposed Method: Gradient Attention Hashing (GAH)

The authors introduce Gradient Attention deep Hashing (GAH), which employs a "learning to learn" strategy to overcome the gradient dilemma.

Gradient Attention Network (GAN)

The core of GAH is a small auxiliary network consisting of two fully connected layers. This Gradient Attention Network (GAN) is integrated into the back-propagation path of the original hashing model to modulate the flow of information.

Architecture of the Gradient Attention Network (GAN) integrated with the hashing model.

Dynamic Weighting and Selective Updates

For every pair of training instances, the GAN receives the current hash codes and their raw gradients as input. It outputs attention weights ($\beta$) that modulate these gradients before they are applied. By learning to downplay the gradient of one instance in a pair, the GAN enables asymmetric updates. This allows one code to move toward its target bit more effectively without the risk of a mutual overshoot, effectively bypassing the sign-switching trap.

The GAN is optimized to maximize the decrease in the primary hashing model's loss, ensuring that the attention weights consistently favor faster convergence.

Experimental Evaluation

The GAH framework was evaluated on three standard benchmarks: CIFAR-10, NUS-WIDE, and a subset of ImageNet.

Key Results

  • Enhanced Accuracy: GAH consistently outperformed state-of-the-art competitors like HashNet and Greedy Hash (GH). Mean Average Precision (MAP) scores improved by 0.1% to 3.9% across various bit lengths.
  • Accelerated Convergence: The training loss for GAH decreases significantly faster than standard models. On CIFAR-10, GAH reduced the frequency of the "dilemma" (stuck bit pairs) by 13.1% to 61.9% within the first 200 iterations.
  • Implicit Quantization: Unlike many methods that require complex quantization loss terms to force values into binary bits, GAH naturally pushes values toward their targets by resolving the "tug-of-war" inherent in standard pairwise updates.

Conclusion and Future Work

Gradient Attention Hashing provides a robust solution to the gradient dilemma in pairwise deep hashing. By introducing a lightweight attention mechanism, the model achieves both higher retrieval accuracy and faster training speeds. While currently optimized for pairwise loss functions, future work could extend this mechanism to triplet or list-wise ranking structures. Additionally, while the GAN adds a small computational step, the reduction in total training iterations makes it a highly efficient choice for large-scale image retrieval tasks.

Summary updated

Abstract

Recent years have witnessed the success of learning to hash in fast large-scale image retrieval. As deep learning has shown its superior performance on many computer vision applications, recent designs of learning-based hashing models have been moving from shallow ones to deep architectures. However, based on our analysis, we find that gradient descent based algorithms used in deep hashing models would potentially cause hash codes of a pair of training instances to be updated towards the directions of each other simultaneously during optimization. In the worst case, the paired hash codes switch their directions after update, and consequently, their corresponding distance in the Hamming space remain unchanged. This makes the overall learning process highly inefficient. To address this issue, we propose a new deep hashing model integrated with a novel gradient attention mechanism. Extensive experimental results on three benchmark datasets show that our proposed algorithm is able to accelerate the learning process and obtain competitive retrieval performance compared with state-of-the-art deep hashing models.
BibTeX
@inproceedings{huang2019accelerate,
  author = {Long-Kai Huang and Jianda Chen and Sinno Jialin Pan},
  title = {{Accelerate Learning of Deep Hashing With Gradient Attention}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00537},
}
Figure