Deep Polarized Network for Supervised Learning of Accurate Binary Hashing Codes

Lixin Fan, Kam Woh Ng, Ce Ju, Tianyu Zhang, Chee Seng Chan

Introduction

In large-scale image retrieval, "learning to hash" (LtH) has become the standard approach for mapping high-dimensional image data into compact binary codes. The primary objective is to ensure that similar images are mapped to nearby points in Hamming space. However, training these networks is challenging due to the non-differentiable nature of binary constraints. Traditional methods often rely on continuous relaxation, which introduces significant quantization errors, or complex pairwise/triplet losses that are computationally expensive ($O(N^2)$ or $O(N^3)$).

This paper introduces the Deep Polarized Network (DPN), a simplified framework that addresses these issues through a novel "polarization loss." By pushing network outputs toward extreme values, DPN achieves state-of-the-art accuracy without the need for sophisticated binary optimization solvers or expensive pairwise labels.

The Deep Polarized Network (DPN)

The DPN architecture simplifies the hashing process by replacing complex similarity-based losses with a pointwise Polarization Loss ($L_P$). Instead of comparing images to one another, the network pushes each output channel toward a pre-assigned target binary vector ($t$) for its class.

The DPN architecture featuring the Polarization Loss branch and the Sign(v) branch for binary code generation.

Polarization Loss

The core innovation is the bit-wise hinge-like loss function: $$L_P(v, t) = \sum_{i=1}^{K} \max(m - v_i \cdot t_i, 0)$$ where $v$ is the network output, $t$ is the target bit, and $m$ is a margin. This function forces the network not only to match the sign of the target bit but to "polarize" the values far from zero, effectively eliminating the gap between continuous training values and discrete binary codes.

Target Vector Assignment and Ternary Inference

The authors demonstrate that simple Random Assignment of unique binary strings to classes is remarkably effective, provided the bits are balanced (approximately 50% ones and zeros). For inference, they introduce a Ternary Assignment (DPN-T) trick: network outputs within a small margin $[-m, m]$ are treated as zero. This ignores "uncertain" bits, significantly boosting retrieval precision.

Theoretical Equivalence and Insights

A key theoretical contribution is the proof that minimizing this pointwise polarization loss is equivalent to simultaneously minimizing intra-class Hamming distance and maximizing inter-class Hamming distance. This allows DPN to reap the benefits of pairwise learning at a fraction of the computational cost.

Empirical validation of the convergence of the proposed loss compared to standard cross-entropy and similarity-based losses.

As shown in the loss curves, the DPN objective (LHS) converges reliably and maintains a consistent relationship with standard similarity losses, validating the theoretical framework empirically.

Experimental Evaluation

The DPN was evaluated on three standard benchmarks: CIFAR-10, NUS-WIDE, and ImageNet100.

  • Accuracy: On ImageNet100 (64 bits), DPN-T achieved a Mean Average Precision (mAP) of 0.756, outperforming established methods like HashNet (0.684) and GreedyHash (0.688).
  • Multi-label Performance: On the multi-label NUS-WIDE dataset, DPN-T reached 0.863 mAP, setting a new state-of-the-art.
  • Efficiency: By using a pointwise loss, DPN avoids the $O(N^2)$ complexity of pairwise similarity matrices, making it significantly faster to train than competing models.

Conclusion

The Deep Polarized Network offers a refreshingly simple yet powerful solution to the challenges of binary hashing. By utilizing polarization to bridge the quantization gap and proving the equivalence between pointwise and pairwise objectives, DPN provides a highly efficient and accurate path for large-scale image retrieval. Future work may explore more stable adaptive target assignment strategies for datasets with extremely large numbers of classes.

Summary updated

Abstract

This paper proposes a novel deep polarized network (DPN) for learning to hash, in which each channel in the network outputs is pushed far away from zero by employing a differentiable bit-wise hinge-like loss which is dubbed as polarization loss. Reformulated within a generic Hamming Distance Metric Learning framework [Norouzi et al., 2012], the proposed polarization loss bypasses the requirement to prepare pairwise labels for (dis-)similar items and, yet, the proposed loss strictly bounds from above the pairwise Hamming Distance based losses. The intrinsic connection between pairwise and pointwise label information, as disclosed in this paper, brings about the following methodological improvements: (a) we may directly employ the proposed differentiable polarization loss with no large deviations incurred from the target Hamming distance based loss; and (b) the subtask of assigning binary codes becomes extremely simple --- even random codes assigned to each class suffice to result in state-of-the-art performances, as demonstrated in CIFAR10, NUS-WIDE and ImageNet100 datasets.
BibTeX
@inproceedings{fan2020deep,
  author = {Lixin Fan and Kam Woh Ng and Ce Ju and Tianyu Zhang and Chee Seng Chan},
  title = {{Deep Polarized Network for Supervised Learning of Accurate Binary Hashing Codes}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/115},
}
Figure