Abstract
Recent years have witnessed wide application of hashing for large-scale image retrieval. However, most existing hashing methods are based on handcrafted features which might not be optimally compatible with the hashing procedure. Recently, deep hashing methods have been proposed to perform simultaneous feature learning and hash-code learning with deep neural networks, which have shown better performance than traditional hashing methods with hand-crafted features. Most of these deep hashing methods are supervised whose supervised information is given with triplet labels. For another common application scenario with pairwise labels, there have not existed methods for simultaneous feature learning and hash-code learning. In this paper, we propose a novel deep hashing method, called deep pairwise-supervised hashing (DPSH), to perform simultaneous feature learning and hash-code learning for applications with pairwise labels. Experiments on real datasets show that our DPSH method can outperform other methods to achieve the state-of-the-art performance in image retrieval applications.
Introduction
In large-scale image retrieval, hashing is essential for speed and memory efficiency, transforming high-dimensional image data into compact binary codes. Traditional hashing methods often rely on "hand-crafted" features (e.g., GIST or SIFT), which are not necessarily optimized for the hashing process. While deep hashing methods have emerged to learn features and codes simultaneously, most were designed for triplet labels (ranking-based). For the more common scenario of pairwise labels (similarity/dissimilarity), existing solutions like CNNH were "two-stage," lacking a feedback loop between feature learning and hash-code generation.
Deep Pairwise-Supervised Hashing (DPSH)
The authors propose DPSH, an end-to-end architecture that integrates feature learning and hashing into a single pipeline.
The framework consists of three main components:
- Feature Learning: A Convolutional Neural Network (CNN-F) that maps raw pixels to continuous image representations.
- Hashing Layer: A fully-connected layer that projects these representations into a latent space corresponding to the target hash code length.
- Loss Function: A novel objective function based on the likelihood of pairwise labels.
To address the non-differentiable nature of discrete binary bits, DPSH employs a relaxation strategy. By minimizing the difference between continuous variables and binary codes through a regularization term, the network can be trained via standard back-propagation while effectively pushing outputs toward valid binary values.
Experimental Setup
The model was evaluated on two major benchmarks:
- CIFAR-10: 60,000 single-label images across 10 classes.
- NUS-WIDE: A multi-label dataset with nearly 270,000 images. Two images are considered similar if they share at least one of the 21 most frequent concepts.
Key Insights and Robustness
A critical finding of the study is the importance of the feedback loop. A variant called DPSH0, where CNN weights were frozen, was significantly outperformed by the full DPSH model. This confirms that simultaneous learning—where hashing requirements refine the CNN's image representations—is the primary driver of performance.
Furthermore, DPSH demonstrates remarkable robustness to its hyper-parameters. As shown in the sensitivity analysis, the model maintains stable Mean Average Precision (MAP) across a wide range of values for the regularization parameter $\eta$.
Results
DPSH achieved state-of-the-art results across both datasets, often by a substantial margin:
- CIFAR-10 (48-bit): DPSH achieved a MAP of 0.757, outperforming CNNH (0.489) and the triplet-based NINH (0.581).
- NUS-WIDE (48-bit): DPSH reached a MAP of 0.851, significantly higher than CNNH's 0.608.
- Vs. Non-Deep Methods: Even when traditional methods were paired with deep features, DPSH remained superior because its features were specifically tailored for the hashing task during training.
Conclusion
DPSH represents the first end-to-end deep learning framework for simultaneous feature learning and hash-code generation using pairwise labels. While the study primarily utilized the CNN-F architecture and an approximate discrete optimization, the results clearly demonstrate the superiority of integrated learning over two-stage approaches. Future work may explore deeper architectures like ResNet and further optimize the scalability of pairwise training for massive datasets.
Summary updated