Abstract
Recent years have witnessed wide application of hashing for large-scale image retrieval. However, most existing hashing methods are based on hand-crafted features which might not be optimally compatible with the hashing procedure. Recently, deep hashing methods have been proposed to perform simultaneous feature learning and hash-code learning with deep neural networks, which have shown better performance than traditional hashing methods with hand-crafted features. Most of these deep hashing methods are supervised whose supervised information is given with triplet labels. For another common application scenario with pairwise labels, there have not existed methods for simultaneous feature learning and hash-code learning. In this paper, we propose a novel deep hashing method, called deep pairwise-supervised hashing (DPSH), to perform simultaneous feature learning and hash-code learning for applications with pairwise labels. Experiments on real datasets show that our DPSH method can outperform other methods to achieve the state-of-the-art performance in image retrieval applications.
Introduction
In the era of big data, efficient image retrieval is paramount. Hashing has emerged as a key technique, converting high-dimensional image data into compact binary codes (e.g., 12, 24, or 48 bits). This transformation enables rapid similarity searches and significant memory savings. Traditionally, hashing relied on "hand-crafted" features like GIST or SIFT. However, these features are often decoupled from the hashing process, leading to sub-optimal performance.
Deep hashing addresses this by using neural networks for simultaneous feature and hash-code learning. While many deep methods utilize triplet labels (e.g., "Image A is more similar to B than to C"), pairwise labels ("Image A and B are similar/dissimilar") are often more readily available. Previous pairwise deep methods, such as CNNH, typically followed a two-stage approach: learning hash codes first and then training a network to fit them. This lack of feedback between feature learning and hashing limits their effectiveness.
The DPSH Framework
Deep Pairwise-supervised Hashing (DPSH) is an end-to-end framework designed to overcome these limitations. The architecture integrates feature learning and hash function learning into a unified pipeline, as illustrated in Figure 1.
The framework consists of three main components:
- Feature Learning: A Convolutional Neural Network (CNN), specifically the CNN-F architecture, extracts continuous feature vectors from raw image pixels.
- Hash Function: A fully-connected layer maps these features into a latent space ($u_i$) that represents the potential hash codes.
- Objective Function: A loss function based on the likelihood of pairwise labels guides the learning process.
A significant challenge in deep hashing is the non-differentiable nature of the "sign" function used to generate binary bits ($-1$ or $+1$). DPSH addresses this through a discrete optimization strategy. It defines a likelihood function where the similarity probability depends on the inner product of the latent representations. To bridge the gap between continuous outputs and binary codes, a regularization term ($\eta ||b_i - u_i||^2$) is introduced. This allows the network to learn using continuous values while penalizing deviations from the target binary bits, facilitating smooth gradient-based optimization.
Experimental Results
The performance of DPSH was evaluated on two benchmark datasets: CIFAR-10 (a single-label dataset with 60,000 images) and NUS-WIDE (a multi-label dataset with nearly 270,000 images).
DPSH consistently outperformed both traditional and deep baselines. For instance, on CIFAR-10 with 48-bit codes, DPSH achieved a Mean Average Precision (MAP) of 0.757, significantly higher than the two-stage CNNH (0.489) and the triplet-based NINH (0.581). The importance of the end-to-end feedback loop was further validated by comparing DPSH to a variant with a frozen CNN (DPSH0), where DPSH showed marked superiority.
The sensitivity of the model to the hyper-parameter $\eta$ was also analyzed. As illustrated in Figure 2, DPSH maintains stable performance across a range of $\eta$ values, though optimal performance is typically found around $\eta=10$ or $100$. Additionally, the model demonstrates high efficiency, training at approximately 290 images per second on a single GPU.
Conclusion and Future Work
DPSH demonstrates that simultaneous feature and hash-code learning using pairwise labels can achieve state-of-the-art results in image retrieval. The end-to-end architecture allows the hashing requirements to directly inform the feature extraction process. Future research could explore the impact of more advanced CNN backbones like ResNet and investigate more sophisticated pair selection strategies to further enhance scalability and performance on massive datasets.
Summary updated