Abstract
Representing images by compact hash codes is an attractive approach for large-scale content-based image retrieval. In most state-of-the-art hashing-based image retrieval systems, local descriptors are first aggregated as a global representation vector, which is then subjected to a hashing function. Traditionally, these processes are designed independently, potentially leading to suboptimal codes. This paper proposes a novel unsupervised hashing framework where feature aggregating and hashing are optimized jointly. By generating aggregated representations that are better reconstructed by binary codes, we achieve more discriminative hash codes and improved retrieval accuracy. We also introduce a fast version of the Binary Autoencoder (BA) and extend the framework for supervised hashing. Extensive experiments demonstrate that our methods outperform existing state-of-the-art techniques.
Introduction
Large-scale image retrieval relies on compressing high-dimensional image data into compact binary hash codes. The standard pipeline involves extracting local features (e.g., SIFT), aggregating them into a global vector (e.g., VLAD), and then applying a hashing function. However, the independent design of these two stages creates a "semantic gap": an aggregation method might optimize for Euclidean distance but produce vectors that are difficult to represent accurately with binary bits. Furthermore, high-performing hashing methods like the Binary Autoencoder (BA) are often computationally prohibitive due to NP-hard optimization constraints.
Methodology
To address these challenges, we propose three primary innovations:
Relaxed Binary Autoencoder (RBA)
To overcome the computational bottleneck of the standard Binary Autoencoder, we developed the Relaxed Binary Autoencoder (RBA). Instead of directly optimizing the non-differentiable sign function, RBA employs a quadratic penalty method with an auxiliary variable. This relaxation allows for an alternating optimization strategy with closed-form solutions.
As shown in the figure above, RBA is significantly more efficient, training over 10 times faster than the original BA while maintaining or even improving retrieval accuracy.
Simultaneous Feature Aggregating and Hashing (SAH)
The SAH framework jointly optimizes hashing parameters and aggregated representations. The objective function ensures that the resulting global vectors are both faithful to the original local features and easily reconstructible by binary codes. This coupling ensures that the aggregation step is "aware" of the subsequent hashing constraints.
Supervised SAH (SASH)
For scenarios where labels are available, SASH minimizes reconstruction loss relative to label vectors. To handle test images without labels, we learn a linear mapping $P$ that projects standard aggregated features into the optimized feature space, effectively bridging the gap between training and inference.
Experimental Results
We evaluated our methods on several benchmarks, including CIFAR10, MNIST, SIFT1M, and Oxford5k, using both hand-crafted (SIFT) and deep features (VGG-16).
Retrieval Accuracy
The joint optimization in SAH leads to superior discriminative power. On the Oxford5k dataset, SAH outperformed the best competitors by approximately 2-3% mAP.
In supervised settings, SASH showed even more dramatic gains. On CIFAR10, SASH achieved a mAP of 53-63%, significantly outperforming the popular Supervised Discrete Hashing (SDH) method, which achieved 40-48%. SASH also proved competitive with end-to-end deep supervised hashing methods, particularly in generalizing to unseen classes.
Conclusion
By jointly optimizing feature aggregation and hashing, we have demonstrated that binary codes can be made significantly more discriminative and efficient. The Relaxed Binary Autoencoder provides a scalable foundation for this framework, reducing training times by an order of magnitude. Future work may explore non-linear mappings for supervised inference and full end-to-end integration with convolutional layers to further enhance performance.
Summary updated