Abstract
Quantization has been widely adopted for large-scale multimedia retrieval due to its effectiveness of coding highdimensional data. Deep quantization models have been demonstrated to achieve the state-of-the-art retrieval accuracy. However, training the deep models given a large-scale database is highly time-consuming as a large amount of parameters are involved. Existing deep quantization methods often sample only a subset from the database for training, which may end up with unsatisfactory retrieval performance as a large portion of label information is discarded. To alleviate this problem, we propose a novel model called Similarity Preserving Deep Asymmetric Quantization (SPDAQ) which can directly learn the compact binary codes and quantization codebooks for all the items in the database efficiently. To do that, SPDAQ makes use of an image subset as well as the label information of all the database items so the image subset items and the database items are mapped to two different but correlated distributions, where the label similarity can be well preserved. An efficient optimization algorithm is proposed for the learning. Extensive experiments conducted on four widely-used benchmark datasets demonstrate the superiority of our proposed SPDAQ model.
Introduction
In large-scale image retrieval, the primary challenge is to identify semantically similar images within massive databases almost instantaneously. Quantization techniques address this by compressing high-dimensional features into compact binary codes. While deep learning has significantly improved retrieval accuracy, it faces a "scalability wall." Training deep networks on millions of images is computationally prohibitive because pairwise similarity calculations scale quadratically ($O(N^2)$) or even cubically ($O(N^3)$).
To mitigate this, standard practices involve training on a small random subset of the database. However, this approach discards substantial supervision (label) information, leading to sub-optimal codebooks. We introduce Similarity Preserving Deep Asymmetric Quantization (SPDAQ), a framework that utilizes all available label information while maintaining computational efficiency through an asymmetric learning strategy.
The SPDAQ Framework
The core innovation of SPDAQ lies in its asymmetric architecture, which bridges the gap between efficiency and accuracy. Unlike symmetric pipelines that treat query and database items identically, SPDAQ employs two distinct paths.
Asymmetric Architecture and Loss
As shown in Figure 1, SPDAQ processes a small subset ($M$) through a Convolutional Neural Network (CNN-F) to learn continuous, "unquantized" embeddings. Simultaneously, it directly learns discrete "quantized" embeddings—comprising binary codes and codebooks—for the entire database ($N$).
The model is optimized using a Similarity Preserving Loss based on Asymmetric Quantizer Distance (AQD). This loss ensures that the inner product between the continuous features of the subset and the discrete codes of the database items aligns with their semantic similarity derived from labels. By correlating the small subset with the large database, training complexity is reduced to $O(MN)$. Since $M \ll N$, it becomes feasible to incorporate label information from every database item.
Joint Optimization
SPDAQ optimizes a dual-objective function: a classification loss to enhance feature discriminativeness and the similarity-preserving loss. An alternating optimization algorithm is employed to navigate the discrete constraints inherent in binary coding, allowing for direct learning of codes rather than relying on post-hoc rounding.
Experimental Results
The model was evaluated on four benchmark datasets: CIFAR-10, NUS-WIDE-21, NUS-WIDE-81, and MS-COCO. SPDAQ consistently outperformed state-of-the-art deep hashing and quantization models such as DVSQ and DTQ.
Accuracy and Efficiency
On CIFAR-10, SPDAQ achieved a Mean Average Precision (mAP) of 88.4% at 8 bits, significantly higher than DVSQ’s 80.3%. A critical advantage of SPDAQ is its convergence speed. On the large-scale NUS-WIDE-21 dataset, SPDAQ converged in approximately 3.5 hours, whereas competing models like DPSH-ALL required over 10 hours while achieving lower mAP (~84% vs. SPDAQ’s ~90%).
As illustrated in Figure 2, SPDAQ (represented by red triangles) reaches a higher mAP plateau much faster than its counterparts. This demonstrates that incorporating label correlations for the entire database—even for images not directly processed by the CNN—drastically improves codebook quality.
Conclusion
SPDAQ demonstrates that asymmetry in deep quantization is a powerful tool for scalability. By "anchoring" the database's discrete codes to a high-quality continuous feature space learned from a subset, the model achieves state-of-the-art performance without the prohibitive cost of full-database CNN training. Future work may explore more modern backbone architectures and advanced discrete optimization techniques to further refine the greedy local search currently employed for binary code generation.
Summary updated