Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching

Tao Feng, Jie Zhang, Huashan Liu, Zhijie Wang, Shengyuan PangICASSP 2023

Abstract

Deep Hashing Retrieval (DHR) is a cornerstone of modern large-scale image search, yet the computational cost of training on massive datasets remains a significant hurdle. This paper presents Information-intensive Feature-Embedding Matching (IEM), a novel dataset condensation method tailored for retrieval tasks. Unlike traditional methods that focus on classification, IEM preserves the fine-grained feature distributions necessary for effective hashing. By employing network and dataset augmentations, IEM creates a compact set of synthetic images that maintain high retrieval accuracy while drastically reducing training overhead. Experimental results on CIFAR-10 and ImageNet demonstrate that IEM significantly outperforms existing state-of-the-art condensation techniques.

Introduction

Deep Hashing Retrieval (DHR) works by converting images into compact binary "hash codes," allowing systems to find similar images by calculating simple Hamming distances. However, as datasets grow into the millions, training these models becomes a bottleneck due to complex optimization objectives and high hardware demands.

While Dataset Condensation (DC) has been successful in image classification—creating a handful of synthetic images that represent an entire category—it hasn't been effectively applied to retrieval. Retrieval is inherently more difficult than classification: a model doesn't just need to identify a category; it must understand fine-grained feature distributions to distinguish between different instances within that category. Existing DC methods, such as gradient matching, often fail in this context because the optimization landscape of hashing is significantly more volatile than standard classification.

Methodology: Information-intensive Feature-Embedding Matching (IEM)

The authors propose IEM, which shifts the focus from matching gradients to Distribution Matching (DM) in the feature-embedding space. The core objective is to ensure that synthetic images produce a feature distribution that closely mimics the real dataset. To make this effective for retrieval, IEM introduces two key augmentations:

  1. Network Augmentation (NA): Instead of matching distributions using a single static network, IEM applies random weight perturbations to the initial model. This forces the synthetic data to capture features that are robust across various model initializations, preventing the synthetic set from overfitting to a specific starting point.
  2. Dataset Augmentation (DA): To pack maximum information into a tiny synthetic set, the authors use a "multi-formation" strategy. They resize multiple images from the same class and tile them together into a single synthetic image. This approach ensures the model encounters a wider variety of visual features even when the number of training images is extremely low.
The IEM framework overview, illustrating the multi-formation strategy and distribution matching process.

The final training involves a loss function ($L_{IEM}$) that minimizes the discrepancy between the mean feature embeddings of the real and synthetic sets.

Experimental Evaluation

The researchers evaluated IEM on standard benchmarks, condensing them to tiny fractions of their original size (e.g., 1, 10, or 50 images per class):

  • CIFAR-10: A classic low-resolution dataset.
  • ImageNet-subset (ImageNet10 & ImageNet20): High-resolution subsets of the ImageNet dataset, used to test scalability to complex visual data.

Key Results and Discussion

IEM demonstrates superior performance across multiple metrics and datasets. On CIFAR-10, when condensed to just 50 images per class (a 10% ratio), IEM achieved a 57.49% mAP (mean Average Precision), significantly outperforming the previous best method, IDC, which reached 46.21%. On ImageNet10, with only 3 images per class, IEM reached 52.79% mAP, nearly tripling the performance of basic random sampling (21.22%).

Performance comparison of various condensation methods on classification and hashing retrieval tasks across different image counts per class.

Beyond accuracy, IEM offers substantial efficiency gains. It reaches convergence much faster than competitors like IDC. Furthermore, the synthetic data created via IEM generalizes well across different hashing architectures, including DHD, DPN, and OrthoHash.

Conclusion

IEM represents a significant leap forward for condensed retrieval, proving that matching feature distributions is more effective than gradient matching for complex hashing tasks. While a performance gap still exists between synthetic sets and the full dataset (e.g., 57% vs 70% on CIFAR-10), IEM provides a practical path for real-world engineering pipelines. Future work may explore adaptive condensation—allocating more synthetic images to complex classes—and expanding the method to cross-modal retrieval tasks, such as text-to-image search.

Summary updated

BibTeX
@inproceedings{feng2023towards,
  author = {Tao Feng and Jie Zhang and Huashan Liu and Zhijie Wang and Shengyuan Pang},
  title = {{Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
}
Figure