Abstract
Unsupervised hashing is a cornerstone of large-scale image retrieval, enabling fast searches in high-dimensional spaces by mapping features to short binary strings. While much research focuses on optimizing the hashing algorithms themselves, the discriminative power of the input features is often overlooked. We introduce Gemb (Gaussian Mixture Model embedding), a straightforward yet powerful pre-processing pipeline designed to enhance feature discrimination. By transforming features into GMM posterior probabilities and applying power normalization, Gemb significantly improves the performance of traditional hashing methods like Iterative Quantization (ITQ), even surpassing several sophisticated deep learning-based hashing frameworks.
Introduction
In the era of massive visual databases, efficient retrieval is paramount. Unsupervised hashing maps high-dimensional image descriptors—such as GIST or CNN activations—into compact binary codes, allowing for near-instantaneous searches via Hamming distance. However, a fundamental challenge remains: if the initial feature vectors for semantically distinct objects are similar, the resulting hash codes will inevitably collide, leading to poor retrieval precision.
Most existing literature treats input features as fixed and focuses on the quantization process. In contrast, this work proposes that an explicit embedding step to enhance feature discrimination is a "missing link" in the unsupervised hashing pipeline. We present Gemb as a solution to bridge this gap.
Methodology: The Gemb Pipeline
The Gemb framework consists of three sequential stages designed to refine raw features before they are passed to a hashing algorithm.
1. Dimensionality Reduction (PCA)
The process begins with Principal Component Analysis (PCA) applied to the raw input features (e.g., 4096-D VGG-FC7 vectors). This step serves multiple purposes: it reduces noise, eliminates redundancy, and lowers the computational burden for the subsequent statistical modeling.
2. GMM Embedding
The core of Gemb lies in the use of a Gaussian Mixture Model (GMM). Rather than using the PCA-reduced vectors directly, we fit a GMM to the feature distribution. For each image, we calculate the posterior probability—the likelihood that the feature belongs to each of the $N$ Gaussian components. This transforms the feature into an $N$-dimensional vector of cluster memberships. By forcing the representation to reflect cluster assignments, Gemb effectively groups semantically similar features together in the new embedding space.
A key finding is that using "full" covariance matrices in the GMM, which capture correlations between dimensions, is superior to the simpler "diagonal" matrices often used in other embedding methods like Fisher Vectors, particularly when the number of clusters is small.
3. Power Normalization
As the number of Gaussian components ($N$) increases, the resulting posterior probability vectors become increasingly sparse. Standard hashing algorithms, which often rely on $l2$-distance similarity, perform poorly on highly sparse data. To mitigate this, we apply a power normalization function: $f(z) = \text{sign}(z)|z|^\alpha$. This "unsparsifies" the data, making the features more robust for the final quantization step.
Experimental Evaluation
The effectiveness of Gemb was evaluated on three benchmark datasets: CIFAR-10, LabelMe-12-50k, and MNIST, using both hand-crafted (GIST) and deep (VGG-FC7) features.
Performance Gains
The empirical results demonstrate a substantial performance boost across the board. For instance, on the MNIST dataset using GIST features, standard ITQ achieves a mean Average Precision (mAP) of 46.37%. Integrating Gemb (Gemb+ITQ) elevates this to 79.97%.
Furthermore, Gemb allows traditional hashing methods to compete with, and often beat, modern deep hashing architectures. On CIFAR-10, Gemb+ITQ using GIST features achieved an mAP of 21.14%, outperforming dedicated deep methods like Deep Hashing (DH, 16.17%) and UH-BDNN.
Parameter Sensitivity and Analysis
The performance of Gemb is influenced by hyperparameters such as the number of Gaussians ($N$) and the PCA dimensions ($D$). The authors utilized the Bayesian Information Criterion (BIC) to analyze the trade-offs and select optimal configurations for different datasets.
Additionally, t-SNE visualizations confirm that Gemb processing results in clusters for different classes that are significantly more distinct and well-separated compared to raw descriptors, directly contributing to the observed performance gains.
Conclusion
Gemb demonstrates that enhancing the discriminative power of input features is as critical as the hashing algorithm itself. By leveraging GMM posteriors and power normalization, Gemb provides a simple, modular, and highly effective way to improve unsupervised hashing performance. This approach suggests that explicit feature embedding is a vital component for building efficient and accurate large-scale image retrieval systems.
Summary updated