Introduction
Cross-modal hashing is a critical technique for searching massive multimedia databases, as it compresses high-dimensional data into compact binary codes to enable rapid retrieval. However, traditional methods typically rely on a "strict assumption": the availability of a training set where different modalities (e.g., images and text) are already linked or paired. In real-world scenarios, such direct supervision is often unavailable. For instance, one might wish to use text queries from a platform like YahooQA to search for images in ImageNet, despite no pre-existing links between these specific datasets.
This paper addresses the transitive retrieval problem, where the goal is to learn hashing across modalities using an auxiliary dataset (e.g., Flickr) as a "bridge." This auxiliary data contains image-text pairs, but follows a different distribution than the target query and database domains.
Methodology: The Transitive Hashing Network (THN)
The authors propose the Transitive Hashing Network (THN), an end-to-end deep learning architecture designed to construct a transitive link between heterogeneous domains. The network consists of an image branch (based on AlexNet) and a text branch (a three-layer MLP), integrated through three core modules:
- Heterogeneous Relationship Learning: The model learns cross-modal correlations from the auxiliary dataset using a pairwise cross-entropy loss. This ensures that semantically similar image-text pairs result in similar hash codes.
- Homogeneous Distribution Alignment: To combat "dataset shift" between the auxiliary and target domains, THN incorporates a Maximum Mean Discrepancy (MMD) module. This aligns the distributions in the Hamming space, ensuring that learned correlations transfer effectively to the target data.
- Quantization Error Minimization: A novel cross-entropy prior is introduced to force neural network activations toward binary values (-1 or 1) during training. This minimizes the semantic gap typically created during binarization.
Experimental Evaluation
The researchers evaluated THN on two major benchmarks: NUS-WIDE (standard cross-modal retrieval) and ImageNet-YahooQA (a custom transitive benchmark). In the transitive task, NUS-WIDE served as the auxiliary bridge.
Key Results
THN consistently outperformed state-of-the-art methods, including deep models like DCMH:
- NUS-WIDE: Achieved a relative increase in Mean Average Precision (MAP) of 9.47% for Image → Text and 2.85% for Text → Image retrieval compared to previous bests.
- ImageNet-YahooQA: The gains were more significant, with relative MAP increases of 5.03% and 6.91%, respectively.
- Robustness: Precision-Recall curves demonstrate that THN maintains high precision even at high recall levels, proving its effectiveness across diverse retrieval scenarios.
Conclusion and Future Work
The success of THN highlights that direct supervision is not mandatory for cross-modal retrieval if a bridge dataset is available and distribution shifts are properly managed via alignment modules like MMD. Future work may involve scaling these techniques to noisier social media data or extending the framework to handle more than two modalities simultaneously.
Summary updated