Introduction
In large-scale industrial search engines like Taobao, users frequently utilize image queries to find products. While image-to-image retrieval is a mature field, real-world product listings are multi-modal, containing both high-resolution images and descriptive text. This task, known as Image-to-Multi-Modal Retrieval (IMMR), presents three significant challenges: data noise in user click logs, information inequality between visual and textual modalities, and the immense scale of industrial datasets containing millions of categories.
To address these hurdles, researchers from Alibaba and Shanghai Jiao Tong University have introduced OCLEAR (Organizing Categories and Learning by Classification for Retrieval), a framework that shifts retrieval from traditional pair-based metric learning to a massive-scale classification paradigm.
The OCLEAR Framework
The OCLEAR framework rests on three primary pillars designed to improve data quality, fusion efficiency, and training scalability.
Category-Oriented Data Governance
Industrial data is often noisy and skewed toward popular items. OCLEAR employs a data governance strategy that organizes billions of images into millions of fine-grained categories (unique product IDs). By merging data from three sources—inherent product metadata, user click behavior, and unsupervised visual clustering—the model becomes robust to the inherent randomness and errors in human behavior.
Concept-Aware Modality Fusion (CAMF)
A core innovation of OCLEAR is the Concept-Aware Modality Fusion module. In IMMR, images provide specific visual details while text offers high-level abstract concepts. Standard fusion methods often fail to balance these, leading to abstract text "washing out" specific visual details.
OCLEAR uses a dual-tower architecture where the document side extracts textual "concepts" using external attention. These concepts then act as the Query (Q) in a cross-attention mechanism, where image features serve as the Key (K) and Value (V). This ensures the image remains the primary signal while the text provides complementary context.
Scaling via Hybrid Parallel Training
Training on tens of millions of categories requires significant computational resources. OCLEAR utilizes a hybrid parallel training approach, splitting the classification layer (ID center proxy vectors) across multiple GPUs. To minimize the massive communication overhead between these units, the framework implements KNN-softmax, which calculates gradients only for the top-K most similar categories for any given sample, significantly accelerating convergence.
Experimental Results
OCLEAR was evaluated on the AliProduct dataset (12 million products, 1.1 billion images) and the Taobao Live (WAB) dataset.
- State-of-the-Art Performance: OCLEAR achieved 78.41% Identical@1 accuracy on AliProduct, significantly outperforming established models like CLIP (71.68%) and Oscar (69.03%).
- Ablation Success: Tests confirmed that the multi-modal fusion module provides a significant boost in Mean Average Precision (MAP) compared to image-only baselines.
- Industrial Impact: Following deployment on Taobao’s image search system, an A/B test showed a +6.45% increase in actual purchases, proving that superior representation learning translates directly to business growth.
Conclusion
OCLEAR demonstrates that shifting from triplet-based retrieval to a massive-scale classification paradigm, combined with concept-aware multi-modal fusion, can significantly improve retrieval performance in large-scale industrial applications. Future work will focus on automated methods to handle long-tailed data distributions during the training process itself.
Summary updated