TL;DR
MAKE is a specialized vision-language pre-training framework designed to bridge the gap between short user queries and complex, multimodal product data in e-commerce search. By introducing dynamic modal weighting and a novel keyword enhancement mechanism, it significantly improves retrieval relevance and has been successfully deployed to serve hundreds of millions of users on Taobao.
What Problem Does This Paper Address?
In e-commerce platforms like Taobao, search happens in two stages: retrieval (finding a few thousand candidates from billions) and ranking (sorting those candidates). This paper focuses on the retrieval phase, which faces three major hurdles:
- Modal Importance Variance: The importance of a product's title versus its image changes depending on the category. For a "white shirt," the image is paramount; for a "laptop," technical specs in the title (like RAM or CPU) are more critical.
- Semantic Imbalance: There is a massive mismatch between user queries and product titles. Users type short, simple phrases, while sellers "stuff" titles with keywords for Search Engine Optimization (SEO), leading to long, grammatically messy strings.
- False Negatives in Training: Standard contrastive learning (like CLIP) uses "in-batch negative sampling," where any query not explicitly paired with a product in a batch is treated as a negative example. In e-commerce, however, many different queries (e.g., "white shirt" and "long-sleeved shirt") might actually be relevant to the same product, leading the model to learn incorrect distinctions.
Methods
The authors propose a three-tower architecture called MAKE (Vision-Language Pre-training based Product Retrieval).
Key Innovations:
- Three-Tower Structure: Unlike standard two-tower models (one for text, one for image), MAKE uses separate encoders for the User Query, the Product Title, and the Product Image. This allows the model to handle the distinct linguistic styles of queries versus SEO-heavy titles.
- Modal Adaptation (MA) Module: This module uses the user query as context to dynamically weight the importance of the title and image. It uses cross-attention to decide, for a specific search, whether the visual or textual features of a product should carry more weight.
- Keyword Enhancement (KE) Mechanism: To solve the false-negative problem, the authors group multiple queries related to the same product into a single training sample. They replace standard contrastive loss with a modified version based on Circle Loss, which helps the model narrow the distance between similar queries in the embedding space.
- Pre-training Objectives: The model is trained using a combination of Masked Language Modeling (MLM), Mask Patch Modeling (MPM), Query-Product Matching (QPM), and Query-Product Classification (QPC).
Datasets
The model was trained and evaluated on massive, real-world industrial data:
- Taobao Industrial Dataset: A billion-level dataset of (query, title, image) triplets collected from Taobao Search logs.
- Evaluation Set: 1.5 million search logs were randomly selected for offline testing.
- Benchmarks: The authors compared MAKE against established V+L models including CLIP, FashionBERT, and CommerceMM.
Key Insights and Ideas
- Contextual Weighting is Key: The MA module proves that "one size does not fit all." As shown in Figure 2, the model learns to prioritize images for visual categories like ornaments (79.4% weight) and titles for technical categories like laptops (67.5% weight).
- Query-Title Separation: Treating user queries and product titles as different "languages" (via separate encoders) is more effective than using a single shared text encoder, due to the SEO-driven noise in product titles.
- Debiasing Popularity: The loss function includes a term ($-\log p$) to account for product frequency, preventing the model from over-focusing on a few extremely popular items at the expense of the "long tail."
Key Results
MAKE demonstrated significant gains over state-of-the-art baselines in both offline and online environments.
Offline Performance: As shown in the table below, MAKE outperformed all baselines across relevance ($P_{rel}$), category accuracy ($P_{cate}$), and Recall@K.
Online Impact: In A/B tests on Taobao Search, MAKE achieved:
- +0.79% increase in Gross Merchandise Volume (GMV).
- +2.20% improvement in overall product relevance.
- +2.01% GMV gain specifically for inactive and new users, suggesting the model is particularly good at understanding intent when historical user data is sparse.
- Efficiency: The additional computational cost was a negligible 2ms.
Limitations and Future Work
The paper does not explicitly detail limitations, but a few are inherent to the approach:
- Data Dependency: The Keyword Enhancement mechanism relies heavily on high-quality click logs to group related queries.
- Update Frequency: The model is currently updated on a weekly basis; moving toward more real-time updates for rapidly changing e-commerce trends could be a future direction.
- Cold Start: While it helps new users, the paper does not deeply explore the "cold start" problem for brand-new products that lack sufficient click history for the KE mechanism.
Summary updated