Abstract
Researchers from JD.com have developed a "hyper-parameter-free" approach to deep semantic retrieval that adaptively adjusts loss function parameters (margin and temperature) for every query-item pair. By replacing fixed global constants with a heuristic based on batch similarity and adding a symmetric metric learning objective to prevent model collapse, they achieved significant performance gains on a 5-billion-sample industrial dataset and successfully deployed the system to a major e-commerce search engine.
Introduction
In modern e-commerce, semantic retrieval is essential for understanding user intent beyond simple keyword matching. State-of-the-art systems typically utilize dual-encoder architectures (e.g., DSSM or BERT) trained with contrastive losses like margin loss or softmax loss. However, these losses depend on two sensitive hyper-parameters: the margin ($\delta$), which controls the separation between positive and negative items, and the temperature ($\tau$), which adjusts the sharpness of the probability distribution.
Traditionally, these are set as global constants via expensive grid searches. This paper argues that a one-size-fits-all approach is flawed because different queries—ranging from general terms like "shoes" to precise specifications like "Nike Air Max 270 Men's Size 10"—possess varying levels of ambiguity and require distinct learning signals.
Proposed Methods
The authors propose a two-pronged solution to make these hyper-parameters adaptive and "free" from manual tuning.
Adaptive Heuristic Method
Instead of fixed values, the model calculates $\delta$ and $\tau$ on the fly. For a given query-item pair, the margin is determined by the similarity between the positive item and the negative items in the current batch. When a negative item is "hard" (highly similar to the positive one), the model adaptively reduces the margin/temperature to focus the learning signal. A stop-gradient operation is employed to ensure that the adaptive margin itself does not destabilize the optimization process.
Symmetric Metric Learning
A significant risk in adaptive learning is "model collapse," where embeddings cluster into a single point to minimize loss. To counter this, the authors introduce a symmetric loss term. While standard retrieval treats the query as the anchor to find items, this term treats the item as the anchor to find the query. This dual-view alignment ensures the embedding space remains well-distributed.
Experimental Evaluation
The study utilizes a massive industrial dataset from JD.com search logs, comprising 5 billion samples over 60 days. Evaluation includes offline metrics such as Recall@K and batch-top@K, alongside online A/B testing tracking conversion rates (UCVR) and revenue per visitor (UV-value).
Key Results
The proposed method was tested against strong baselines, including DSSM, DPSR, and RSR. Using a BERT backbone, the adaptive method significantly outperformed existing models.
- Offline Performance: The adaptive method improved Recall@1 from 0.0094 to 0.0137 (a ~45% relative increase) and showed consistent gains across Recall@50, 500, and 1000.
- Sensitivity Analysis: The authors demonstrated that even minor changes in fixed margins (e.g., 0.1 to 0.5) can drastically degrade performance, highlighting the robustness of the adaptive approach.
- Online Impact: In live A/B tests, the method increased the number of items passing to the ranking phase by 2.0% and improved the conversion rate (UCVR) by 0.45%.
Key Insights and Discussion
- Hardness Correlation: Optimal margins are inversely related to the "hardness" of negative samples. By tying hyper-parameters to batch similarities, the model automatically prioritizes difficult distinctions.
- Symmetry as a Regularizer: Symmetric loss is not merely a performance booster but a necessary safeguard. Without it, adaptive components are prone to collapsing the metric space, as shown in the ablation studies.
- Agnosticism: The method succeeds in retrieval because it does not rely on pre-trained user/item biases, but rather on immediate geometric relationships within the training batch.
Conclusion
This research presents a scalable, adaptive framework for semantic retrieval that eliminates the need for manual hyper-parameter tuning while improving both offline accuracy and online business metrics. Future work will explore scaling this logic beyond batch-level similarities and integrating it with multi-objective learning for industrial ranking systems.
Summary updated