Uni-Match: A Semantic Unified Model for Query-Product Retrieval

Zhenyang Zhu, Rui-Jie Zhu, Yunrui Ge, Qihang ZhaoOpenReview 2023 · Submission to Tiny Papers @ ICLR · OpenReview

TL;DR

This paper introduces Uni-Match, a novel dual-tower model for query-product retrieval that bridges the gap between efficiency and effectiveness. By proposing a "union-interaction" mechanism, Uni-Match achieves retrieval performance close to computationally expensive cross-encoder models while maintaining the speed of dual-tower architectures, making it suitable for large-scale real-world applications.

What Problem Does This Paper Address?

Modern search systems often employ a cascaded architecture, with a "prerank" stage responsible for quickly filtering down potential results before a more computationally intensive "rank" stage. The dominant approach for preranking is the dual-tower model, which encodes queries and documents into separate embeddings offline, allowing for fast online retrieval. However, this separation limits the model's ability to capture nuanced interactions between queries and documents, leading to a trade-off between efficiency and effectiveness. Existing methods that try to improve effectiveness by introducing query-document interaction at this stage (e.g., token-level or sentence-level interactions) often become too slow for practical preranking. This paper aims to achieve the best of both worlds: the efficiency of dual-tower models and the effectiveness of more complex interaction-based models.

Methods

The core innovation of Uni-Match lies in its union-interaction mechanism. This mechanism takes the output embeddings from separate query and document encoders (both based on 6-layer BERT) and unifies them into a shared semantic space. Specifically, it uses an attention mechanism to process the token-level representations of both the query and the document, effectively learning both token-level and sentence-level semantic matching information simultaneously. This is achieved by mapping query and document representations to a common space, allowing for a more comprehensive understanding of their relationship.

To further enhance learning, the authors employ a two-stage curriculum training strategy:

  1. Stage 1 (Easy to Difficult): Listwise InfoNCE loss is used to train the model to learn ranking capabilities, focusing on relative orderings of documents.
  2. Stage 2 (Fine-tuning): Pointwise MSE loss is applied to train the model to learn precise scoring abilities.

Datasets

The paper evaluates Uni-Match on two datasets:

  • kuai_search Dataset: A custom dataset constructed with 120,000 samples, derived from the scoring results of a refined ranking model and supplemented with manual annotations. It has a positive to negative sample ratio of 51%:49%.
  • MS MARCO Dataset: A widely used dataset for passage retrieval.

The paper also mentions that Uni-Match has been deployed in the search system of a shortvideo App, serving millions of users daily.

Key Insights and Ideas

  • Bridging the Efficiency-Effectiveness Gap: The union-interaction mechanism is a clever way to inject interaction information without the prohibitive computational cost of full cross-encoder models. It suggests that a well-designed interaction within a dual-tower framework can significantly boost performance.
  • Unified Semantic Space: The idea of mapping query and document representations into a single semantic space for matching is a key enabler for the model's effectiveness. This allows for a more direct comparison of their semantic content.
  • Curriculum Learning for Complex Matching: The two-stage curriculum training (listwise then pointwise) is crucial for effectively learning the complex semantic relationships required for accurate retrieval. This approach guides the model from broader ranking understanding to finer scoring.
  • Real-World Impact: The successful deployment of Uni-Match in a large-scale shortvideo app highlights its practical viability and the significant business impact it can have.

Key Results

Uni-Match demonstrates strong performance across both offline experiments and online A/B tests:

  • Offline Performance:
    • On the kuai_search dataset, Uni-Match achieved an AUC of 72.74 and a PNR of 1.72, outperforming other dual-tower baselines and coming very close to the interaction-based "Cross Match" model, while being significantly faster (22 ms vs. 127 ms).
  • On the MS MARCO dataset, Uni-Match achieved an AUC of 81.17 and a PNR of 2.11, again surpassing dual-tower baselines and approaching the performance of "Cross Match" with a much lower inference time (15 ms vs. 89 ms).
  • Online A/B Test: The model led to significant improvements in key business metrics:
  • Gross Merchandise Volume (GMV): +2.261%
    • Volume of Order: +5.826%
    • User Order Completion: +1.881%

Limitations and Future Work

  • Limitations: While Uni-Match significantly reduces the effectiveness gap with cross-encoder models, it doesn't fully close it. The paper notes that interaction-based models still achieve slightly higher AUC in some cases, though at a much higher computational cost. The paper doesn't explicitly discuss limitations of the union-interaction mechanism itself, but the inherent trade-off between interaction depth and computational cost remains a general challenge.
  • Future Work: The paper implicitly suggests that further exploration of interaction mechanisms within dual-tower frameworks could lead to even greater gains. The success in a real-world shortvideo search scenario also opens avenues for applying this model to other recommendation and retrieval tasks.

Summary updated

Abstract

For most practical search systems, the cascaded matching-prerank-rank architecture is designed. In the prerank stage, the dual-tower structure is widely used to maintain efficiency. However, due to the lack of interaction between query and document, this architecture could only take into account efficiency but not both effectiveness and efficiency. Inspired by this, we propose a simple but effective dual-tower model: uni-match, which has the efficiency of dual-tower model and the effectiveness close to that of cross model. Sufficient offline and online experiments show the effectiveness of our proposed method. Currently, the Uni-match model has been deployed in the search system of a shortvideo App, providing daily services to hundreds of millions users.
BibTeX
@misc{zhu2023uni,
  author = {Zhenyang Zhu and Rui-Jie Zhu and Yunrui Ge and Qihang Zhao},
  title = {{Uni-Match: A Semantic Unified Model for Query-Product Retrieval}},
  year = {2023},
  url = {https://openreview.net/forum?id=91Bcj6sgcxt},
  howpublished = {OpenReview},
  note = {Submitted to Tiny Papers @ ICLR},
}
Figure