TL;DR
Generative retrieval relies on "Semantic IDs" (SIDs) to find documents, but traditional ways of creating these IDs ignore the query, creating a bottleneck for recall. This paper introduces CARE, a framework that aligns the SID quantization process with actual retrieval needs using contrastive learning and listwise distillation, significantly improving search accuracy in industrial video retrieval.
What Problem Does This Paper Address?
In modern search, Generative Retrieval (GR) is a rising alternative to traditional nearest-neighbor searches. Instead of comparing vectors, a large language model (LLM) "writes" the unique identifier—a Semantic ID (SID)—of a relevant document one digit at a time.
The problem lies in how these SIDs are created (tokenized). Standard methods like residual k-means or RQ-VAE focus entirely on compressing document embeddings so they can be reconstructed perfectly. However, they ignore the query. If the quantization process discards the subtle features that distinguish a relevant document from a "hard negative" for a specific query, the retriever can never recover that lost information. The authors aim to bridge this "quantization gap" by making SID construction retrieval-oriented.
Methods
The authors propose two sequential tokenizers:
- CARE-SID (Contrastive-Anchored Residual Encoding):
- Retrieval Alignment: Before quantization, document embeddings pass through a "residual alignment network" trained with a query-document contrastive loss (InfoNCE). This moves documents closer to the queries that should find them.
- Anchoring: To prevent the model from "collapsing" (assigning everyone the same ID) or drifting too far from the original semantic meaning, they use an anchor loss and keep the original embedding as the reconstruction target.
- R3-VAE Kernel: It uses a sophisticated "rating residual quantization" that selects codewords based on cosine similarity rather than just Euclidean distance.
- CARE-LR (CARE with Listwise Reconstruction):
- Ranking Preservation: Even if a document's reconstruction error is small, its rank relative to other documents might change after quantization. CARE-LR addresses this by distilling the similarity distribution of a candidate list (from the original embeddings) into the SID reconstructions.
- Layer-Resolved Head: They introduce a novel reconstruction head that can distinguish which "digit" of the SID a correction comes from, allowing for more precise fine-tuning of the codebooks without changing the document-only encoding interface.
Datasets
The researchers evaluated their methods using industrial video search data from Tencent:
- Corpus: Over 1 million document (video) rows.
- Queries: 2,990 evaluable queries categorized into four segments: General, Long-tail, High-click, and High-exposure.
- Embeddings: They tested on both a general multimodal embedding and a specialized search-tuned retrieval embedding.
Key Insights and Ideas
- The "Recall Bottleneck": The SID tokenizer sets the upper bound for what a generative retriever can possibly find. If the tokenizer fails, the LLM retriever fails, no matter how powerful it is.
- Anchoring is Non-Negotiable: The authors found that without the "anchor loss" to the original embedding, contrastive training quickly leads to "bucket merging," where distinct documents are forced into the same SID, destroying retrieval precision.
- Embedding Quality vs. Quantizer: A major "aha" moment was discovering that the quality of the input embedding affects recall far more than the choice of quantizer. However, contrastive alignment (CARE-SID) provides the biggest gains when the base embedding hasn't already been tuned for search.
- Listwise > Pointwise: Moving from per-document reconstruction (CARE-SID) to listwise distillation (CARE-LR) ensures that the relative order of search results is preserved even after the data is squashed into discrete tokens.
Key Results
- Baseline Comparison: CARE-SID improved "reconstruction-view" Recall@10 from 62.75% (standard residual k-means) to 65.23%.
- Distillation Boost: CARE-LR pushed this further to 66.09%, a total gain of 3.34 points over the baseline.
- Cold-Start Embeddings: On general embeddings not yet tuned for search, CARE-SID outperformed the R3-VAE baseline by a significant 2.1 to 3.4 points across various recall metrics.
- Deployment: CARE-SID is currently deployed in the generative retrieval stage of a large-scale industrial video search system.
Limitations and Future Work
- Scale of CARE-LR: While CARE-SID is at production scale, the CARE-LR results are currently from a "small-scale pilot" (using 1 million search-funnel lists). The authors plan to scale this up in future iterations.
- End-to-End Latency: The paper focuses on SID-level recall (which is faster to iterate on). While they note that SID-level metrics correlate with end-to-end GR performance, full end-to-end results with the LLM generator are promised for a later version of the paper.
- Integer Constraints: The current reconstruction view uses continuous coefficients that aren't part of the final integer SID. The authors aim to let the listwise signal eventually "reshape" the discrete codes themselves.
For more details, see the full paper: "CARE: Contrastive-Anchored Residual Encoding with Listwise Reconstruction for Retrieval-Oriented Semantic IDs" (arXiv:2611.20721v1).
Summary updated