ESAM: Discriminative Domain Adaptation with Non-Displayed Items to Improve Long-Tail Performance

Zhihong Chen, Rong Xiao, Chenliang Li, Gangfeng Ye, Haochuan Sun, Hongbo DengSIGIR 2020 · DOI

Introduction: The Challenge of Sample Selection Bias

Modern recommendation and search systems are often plagued by the "Matthew Effect," where popular items become increasingly dominant while niche or new items are ignored. This phenomenon is largely driven by Sample Selection Bias (SSB). Ranking models are typically trained on implicit feedback (clicks, purchases) from items that were previously displayed to users. However, at inference time, the model must evaluate the entire item space, most of which consists of non-displayed, long-tail items.

Sample Selection Bias and its impact on long-tail performance.

As shown in the figure above, there is a stark contrast between displayed and non-displayed items: approximately 82% of displayed items are "hot," whereas 85% of non-displayed items belong to the "long-tail." Because the training data is skewed toward popular items, the model fails to learn robust representations for the long-tail, leading to significantly lower performance (e.g., NDCG@20) as item frequency decreases.

The ESAM Framework: Bridging the Domain Gap

To address this, the authors propose the Entire Space Adaptation Model (ESAM). ESAM treats displayed items as a labeled source domain and non-displayed items as an unlabeled target domain. By framing the problem as Unsupervised Domain Adaptation (DA), ESAM aligns the feature distributions of these two domains without requiring additional labels.

The framework introduces three key constraints to the item feature extractor:

  1. Attribute Correlation Alignment (A2C): ESAM aligns the covariance of high-level features between domains. This is based on the insight that correlations between item attributes (e.g., brand and price) are invariant across popular and long-tail items.
  2. Center-Wise Clustering: To create a discriminative feature space, ESAM pulls items with similar feedback closer to a class center and pushes those with different feedback apart.
  3. Self-Training for Target Clustering: To avoid "negative transfer," ESAM uses high-confidence predictions on non-displayed items as pseudo-labels, anchoring the target domain correctly.
Evolution of the feature space alignment as different loss components are added.

The figure above illustrates how these components work together. Starting from a simple supervised loss ($L_s$), adding domain alignment ($L_{DA}$) and discriminative clustering ($L_{DC}$) progressively aligns the source and target distributions while maintaining a clear separation between classes (e.g., clicked vs. non-clicked).

Experimental Results and Real-World Impact

ESAM was evaluated on public benchmarks (MovieLens-1M, CIKM Cup 2016) and a massive Taobao dataset containing 11.9 billion records.

  • Performance Gains: On the CIKM dataset, ESAM improved long-tail NDCG@20 by 3.3%. Notably, it doubled the cold-start performance of the baseline BST model.
  • Industrial Deployment: In a live A/B test on Taobao's search engine, ESAM delivered a +0.85% CTR and a +0.4% GMV increase. Given Taobao's scale, these improvements translate into significant revenue growth.

While highly effective, ESAM's performance is sensitive to the confidence thresholds used for pseudo-labeling. If these thresholds are too loose, the model may ingest false labels, leading to performance degradation. Additionally, processing non-displayed items increases the training footprint.

Conclusion

ESAM provides a scalable, "plug-and-play" solution for the Matthew Effect in recommendation systems. By leveraging non-displayed items through domain adaptation and attribute correlation alignment, it ensures that long-tail items receive fair and accurate representation, ultimately improving both user engagement and platform revenue.

Summary updated

BibTeX
@inproceedings{chen2020esam,
  author = {Zhihong Chen and Rong Xiao and Chenliang Li and Gangfeng Ye and Haochuan Sun and Hongbo Deng},
  title = {{ESAM: Discriminative Domain Adaptation with Non-Displayed Items to Improve Long-Tail Performance}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2020},
  doi = {10.1145/3397271.3401043},
}
Figure