Real-time Attention Based Look-alike Model for Recommender System

Yudan Liu, Kaikai Ge, Xu Zhang, Leyu LinKDD 2019 · DOI · arXiv

Abstract

Modern recommender systems often suffer from the "Matthew effect," where popular items dominate exposure while niche or long-tail content remains undiscovered. This paper introduces the Real-time Attention Based Look-alike Model (RALM), a novel framework designed for efficient audience extension. RALM utilizes an Attention Merge Layer to capture nuanced user profiles and dual attention units (Global and Local) to refine seed user representations. By integrating K-means clustering for computational efficiency, the model achieves real-time performance on massive production datasets. Empirical results from the WeChat "Top Stories" platform demonstrate that RALM significantly improves content exposure (+9.11%), CTR (+1.09%), and topic diversity (+15.94%) while reducing the Gini coefficient, indicating a more equitable recommendation distribution.

1. Introduction

In large-scale recommender systems, deep learning models heavily rely on historical interaction data. This leads to the "Matthew effect," where items with high initial engagement receive disproportionate future exposure, while new or long-tail content struggles to gain traction. Look-alike modeling, or audience extension, addresses this by identifying potential users who share characteristics with a small "seed" group that has already interacted with the content.

However, traditional look-alike models often struggle with the real-time demands of high-traffic news feeds or fail to capture the multi-field interests of diverse user bases. RALM is proposed to overcome these limitations by providing a high-capacity, real-time architecture for effective audience extension.

2. Methodology

The RALM framework consists of two primary learning phases supported by an efficiency-focused clustering mechanism.

2.1 User Representation Learning

Standard architectures often concatenate multi-field user features (e.g., demographics, interests), which can lead to overfitting on dominant features while ignoring subtler signals. RALM introduces an Attention Merge Layer that dynamically weights different feature fields. This ensures the model captures a comprehensive and personalized user profile by looking at the "whole" user rather than just the most obvious tags.

2.2 Look-alike Learning

To compare a target user against a set of seeds, RALM employs two specialized attention mechanisms:

  • Global Attention Unit: A self-attention mechanism that identifies representative seeds and filters out noisy users within the seed set.
  • Local Attention Unit: An adaptive mechanism that identifies which specific seeds are most relevant to the target user being evaluated.
Figure 3: Detailed architecture of RALM, illustrating the User Representation Learning phase with the Attention Merge Layer and the Look-alike Learning phase with dual attention units.

2.3 Real-time Efficiency via Clustering

To maintain real-time performance with millions of seed users, RALM uses asynchronous K-means clustering to group seeds into a small number of clusters ($k=20$). This reduces computational complexity from millions of operations to a manageable few dozen, effectively smoothing out outliers and making the representation more robust.

3. Experimental Results

3.1 Offline Evaluation

RALM was evaluated using traffic logs from WeChat "Top Stories," involving 2.98 million users and 8.89 million samples. In offline tests, RALM achieved an AUC of 0.6012, outperforming the YouTube DNN (0.5903) and significantly surpassing traditional Yahoo Look-alike baselines. The effectiveness of the Attention Merge Layer is particularly evident when compared to standard concatenation methods.

Figure 5: Training performance comparison showing that the Attention Merge Layer achieves lower loss and higher AUC compared to standard feature concatenation.

3.2 Online A/B Testing

During a one-month live trial on WeChat, RALM delivered substantial improvements across key metrics:

  • Exposure: +9.11%
  • CTR: +1.09%
  • Tag Diversity: +15.94%
  • Gini Coefficient: +5.36% (indicating reduced popularity bias)

4. Conclusion

RALM provides a scalable and effective solution for the Matthew effect in recommender systems. By combining dynamic feature weighting with dual-attention look-alike learning and efficient clustering, the model successfully surfaces long-tail content to relevant audiences in real-time. Future work will focus on improving cluster update fluidity for hyper-viral content and enhancing cold-start capabilities for items with zero initial seeds.

Summary updated

BibTeX
@inproceedings{liu2019real,
  author = {Yudan Liu and Kaikai Ge and Xu Zhang and Leyu Lin},
  title = {{Real-time Attention Based Look-alike Model for Recommender System}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2019},
  doi = {10.1145/3292500.3330707},
}
Figure