TL;DR
This paper tackles the challenge of serving personalized ads to infrequent visitors on platforms like Yahoo Answers, where traditional user profiling is difficult due to limited interaction history. They propose a collaborative filtering approach that leverages page content and user features to infer interests, achieving significant improvements in ad click prediction over baseline methods.
What Problem Does This Paper Address?
Modern ad serving systems thrive on user data to personalize ad delivery and predict click-through rates (CTR). However, many users interact with websites only briefly, often arriving via search engines or social media links and then leaving. This "occasional visitor" scenario presents a significant challenge, known as the cold-start problem, where there's insufficient historical data to build accurate user profiles. Platforms like Yahoo Answers, with a substantial portion of traffic from such users, struggle to effectively personalize ads. This paper aims to improve ad serving for these users by finding ways to infer their interests despite limited direct interaction history.
Methods
The core of the paper's approach is a collaborative filtering (CF) framework, specifically using a low-rank matrix factorization (MF) model. To overcome the cold-start problem for occasional visitors, they augment the standard CF model with additional features:
- User Features: Basic demographic information (age, gender, device, location) is used, even if incomplete.
- Page Content Features:
- Latent Dirichlet Allocation (LDA): This technique is used to extract topic distributions from the content of the Yahoo Answers page the user visits.
- Category: The predefined category of the Yahoo Answers question.
- Content Generators: Features derived from the users who asked or answered the question on the page (e.g., asker's age/gender, answerer's age/gender).
- Ad Features: Ad ID, Campaign ID, and Advertiser ID are also incorporated.
These features are represented as latent vectors, and for users with multiple features (like LDA topics or multiple answerers), their latent vectors are combined (often as a weighted sum) to form a comprehensive user representation. The model is trained using Stochastic Gradient Descent (SGD) to optimize a LogLoss function, aiming to maximize the Area Under the ROC Curve (AUC) for ad ranking.
Datasets
The evaluation was performed on a dataset collected from real Yahoo Answers pages and actual Yahoo Answers traffic. This dataset comprised event data logged over 12 days, averaging 9.8 million impressions per day. A key characteristic of this dataset, highlighted in Figure 1, is that over 77% of users were one-time visitors during the observed period, making it a suitable benchmark for studying the cold-start problem.
Key Insights and Ideas
- Page Content is Crucial for Occasional Visitors: The paper strongly suggests that understanding the content of the page a user visits is a powerful proxy for inferring their interests when direct user history is scarce.
- User and Page Features are Nearly Orthogonal: Combining user and page features provides a synergistic lift. The sum of individual lifts from user-only and page-only models is only slightly larger than the combined lift, indicating that these feature sets capture distinct but complementary information.
- Content Generators are Less Informative Than Page Content: Features derived from the users who created the content (askers/answerers) provided minimal additional lift beyond what was already captured by page content features (LDA, category). This suggests that for this specific platform and user behavior, the topic of the question/answer is a stronger signal than the demographics of the content creators.
- Ad Features Provide a Small but Consistent Boost: While user and page features drive the majority of the improvement, incorporating ad-specific features (like Ad ID, Campaign ID) offers a small but consistent and noteworthy enhancement to the model's performance.
Key Results
The proposed CF-UserPageAd algorithm, which combines user, page, and ad features, achieved significant improvements over baseline algorithms:
- AUC Lift: The algorithm achieved a consistent 6-8% improvement in AUC over the popularity baseline by the third day of evaluation. It also showed a nearly equivalent improvement over the Popularity Per Category (PPC) baseline.
- Feature Contributions:
- User Features: Provided a significant lift, even with limited traditional signals.
- Page Features (Category & LDA): Contributed a substantial and consistent lift, with category and LDA showing similar effectiveness.
- Content Generator Features: Provided only a small lift, largely contained within the gains from page features.
- Ad Features: Added a small but consistent improvement.
Limitations and Future Work
- Cold-Start Challenge: While the proposed methods significantly improve performance for occasional visitors, the initial days of evaluation show a period where the algorithm performs worse than the baseline due to the cold-start problem. The model gradually catches up as it accumulates data.
- Content Generator Features: The authors note that features derived from content generators did not yield significant improvements in their setting, suggesting this might be due to the relative sparsity of relevant information or the specific nature of Yahoo Answers. They consider this a viable approach for future instances of similar problems.
- Future Work: The authors suggest online evaluation of their methods on live traffic and experimenting with more elaborated models.
Summary updated