TL;DR
This paper introduces a novel approach to improve native ad quality by predicting user engagement after a click, using survival analysis. By modeling "dwell time" (how long a user stays on an ad's landing page) as a survival outcome, they can better rank ads that lead to genuinely engaging experiences, not just clicks. This leads to improved click-through rates, reduced bounces, and a better overall user experience.
What Problem Does This Paper Address?
Online advertising platforms, especially those serving "native ads" (ads designed to blend in with content), face a challenge: how to effectively rank ads to maximize both advertiser revenue and user satisfaction. Traditional methods often rely heavily on Click-Through Rate (CTR) as a proxy for ad quality. However, high CTR doesn't always translate to a good user experience. Users might click on ads due to misleading titles ("clickbait") or accidental taps, only to immediately leave the landing page. This leads to wasted ad spend for advertisers and a degraded user experience, potentially driving users away from the platform. The paper aims to address this by moving beyond just predicting clicks to predicting post-click engagement, specifically by estimating how long a user will spend on an ad's landing page.
Methods
The core of the paper's contribution lies in applying survival analysis to predict user dwell time on ad landing pages. Unlike traditional regression models that predict a single value, survival analysis models the time until an event occurs. In this context, the "event" is a user leaving the landing page.
The authors leverage a Survival Random Forest model. This is an ensemble of decision trees specifically designed to handle "right-censored" data (where we know an event didn't happen by a certain time, but not exactly when it will). This approach is powerful because:
- It naturally models the distribution of dwell times, rather than just a single point estimate.
- It allows for flexible threshold setting at serving time, meaning they don't need to pre-define what constitutes "good" dwell time for all ads.
- It can incorporate a rich set of features from the ad's landing page to make predictions.
The paper also introduces a new ad quality score that combines the probability of a click with the probability of a dwell time exceeding a certain threshold: ad-quality(a, c) = Pr(click, DT > φ|a, c).
Datasets
The paper uses a dataset of 50,000 observations (ad, dwellTime) from 2.5K ads provided by over 850 advertisers. This dataset was split into 80% for training and 20% for testing. The features extracted for each ad are extensive, covering aspects of the "document object" (links, text size), "readability," "mobile optimization," "media" content, "input" elements, "semantic similarity" between the ad creative and landing page, and "history" (past performance metrics like dwell time and bounce rate).
Key Insights and Ideas
- Dwell time is a better proxy for post-click engagement than CTR alone: The study strongly suggests that how long a user stays on a landing page is a more reliable indicator of genuine interest and satisfaction than just whether they clicked.
- Survival analysis is a natural fit for dwell time prediction: The framework elegantly handles the time-to-event nature of user engagement, allowing for more nuanced predictions than simple regression.
- Landing page features are crucial for predicting engagement: The research highlights that features related to the landing page's content (document object, readability) are highly predictive of dwell time, confirming the importance of the post-click experience.
- Native ads require a different approach: Unlike search ads with explicit queries, native ads rely more heavily on landing page characteristics to infer user intent and engagement.
Key Results
- Offline Evaluation: The Survival Random Forest model consistently outperformed linear models (logistic and Cox regression) and standard Random Forests (classification and regression) in predicting dwell time across various thresholds. The AUC scores for Survival Random Forest were generally higher, especially for longer dwell times.
- Online Evaluation (A/B Testing): When deployed in a live setting, the proposed "AdQuality" system (which incorporates post-click engagement) showed significant improvements:
- Dwell Time: 13.3% higher than the control group.
- CTR: 6.8% uplift.
- Bounce Rate: 10.3% decrease.
- Feature Importance: Historical features (current estimate of dwell time) and landing page features (document object, readability) were found to be the most important predictors.
Limitations and Future Work
- Threshold Determination: The paper acknowledges that determining the optimal threshold for "good" dwell time is a challenge. They suggest that this threshold might vary by ad category (e.g., finance ads might naturally have longer dwell times than beauty ads) and potentially by user. Future work could involve personalized thresholds or models that account for ad categories.
- User-Level Engagement: While the current model focuses on ad and landing page features, future work could explore user-specific engagement patterns.
- Balancing Quality and Revenue: The authors note the importance of dynamically balancing ad quality with revenue maximization, as high quality doesn't always correlate with high bids. Their framework can be adapted to integrate other prediction modules for this purpose.
- Visual Features: The paper suggests that visual features of landing pages (e.g., image quality) could also significantly impact user engagement and are a promising area for future research.
Summary updated