TL;DR
This paper introduces a framework to move beyond binary "click" signals in recommendation systems by reweighting them using dwell time. By defining a new "valid read" behavior and applying a novel sigmoid-based normalization function, the authors successfully filtered out clickbait and improved user satisfaction in a massive real-world deployment at WeChat.
What Problem Does This Paper Address?
In modern recommender systems, the "click" is the gold standard for positive feedback. However, clicks are inherently noisy. A user might click on a "clickbait" headline only to leave immediately when the content doesn't match the title. Standard models treat this dissatisfied bounce the same as a deep, meaningful read because both are binary "1s" in a training set.
While dwell time (the duration a user spends on an item) is a known proxy for satisfaction, it presents two major challenges:
- Bias toward long content: Directly optimizing for raw dwell time unfairly favors long-form videos or articles, potentially starving "light" users or short-form content creators.
- Diminishing returns: The difference in satisfaction between spending 1 second vs. 15 seconds is massive, whereas the difference between 601 seconds vs. 615 seconds is negligible.
Methods
The authors propose a Click Reweighting framework consisting of two main components:
1. Valid Read Selection
Instead of treating every click as a positive label, they define a "valid read" based on three criteria to ensure fairness across different users and items:
- T1 (Global Threshold): Dwell time exceeds a common-sense threshold (approx. 15s).
- T2 (Light User Patch): Clicks from users who rarely interact (less than 7 clicks/week) are preserved to avoid losing their sparse data.
- T3 (Relative Item Threshold): Clicks that fall within the top 90% of dwell times for that specific item, which protects high-quality short-form content.
2. Normalized Dwell Time (NDT) Function
To quantify the "intensity" of a click, they map raw dwell time ($T$) to a weight ($T_N$) using a specialized sigmoid function: $$T_N = \frac{A}{1 + \exp(-\frac{T - offset}{\tau})} - B$$
This function is designed with two key properties:
- Sensitivity (C1): The gradient is steepest near the "valid read" threshold ($offset$), helping the model clearly distinguish between a bounce and a real read.
- Saturation (C2): The curve flattens for very long dwell times, preventing "heavy" users or exceptionally long items from dominating the training process.
3. Multi-Task Learning (MTL)
The final model uses a shared-bottom architecture with two MLP towers: one predicts the probability of a "valid read," and the other predicts the weighted valid read probability.
Datasets
The model was evaluated on a massive dataset from WeChat Top Stories:
- Scale: 29.7 million users, 5.3 million items, and 751 million total instances.
- Composition: The dataset included 104 million clicks, of which 89.6 million were classified as "valid reads."
- Evaluation: Both offline AUC metrics and a 7-day online A/B test involving 5 million users.
Key Insights and Ideas
- Information Gain vs. Time: The core philosophy is that a recommender should help users get information efficiently. More time spent isn't always better; more valid time is.
- The Log-Normal Distribution: The authors observed that log dwell time roughly follows a Gaussian distribution. This insight allowed them to set their $offset$ and saturation points ($x_h$) based on standard deviations ($\mu \pm \sigma$) of the global distribution.
- Efficiency over Duration: Interestingly, the online A/B test showed that while total dwell time (DT) slightly decreased, the number of clicks (ACN) and impressions (AIN) increased. This suggests users were finding high-quality content faster and consuming more of it, rather than getting "stuck" on a few long items.
Key Results
The proposed VR+NDT (Valid Read + Normalized Dwell Time) model outperformed all baselines:
- Offline: Achieved a 5.62% Relative Improvement in valid read prediction over a standard CTR model.
- Online A/B Test:
- CTR: +2.577%
- Average Click Number (ACN): +4.139%
- Average Impression Number (AIN): +1.545%
The "Dwell Time Migration" analysis (Figure 3) confirmed that the model successfully increased dwell time for short-duration behaviors (P10/P20 quantiles) while reducing the over-emphasis on extremely long-duration behaviors.
Limitations and Future Work
While effective, the authors note that the current "valid read" definition is relatively straightforward. Future work could involve:
- Sophisticated Modeling: Exploring more complex ways to define valid reads beyond fixed thresholds and quantiles.
- Long-term Metrics: Investigating the impact of click reweighting on long-term user retention and ecosystem health.
- Feature Engineering: Better encoding of content features specifically for dwell time prediction, rather than relying on shared features with CTR towers.
Summary updated