Abstract
Predicting Conversion Rate (CVR) in online advertising is complicated by delayed feedback, where conversions occur long after a click. This creates a trade-off between data freshness and label accuracy. We present the Multi-Interval Screening and Synthesizing (MISS) model, which addresses this by training multiple heads on varying waiting windows and dynamically aggregating their perspectives. MISS incorporates an assembled data pipeline to maintain freshness and a global positive weighting strategy to mitigate bias. Experimental results on large-scale datasets show that MISS significantly outperforms state-of-the-art methods, achieving up to a 16.8% relative improvement in AUC.
Introduction: The Delayed Feedback Problem
In online advertising, the Cost-Per-Action (CPA) model requires real-time Conversion Rate (CVR) prediction. However, a significant gap often exists between an ad click and the actual conversion. This delay introduces two primary challenges:
- The Freshness Trap: Training on immediate data leads to "fake negatives"—users who will convert later but are currently labeled as non-converters.
- The Staleness Trap: Waiting for the full attribution window (e.g., 30 days) ensures accurate labels but results in a model trained on outdated trends.
Existing solutions often rely on a single "optimal" window or complex auxiliary models that are computationally expensive and prone to bias.
Methodology: The MISS Framework
The MISS framework is designed to exploit information across multiple temporal scales simultaneously. It consists of three core components:
1. Multi-Interval Screening Modeling
Instead of a single output, the model employs multiple "heads," each trained on a different waiting window (e.g., 1 day, 7 days, 30 days). Shorter windows capture the most recent user behaviors but contain more noise, while longer windows provide ground-truth accuracy at the cost of staleness.
2. Assembled Pipeline Aggregation
A lightweight synthesizing model learns to dynamically weight the predictions from the various screening heads. To ensure the aggregator remains fresh, MISS uses an Assembled Data Pipeline. This pipeline substitutes old positive samples with the latest converted samples that share similar delay characteristics, effectively mimicking the "ideal" distribution without the 30-day wait.
3. Global Positive Weighting
To combat the bias of fake negatives, MISS applies a low-cost importance sampling technique. Rather than training a heavy auxiliary model to predict individual sample weights, MISS uses a global hyperparameter ($\alpha$) to amplify the importance of positive samples. This maintains the unique perspectives of each head while reducing overall bias.
Experimental Evaluation
The model was evaluated on two massive real-world datasets: the Criteo Conversion Logs (60 days of data) and the Tencent Advertising Algorithm Competition 2017 (9 days of data).
Key Results
MISS demonstrated superior performance across all metrics compared to high-profile baselines like ES-DFM, DEFUSE, and FTP:
- Accuracy Gains: MISS achieved a 16.8% relative improvement in AUC on the Criteo dataset and a 6.1% improvement on the Tencent dataset.
- Calibration: The model showed significantly lower Negative Log-Likelihood (NLL), indicating that its predicted probabilities are more accurate reflections of real-world conversion likelihoods.
- Ablation Success: Removing the "duplication mechanism" (the process of substituting delayed positives) led to a significant performance drop, confirming that the method of ingesting delayed data is critical.
Discussion and Conclusion
The core innovation of MISS is treating different waiting windows as distinct "views" of the same problem, analogous to multi-scale processing in computer vision. By substituting old positives with fresh ones, the model maintains a high degree of temporal relevance.
While highly effective, the model's performance is somewhat sensitive to the global weighting hyperparameter ($\alpha$), which may require tuning for specific business contexts. Furthermore, while the architecture is designed to be lightweight, the overhead of maintaining multiple heads must be weighed against the substantial gains in predictive accuracy. Future work may explore deeper feature interactions between the shared layers and the synthesizing component.
Summary updated