Conversion Rate Prediction with Delayed Feedback

54 concepts43 papers1 proposal

In streaming machine learning systems—such as those powering online advertising, e-commerce recommendations, and real-time bidding—models must be updated continuously to capture shifting user trends and real-time market dynamics. However, these systems face a fundamental temporal mismatch: while negative signals (such as a user clicking an ad and leaving without buying) are observed instantly, positive signals (such as completing a purchase) often suffer from significant temporal latency.

This creates a severe dilemma between model freshness and label accuracy. If a system trains on real-time data immediately to remain fresh, pending positive outcomes are incorrectly recorded as negatives, introducing a systematic label bias. Conversely, if the system waits for definitive labels to ensure accuracy, the model becomes stale, degrading performance during rapid market changes.

Timeline of a Delayed Conversion:
[Ad Impression] ---> [User Click] -----------------------------> [Purchase / Conversion]
                           |                                               |
                           v                                               v
                 Immediate Negative Label?                       Delayed Positive Label!
                 (Causes Label Bias if trained early)            (Causes Stale Model if we wait)

The Freshness versus Accuracy Dilemma

When deploying web-scale predictive models, optimizing for immediate actions like click-through rate (CTR) does not necessarily capture downstream value, such as a completed purchase. As noted by Thore Graepel et al. in their deployment of Bing's adPredictor framework, extending online learning to downstream conversion events requires addressing significantly longer feedback loops and sparser data distributions (Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine).

If we ingest streaming data immediately, the lag between a user action and a conversion creates a label bias where pending positive outcomes are recorded as false negatives. Masahiro Kato et al. demonstrate that this systematic error prevents models from learning the true relationship between features and outcomes (Learning Classifiers under Delayed Feedback with a Time Window Assumption).

On the other hand, waiting for a long observation window to confirm that no conversion occurred introduces a data freshness gap. During high-stakes events like e-commerce sales promotions, conversion rates fluctuate rapidly; training on stale historical data prevents the model from adapting to real-time user behavior when it is most critical (Zhangming Chan et al., Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse Approach).

Modeling Latency via Survival Analysis and Importance Sampling

To resolve this trade-off, researchers have moved away from treating labels as static binary indicators. Instead, modern approaches model the elapsed time since the initial interaction as a continuous random variable, treating unobserved conversions as censored data.

One prominent solution is to use importance sampling to re-weight the loss function based on how much time has passed since the click. Jia-Qi Yang et al. proposed ES-DFM, a framework that explicitly models the elapsed time to achieve unbiased conversion rate (CVR) estimation in streaming environments (Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling). By calculating the probability that a conversion would have occurred within the elapsed window, the loss function downweights recent "negative" samples that haven't had enough time to convert, eliminating the systematic underestimation of conversion probabilities.

An alternative perspective integrates survival analysis directly into deep learning architectures to calibrate predictions post-click. Yumin Su et al. developed a two-stage framework that leverages post-click user behaviors to dynamically model the time-to-conversion distribution (An Attention-based Model for Conversion Rate Prediction with Delayed Feedback via Post-click Calibration). This allows the model to adjust its confidence based on how long an impression has remained inactive.

Label Correction and Multi-Window Architectures

Rather than re-weighting the loss of unconfirmed samples, another class of techniques dynamically corrects the labels themselves or trains models across multiple temporal horizons simultaneously.

Yifan Wang et al. introduced ULC (Unbiased Label Correction), which deploys an auxiliary model tasked with predicting whether an apparent negative sample will convert in the future (Unbiased Delayed Feedback Label Correction for Conversion Rate Prediction). The system uses these pseudo-labels to correct streaming data in real time, though the authors note that accuracy degrades for extremely delayed samples that stretch across weeks.

Multi-Interval Screening and Synthesizing (MISS):
Streaming Click ---> Head 1 (15-min window)  ---\
                ---> Head 2 (1-hour window)  ----> Dynamic Synthesis ---> Final CVR Prediction
                ---> Head 3 (1-day window)   ---/

To bypass the limitations of a single, manually tuned observation window, Qiming Liu et al. proposed the MISS (Multi-Interval Screening and Synthesizing) model (Online Conversion Rate Prediction via Multi-Interval Screening and Synthesizing under Delayed Feedback). MISS trains multiple parallel model heads, each assigned to a different waiting window (e.g., minutes, hours, days). A global weighting strategy dynamically synthesizes these distinct temporal perspectives, allowing the system to capture immediate behavioral trends while maintaining long-term calibration accuracy.

Downstream Impacts on Bandits and Real-Time Bidding

The challenge of delayed feedback extends beyond static classification into sequential decision-making frameworks, where algorithms must choose actions based on historical rewards.

In Real-Time Bidding (RTB) environments, demand-side platforms must optimize bids and pace budget delivery smoothly. As shown by Kuang-chih Lee et al. (Real time bid optimization with smooth budget delivery in online advertising) and Paul Grigas et al. (Profit Maximization for Online Advertising Demand-Side Platforms), conversion latency disrupts real-time feedback loops. Di Wu demonstrates that relying on immediate, uncalibrated performance estimates rather than true, delayed conversion data leads to highly suboptimal bidding decisions and poor long-term budget allocation (Bid Optimization by Multivariable Control in Display Advertising).

On the theoretical front, standard multi-armed bandit algorithms assume rewards are revealed immediately. When rewards are stochastically delayed, a learner is forced to make subsequent decisions blindly. To address this, Claire Vernade et al. (Linear bandits with Stochastic Delayed Feedback) and Benjamin Howson et al. (Delayed Feedback in Generalised Linear Bandits Revisited) extended generalized linear bandits to handle partially observable, delayed, or permanently censored feedback, proving that near-optimal regret bounds remain achievable if the underlying delay distribution is properly integrated into the action-exploration loop.

Key Foundational Work

Real Time Bid Optimization with Smooth Budget Delivery in Online Advertising (2013) — contributed 2 concept elements to this direction.
Online Learning with Costly Features and Labels (2013) — contributed 1 concept elements to this direction.
Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine (2010) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_delayed_feedback_f79eefa2,
  title = {delayed feedback},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/delayed-feedback/rdr_f79eefa2c4064f0fa4642ee08b56b188},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_delayed_feedback_f79eefa2_v1,
  title = {delayed feedback},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/delayed-feedback/rdr_f79eefa2c4064f0fa4642ee08b56b188/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake