In digital advertising and e-commerce, the ultimate goal is often to predict a "conversion"—the moment a user actually buys a product or signs up for a service. However, modeling this behavior is notoriously difficult because conversions are "needles in a haystack." While billions of ads are shown (impressions) and millions are clicked, only a tiny fraction result in a purchase.
This creates a fundamental problem: extreme data sparsity. If a model only learns from the few successful conversions, it lacks the signal to understand the complex nuances of user intent. Researchers have converged on a shared solution: instead of modeling conversions in isolation, we must model the entire user journey—from the first impression to the final click—using multi-task learning.
The Sparsity Gap and Sample Selection Bias
Traditional models for Conversion Rate (CVR) prediction often suffer from two chronic issues. First, as noted by Yinrui Li and colleagues at Pinterest, the imbalance between clicks and purchases makes it difficult for models to distinguish high-intent users from superficial browsers.
Second, there is the Sample Selection Bias (SSB). Most CVR models are trained only on "clicked" data. However, at runtime, the model must rank items across the "entire space" of all possible impressions. As Zixuan Xu explains, this creates a training-serving gap; the model never learns what a "non-clicked" conversion might look like, leading to poor calibration in the real world.
Bridging the Funnel with Multi-Task Learning
To overcome this, researchers like Chen Gao have proposed Neural Multi-Task Recommendation (NMTR). The intuition is simple: while purchases are rare, clicks and "add-to-cart" actions are abundant. By sharing embeddings across these different behaviors, the model can use the "dense" signal from clicks to help the "sparse" task of predicting purchases.
This approach treats the conversion funnel as a hierarchy of objectives. In Multi-Objective Personalized Product Retrieval in Taobao Search, Yukun Zheng and others demonstrate that optimizing for relevance, exposure, clicks, and purchases simultaneously allows the model to learn a "smooth gradient" of user intent. The frequent, low-intent signals act as a scaffold for the rare, high-intent ones.
Modeling the Entire Space
The breakthrough in this field came with the Entire Space Multi-Task Model (ESM$^2$) framework. Developed by Hong Wen and researchers at Alibaba, ESM$^2$ decomposes the probability of a conversion into a chain of events: $$P(\text{Conversion}) = P(\text{Click}) \times P(\text{Conversion} | \text{Click})$$
By modeling $P(\text{Click})$ and $P(\text{Conversion})$ over the entire space of impressions, the model eliminates sample selection bias. It learns from every ad shown, not just the ones clicked. This concept has been extended by Zhenhao Jiang in ESMC, which introduces parameter constraints to handle multi-session user behaviors more accurately.
Capturing Sequential Dependencies
User behavior isn't just a set of independent tasks; it's a sequence. Dongbo Xi introduced the Adaptive Information Transfer Multi-task (AITM) framework to explicitly capture these dependencies. AITM uses an "information transfer" unit to pass knowledge from earlier stages of the funnel (like clicking) to later stages (like activating a credit card), ensuring that the model respects the logical order of user actions.
Similarly, Menghan Wang proposed using differentiable sorting in the MTLDS framework. By forcing the model to learn the rank-order of behaviors (e.g., a purchase is "stronger" than a click), the system gains a deeper understanding of the user's progression through the conversion funnel.
Why It Works: The Power of Shared Representations
The reason these multi-task approaches succeed where single-task models fail is regularization through shared knowledge.
- Feature Enrichment: Sparse identifiers (like a specific user ID) may only appear once in purchase data but dozens of times in click data. Multi-task learning allows the model to build a robust embedding for that user based on their clicks, which then informs the purchase prediction.
- Gradient Stability: In a sparse environment, the loss function for purchases is often flat or noisy. The auxiliary click task provides a consistent gradient that keeps the model's weights in a sensible region of the feature space.
- Counterfactual Reasoning: Recent work like ESCIM even uses causal inference to "imagine" what would have happened if a user had clicked an item they ignored, providing synthetic labels that further combat sparsity.
By shifting the focus from "will they buy this?" to "how is this user moving through the funnel?", researchers have transformed CVR prediction from a game of chance into a structured modeling of the entire user experience.