Personalized Inter-Task Contrastive Learning for CTR&CVR Joint Estimation

Zihan Lin, Xuanhua Yang, Shaoguo Liu, Xiaoyu Peng, Wayne Xin Zhao, Liang Wang, Bo ZhengarXiv 2022 · DOI · arXiv

Researchers from Alibaba and Renmin University have introduced AdaFTR, a novel multi-task learning (MTL) framework designed to adaptively align representations between Click-Through Rate (CTR) and Conversion Rate (CVR) tasks. By leveraging contrastive learning with an instance-level adaptive temperature, the model dynamically determines the optimal degree of information sharing for every individual user-item interaction, leading to significant performance gains in large-scale industrial advertising systems.

The Challenge of Joint Estimation

In online advertising, accurately predicting both the probability of a click (CTR) and the probability of a purchase given a click (CVR) is critical. While these tasks are inherently related, their relationship is highly nuanced and context-dependent. For instance, certain products may attract high click-through rates due to visual appeal but suffer from low conversion rates, while others may have a more direct correlation between interest and purchase.

Existing MTL models, such as MMoE (Multi-gate Mixture-of-Experts) or PLE (Progressive Layered Extraction), typically rely on "coarse-grained" sharing mechanisms. These models decide how much information to share at a global task level or through static architectural gates. This rigidity often leads to a trade-off: forcing representations to be too similar can erase task-specific nuances, while keeping them too distinct prevents the data-sparse CVR task from benefiting from the more abundant CTR data—a challenge known as Sample Selection Bias (SSB).

Comparison of multi-task learning paradigms: (a) Parameter Sharing, (b) Task-level Representation Alignment, and (c) AdaFTR's instance-level adaptive contrasting.

The AdaFTR Framework

To address these limitations, the authors propose AdaFTR (Adaptive Fine-grained Task Relatedness). The framework builds upon a standard parameter-sharing backbone but introduces a sophisticated alignment mechanism based on contrastive learning.

1. Contrastive Alignment

For every instance, the model generates distinct CTR and CVR representations. In the contrastive learning setup, these two representations are treated as a "positive pair," while representations from other instances within the same training batch serve as "negatives." The goal is to pull the representations of the same instance closer together in the embedding space when the tasks are related.

2. Instance-Level Adaptive Temperature (τ)

The core innovation of AdaFTR is making the "temperature" parameter ($\tau$) in the contrastive loss (InfoNCE) learnable and instance-specific. In standard contrastive learning, $\tau$ is a fixed hyperparameter that controls the strength of the alignment. AdaFTR uses a small auxiliary network to predict the optimal $\tau$ for each specific user-item interaction based on its unique features.

Detailed architecture of AdaFTR, highlighting the contrastive loss with adaptive temperature and the auxiliary relatedness network.

3. Relatedness Prediction and Gradient Management

The auxiliary network is trained to predict "task relatedness"—specifically, whether the CTR and CVR labels for an instance will match. If the tasks are predicted to be highly related for a specific instance, the model assigns a lower temperature, enforcing a stricter alignment of representations. To prevent this auxiliary task from interfering with the primary prediction objectives, the authors implement a gradient cut-off, ensuring the contrastive loss only updates the task-specific towers.

Experimental Results

The model was rigorously evaluated on two large-scale datasets: Ali-CCP (85 million impressions) and Ecomm-Ads (0.7 billion impressions).

  • Offline Performance: AdaFTR consistently outperformed state-of-the-art baselines including MMoE and PLE. On the Ali-CCP dataset, it achieved a +2.4% AUC lift in CVR, a substantial margin in the context of industrial recommendation systems.
  • Online Impact: During a 7-day live A/B test within Alibaba’s financial advertising system, AdaFTR delivered a +4.82% increase in CVR and a -5.78% reduction in Cost-Per-Click (CPC).
  • Operational Efficiency: A key advantage of AdaFTR is that the auxiliary relatedness network is only utilized during the training phase. Consequently, the model adds zero latency to the online inference stage compared to standard MTL architectures.

Conclusion

AdaFTR demonstrates that moving from task-level to instance-level relatedness modeling can significantly improve the performance of multi-task systems. By treating the contrastive temperature as a dynamic "dial" for alignment strength, the model effectively navigates the complexities of task interference and data sparsity. Future work may explore applying this adaptive contrastive mechanism to other domains, such as multi-domain recommendation or cross-modal retrieval.

Summary updated

Abstract

In modern advertising and recommender systems, multi-task learning (MTL) paradigm has been widely employed to jointly predict diverse user feedbacks (e.g. click and purchase). While, existing MTL approaches are either rigid to adapt to different scenarios, or only capture coarse-grained task relatedness, thus making it difficult to effectively transfer knowledge across tasks. To address these issues, in this paper, we propose an Adaptive Fine-grained Task Relatedness modeling approach, AdaFTR, for joint CTR-CVR estimation. Our approach is developed based on a parameter-sharing MTL architecture, and introduces a novel adaptive inter-task representation alignment method based on contrastive learning.Given an instance, the inter-task representations of the same instance are considered as positive, while the representations of another random instance are considered as negative. Furthermore, we explicitly model fine-grained task relatedness as the contrast strength (i.e. the temperature coefficient in InfoNCE loss) at the instance level. For this purpose, we build a relatedness prediction network, so that it can predict the contrast strength for inter-task representations of an instance. In this way, we can adaptively set the temperature for contrastive learning in a fine-grained way (i.e. instance level), so as to better capture task relatedness. Both offline evaluation with public e-commerce datasets and online test in a real advertising system at Alibaba have demonstrated the effectiveness of our approach.
BibTeX
@misc{lin2022personalized,
  author = {Zihan Lin and Xuanhua Yang and Shaoguo Liu and Xiaoyu Peng and Wayne Xin Zhao and Liang Wang and Bo Zheng},
  title = {{Personalized Inter-Task Contrastive Learning for CTR\&CVR Joint Estimation}},
  year = {2022},
  eprint = {2208.13442},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.13442},
}
Figure