OptMSM: Optimizing Multi-Scenario Modeling for Click-Through Rate Prediction

Xing Tang, Yang Qiao, Yuwen Fu, Fuyuan Lyu, Dugang Liu, Xiuqiang He

TL;DR

OptMSM (Optimizing Multi-Scenario Modeling) is a new framework designed to boost click-through rate (CTR) predictions across diverse app environments simultaneously. By using an orthogonality constraint to "untangle" shared data and a hypernetwork to customize prediction layers, it achieves significant performance gains in both offline benchmarks and real-world financial platforms.

What Problem Does This Paper Address?

In large-scale industrial recommendation systems—like Amazon or Tencent—users interact with different "scenarios." For example, a user might browse a general homepage (low intent) or a specific investment portfolio page (high intent).

Ideally, we’d build a custom model for every scenario, but that’s a resource nightmare. Instead, researchers use Multi-Scenario Modeling (MSM) to train one unified model for everything. The challenge? Negative transfer. This happens when data from one scenario "pollutes" another, leading to a "jack of all trades, master of none" situation. Most existing solutions focus on how to represent features but neglect the final prediction stage, leading to performance bottlenecks and massive, inefficient models.

Methods

The authors propose OptMSM, which optimizes the process in three distinct stages:

  1. Input Feature Partition: Unlike standard models that throw all data into one bucket, OptMSM explicitly splits features into scenario-specific (e.g., "position on the page") and scenario-shared (e.g., "user age").
  2. Scenario-Enhanced Learning Module: To prevent scenarios from interfering with each other, the model uses an orthogonality constraint. This is a mathematical trick that forces the internal representations of different scenarios to be "perpendicular" to one another in vector space, ensuring they remain distinct and independent.
  3. Scenario-Specific Prediction Module (The Hypernetwork): This is the "secret sauce." Instead of using a static neural network for the final prediction, OptMSM uses a Hypernetwork. This is a small network that dynamically generates the weights (parameters) for the main prediction network based on the current scenario. This allows the model to adapt its "thinking" instantly depending on where the user is in the app.

Datasets

The researchers validated OptMSM on three major datasets:

  • Ali-CCP: A public dataset from Taobao traffic logs (3 scenarios).
  • AliExpress: A public search system dataset covering user behaviors across 4 different countries.
  • Production Dataset: A private, massive dataset from Tencent Licaitong (a financial platform) involving four weeks of user feedback logs.

Key Insights and Ideas

  • Disentanglement is Key: The "aha moment" here is that simply sharing parameters isn't enough. By forcing representations to be orthogonal, the model successfully separates "common knowledge" from "scenario-specific quirks," which directly mitigates negative transfer.
  • Moving Beyond Representation: Most research stops at learning better "embeddings" (vector representations). OptMSM shows that the prediction architecture itself needs to be scenario-aware. Using a hypernetwork to generate adaptive parameters is a more efficient way to achieve "personalization" for a scenario than just adding more layers.
  • Lightweight Adaptation: The authors used SE-Net (Squeeze-and-Excitation) blocks for gating, which adds very little computational overhead while significantly improving how the model "attends" to different features.

Key Results

OptMSM consistently outperformed heavyweights like MMoE, PLE, and STAR:

  • Offline: It achieved the highest AUC (Area Under the Curve) across almost all scenarios in the Ali-CCP and AliExpress datasets.
  • Online A/B Testing: In a real-world deployment on Tencent Licaitong, OptMSM delivered a +1.42% to +1.76% lift in CTR.
  • Business Impact: More impressively, it saw a +5.82% to +7.10% lift in PAPM (Purchase Amount Per Mille), proving that better CTR modeling translates directly to higher revenue.
  • Efficiency: Despite the added complexity, the training cost only increased by about 4.3% to 4.6% compared to simpler baselines, making it viable for production.

Limitations and Future Work

While OptMSM is highly effective, the paper notes a few areas for growth:

  • Hyperparameter Sensitivity: The orthogonality constraint relies on a hyperparameter ($\lambda$) that needs careful tuning; if it's too high, it might stifle the model's ability to learn shared patterns.
  • Real-time Latency: While training costs are discussed, the paper doesn't provide a deep dive into the inference latency of the hypernetwork in a high-concurrency environment (though the online test suggests it's manageable).
  • Long-term Dynamics: The current model focuses on immediate clicks and purchases. Future work could explore how multi-scenario modeling affects long-term user retention or lifetime value.

Summary updated

Abstract

A large-scale industrial recommendation platform typically consists of multiple associated scenarios, requiring a unified click-through rate (CTR) prediction model to serve them simultaneously. Existing approaches for multi-scenario CTR prediction generally consist of two main modules: i) a scenario-aware learning module that learns a set of multi-functional representations with scenario-shared and scenario-specific information from input features, and ii) a scenario-specific prediction module that serves each scenario based on these representations. However, most of these approaches primarily focus on improving the former module and neglect the latter module. This can result in challenges such as increased model parameter size, training difficulty, and performance bottlenecks for each scenario. To address these issues, we propose a novel framework called OptMSM (\textbf{Opt}imizing \textbf{M}ulti-\textbf{S}cenario \textbf{M}odeling). First, we introduce a simplified yet effective scenario-enhanced learning module to alleviate the aforementioned challenges. Specifically, we partition the input features into scenario-specific and scenario-shared features, which are mapped to specific information embedding encodings and a set of shared information embeddings, respectively. By imposing an orthogonality constraint on the shared information embeddings to facilitate the disentanglement of shared information corresponding to each scenario, we combine them with the specific information embeddings to obtain multi-functional representations. Second, we introduce a scenario-specific hypernetwork in the scenario-specific prediction module to capture interactions within each scenario more effectively, thereby alleviating the performance bottlenecks. Finally, we conduct extensive offline experiments and an online A/B test to demonstrate the effectiveness of OptMSM.
BibTeX
@inproceedings{tang2023optmsm,
  author = {Xing Tang and Yang Qiao and Yuwen Fu and Fuyuan Lyu and Dugang Liu and Xiuqiang He},
  title = {{OptMSM: Optimizing Multi-Scenario Modeling for Click-Through Rate Prediction}},
  booktitle = {ECML/PKDD},
  year = {2023},
  doi = {10.1007/978-3-031-43427-3\_34},
}

Loading related content...

Figure