Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction

Moyu Zhang, Yujun Jin, Yun Chen, Jinxin Hu, Yu Zhang, Xiaoyi Zeng

TL;DR

Existing generative models for Click-Through Rate (CTR) prediction suffer from an "asymmetric paradigm" flaw: they use generative objectives during training to learn better representations but revert to standard discriminative methods during online inference. This paper introduces SGCTR (Symmetric Masked Generative Paradigm), which bridges this gap by using the model's generative capabilities during inference to iteratively denoise and refine input features, leading to significant accuracy gains in both offline benchmarks and real-world online production.

What Problem Does This Paper Address?

Click-Through Rate (CTR) prediction is the backbone of modern recommender systems and online advertising. Traditionally, models use a discriminative paradigm—taking user and item features to output a binary classification score (click vs. no-click). Recently, researchers have integrated generative paradigms (like discrete diffusion or auto-regressive sequencing) during training to help the model capture complex feature dependencies and combat data sparsity.

However, a critical bottleneck remains: inference asymmetry. During online deployment, these generative models discard their generation capabilities and predict directly on raw, potentially noisy input samples. The authors tackle this limitation by asking: Can we use the model's generative power during inference to actively clean and optimize the input data before making a prediction?

Methods

SGCTR establishes a symmetric framework split into two distinct phases:

  1. Training Phase (Discrete Diffusion): The model learns feature correlations by corrupting input samples with a binary mask token based on a noise scheduling function. To handle the massive search space caused by high-cardinality ID features, it utilizes a sampled softmax approximation, calculating loss over a random subset of negative samples.

  2. Inference Phase (Iterative Refinement): Instead of a single-forward pass, SGCTR treats inference as an active, iterative denoising process:

  3. Initialization: Stable user-side features are kept, while item-side and cross-features are initially masked.

  4. Feature Generation & Confidence Scoring: The model predicts the masked features in parallel. It then computes a confidence score for each prediction based on its cosine similarity to the original input feature.
  5. Feature Redefine and Mask: Features with the lowest confidence scores are re-masked for the next round, while highly confident features are retained and scaled by their confidence weights.
  6. Prediction: After $T$ discrete steps, the fully refined sample is fed into the final scoring function to predict the CTR.

To make this practical for low-latency online deployment, the authors use Key-Value (KV) caching to avoid re-computing stable features, keeping the overall computational complexity at $O(n)$.

Datasets

The authors evaluated SGCTR on four large-scale datasets:

  • Criteo: A canonical public benchmark containing 45 million impressions, 13 continuous features, and 26 categorical features.
  • Avazu: 40 million chronologically ordered ad click logs across 23 feature fields.
  • Malware: A Kaggle competition dataset with 8.9 million impressions and 81 feature fields.
  • Industrial Dataset: A massive proprietary dataset gathered from a real-world e-commerce platform's online advertising system, containing 782 million impressions across 72 feature fields.

Key Insights and Ideas

  • Symmetry Unlocks Potential: Generative models shouldn't just be passive feature extractors during training. Allowing the model to "infer as it trains" lets it actively correct noisy or missing data at runtime.
  • The Power of Concave Scheduling: When deciding how many features to mask at each step, concave functions (like Cosine or Square schedules) perform best. They maintain a high mask ratio early on, forcing the model to focus on refining a small core of highly confident features before filling in the rest.
  • More Iterations Aren't Always Better: Performance peaks at a moderate number of steps ($T=5$ to $8$) and declines if the loop runs too long. Too many steps impose overly strict constraints on feature selection and degrade the final prediction score.
Prediction performance under different iteration steps across Criteo and Industrial datasets

Key Results

SGCTR consistently outperformed both traditional discriminative baselines (like DeepFM, DCN, and HSTU) and state-of-the-art generative baselines (DGenCTR) across all datasets.

Table 2: Prediction performance on datasets of CTR predic- tion models. * indicates p-value < 0.05 in the significance test.
  • Offline Evaluation: SGCTR achieved statistically significant improvements in Area Under the ROC Curve (AUC) across all four benchmarks, proving that inference-phase feature refinement works.
  • Online Production Lift: In a 10-day online A/B test on a large-scale e-commerce platform, SGCTR achieved a 2.1% relative improvement in CTR compared to a production-grade DGenCTR baseline.

Limitations and Future Work

The primary limitation acknowledged by the authors involves generating high-cardinality ID features from scratch. In an ablation study variant where the model completely replaced original features with purely generated ones (SGCTR(GenFea)), performance dropped significantly. Because of this, the authors had to compromise by using a redefinition/weighting method rather than pure generation.

Future work will focus on enhancing the generative fidelity of the model so it can reliably synthesize complex, high-dimensional ID features from scratch without relying on original inputs as a crutch.

Summary updated

Abstract

Generative models are increasingly being explored in click-through rate (CTR) prediction field to overcome the limitations of the conventional discriminative paradigm, which rely on a simple binary classification objective. However, existing generative models typically confine the generative paradigm to the training phase, primarily for representation learning. During online inference, they revert to a standard discriminative paradigm, failing to leverage their powerful generative capabilities to further improve prediction accuracy. This fundamental asymmetry between the training and inference phases prevents the generative paradigm from realizing its full potential. To address this limitation, we propose the Symmetric Masked Generative Paradigm for CTR prediction (SGCTR), a novel framework that establishes symmetry between the training and inference phases. Specifically, after acquiring generative capabilities by learning feature dependencies during training, SGCTR applies the generative capabilities during online inference to iteratively redefine the features of input samples, which mitigates the impact of noisy features and enhances prediction accuracy. Extensive experiments validate the superiority of SGCTR, demonstrating that applying the generative paradigm symmetrically across both training and inference significantly unlocks its power in CTR prediction.
BibTeX
@inproceedings{zhang2026infer,
  author = {Moyu Zhang and Yujun Jin and Yun Chen and Jinxin Hu and Yu Zhang and Xiaoyi Zeng},
  title = {{Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction}},
  booktitle = {The Web Conference},
  year = {2026},
  doi = {10.1145/3774904.3792865},
}
Figure