Disguise Adversarial Networks for Click-through Rate Prediction

Yue Deng, Yilin Shen, Hongxia Jin

Introduction

In online advertising, Click-Through Rate (CTR) prediction is a critical task. However, it is plagued by a chronic class imbalance problem: users click on very few ads, resulting in datasets where non-clicks (negatives) vastly outnumber clicks (positives), often by ratios exceeding 5:1. Standard machine learning models struggle with this disparity. Down-sampling negatives discards information, while over-sampling positives with synthetic methods like SMOTE often fails to capture the complex, heterogeneous nature of tabular ad data.

The DAN Framework

To address these challenges, the authors propose the Disguise Adversarial Network (DAN). Unlike traditional Generative Adversarial Networks (GANs) that generate data from random noise—a process that often fails for structured tabular data—DAN transforms real negative samples into potential positives.

Architecture of the Disguise Adversarial Network (DAN).

The framework consists of two main components:

  1. Disguise Neural Network ($T$): This network takes a real negative sample and applies "Ads Makeup"—subtle feature modifications—to create a "disguised" sample ($z$) that mimics a positive click. An $\ell_1$ distance penalty ensures these modifications remain realistic.
  2. Discriminator Neural Network ($D$): Serving as the actual CTR predictor, it is trained on real clicks, real non-clicks, and disguised samples.

The adversarial training employs a "mild objective" based on discriminative clustering. Instead of forcing hard labels on disguised data, the model minimizes conditional entropy, encouraging the discriminator to find a clear "info margin" between classes.

Experimental Setup

The model was evaluated on two large-scale industry datasets:

  • Criteo (Display Ads): 46 million logs over 7 days.
  • Avazu (Mobile Ads): 40 million logs over 10 days. A sliding window approach was used to simulate real-world temporal dynamics in ad delivery.

Performance and Robustness

DAN consistently outperforms a wide range of baselines, including standard Deep Neural Networks (DNN), Support Vector Machines (SVM), and specialized imbalance-handling techniques like SMOTE and ADASYN.

Performance comparison of DAN against supervised, imbalance-handling, and GAN-based methods.

Key findings include:

  • Superior AUC: On the Criteo dataset, DAN achieved an AUC of 0.75, significantly higher than the 0.69 achieved by a standard DNN.
  • Data Efficiency: DAN is remarkably robust to data scarcity. When training data was reduced to just 10% of the original volume, DAN's AUC only dropped from 0.75 to 0.70, whereas other models experienced much steeper declines.
  • Effective Augmentation: By using real negatives as "seeds," DAN effectively learns the manifold of ad data better than noise-based GANs like SGAN.

Conclusion and Limitations

DAN offers a powerful solution for CTR prediction under heavy class imbalance by leveraging "disguised" data to sharpen decision boundaries. However, the approach comes with increased training complexity, taking approximately 3x longer to train than a standard DNN. Future work may explore scaling this architecture to multi-class engagement scenarios and improving recommendation frequency.

Summary updated

Abstract

We introduced an adversarial learning framework for improving CTR prediction in Ads recommendation. Our approach was motivated by observing the extremely low click-through rate and imbalanced label distribution in the historical Ads impressions. We hence proposed a Disguise-Adversarial-Networks (DAN) to improve the accuracy of supervised learning with limited positive-class information. In the context of CTR prediction, the rationality behind DAN could be intuitively understood as ``non-clicked Ads makeup''. DAN disguises the disliked Ads impressions (non-clicks) to be interesting ones and encourages a discriminator to classify these disguised Ads as positive recommendations. In an adversarial aspect, the discriminator should be sober-minded which is optimized to allocate these disguised Ads to their inherent classes according to an unsupervised information theoretic assignment strategy. We applied DAN to two Ads datasets including both mobile and display Ads for CTR prediction. The results showed that our DAN approach significantly outperformed other supervised learning and generative adversarial networks (GAN) in CTR prediction.
BibTeX
@inproceedings{deng2017disguise,
  author = {Yue Deng and Yilin Shen and Hongxia Jin},
  title = {{Disguise Adversarial Networks for Click-through Rate Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2017},
  doi = {10.24963/ijcai.2017/220},
}
Figure