VFed-SSD: Towards Practical Vertical Federated Advertising

Wenjie Li, Qiaolin Xia, Junfeng Deng, Hao Cheng, Jiangming Liu, Kouying Xue, Yong Cheng, Shu‐Tao Xia

Abstract

As an emerging secure learning paradigm in leveraging cross-agency private data, vertical federated learning (VFL) is expected to improve advertising models by enabling the joint learning of complementary user attributes privately owned by the advertiser and the publisher. However, there are two key challenges in applying it to advertising systems: a) the limited scale of labeled overlapping samples, and b) the high cost of real-time cross-agency serving. In this paper, we propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. We identify that: i) there are massive unlabeled overlapped data available in advertising systems, and ii) we can keep a balance between model performance and inference cost by decomposing the federated model. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data and propose the Split Knowledge Distillation (SplitKD) schema to avoid cross-agency serving. Empirical studies on three industrial datasets exhibit the effectiveness of our methods, with the median AUC over all datasets improved by 0.86% and 2.6% in the local and the federated deployment mode respectively. Overall, our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.

Introduction

Vertical Federated Learning (VFL) allows multiple parties, such as an advertiser and a publisher, to collaboratively train models on their combined features without exposing raw data. In the context of digital advertising, this enables the use of complementary user attributes to improve prediction accuracy for tasks like Click-Through Rate (CTR) or Conversion Rate (CVR) estimation. Despite its potential, VFL faces significant hurdles in production environments, specifically regarding data scarcity and inference latency.

Challenges in Vertical Federated Advertising

The Data Scarcity Trap

VFL typically relies on "labeled overlapping samples"—users present in both parties' databases with associated outcome labels (e.g., a click). In practice, this intersection is often small, leading to model overfitting and limited generalization.

The Latency Wall

Standard VFL architectures require real-time cross-silo inference. For an advertiser (Party A) to make a prediction, it must request intermediate representations from the publisher (Party B). In high-frequency ad auctions where decisions must be made in under 50ms, the network overhead of such cross-agency communication is often prohibitive.

VFL architecture and the latency constraint of cross-agency serving.

Proposed Method: VFed-SSD

To address these challenges, we introduce VFed-SSD (Semi-supervised Split Distillation), a framework that leverages unlabeled data and enables local serving.

1. Matched Pair Detection (MPD)

To overcome data scarcity, we utilize the vast amount of unlabeled historical data available. We propose MPD, a self-supervised pretext task where the model learns to distinguish between "matched" pairs (features belonging to the same user across parties) and "unmatched" pairs. This allows the model to capture the underlying correlations between vertically partitioned feature spaces before fine-tuning on sparse labeled data. Mathematically, this task is equivalent to maximizing the Point-wise Mutual Information (PMI) between the two parties' hidden representations.

2. Split Knowledge Distillation (SplitKD)

To eliminate real-time serving latency, we employ a teacher-student distillation approach. A complex federated "Teacher" model (SplitNN) is trained on features from both parties. A "Student" model is then trained to mimic the teacher's output using only the local features of Party A. At serving time, the Student model runs entirely on local infrastructure, inheriting the "knowledge" of the federated features without requiring real-time communication with Party B.

Experimental Results

We evaluated VFed-SSD on three datasets: Game-Ad, Game-Social, and Criteo-NC. Our results demonstrate that the MPD task significantly improves performance by utilizing unlabeled data.

Performance results in the federated setting showing significant AUC improvements.

In the federated setting, our VFL-MPD method achieved a median AUC improvement of 2.6%. In the local serving mode (using SplitKD), VFed-SSD outperformed baseline local models by a median of 0.86% AUC. Notably, in the Game-Social dataset, the lift reached 1.23%. In the competitive landscape of digital advertising, these improvements represent substantial gains in revenue and efficiency.

Conclusion

VFed-SSD provides a practical path for deploying Vertical Federated Learning in real-time advertising. By leveraging unlabeled data through self-supervised learning and bypassing the latency wall via split distillation, we enable advertisers to benefit from cross-agency data insights with minimal deployment costs and high inference efficiency.

Summary updated

Abstract

As an emerging secure learning paradigm in lever-aging cross-agency private data, vertical federatedlearning (VFL) is expected to improve advertising models by enabling the joint learning of complementary user attributes privately owned by the advertiser and the publisher. However, there are two key challenges in applying it to advertising systems: a) the limited scale of labeled overlapping samples, and b) the high cost of real-time cross-agency serving. In this paper, we propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. We identify that: i)there are massive unlabeled overlapped data available in advertising systems, and ii) we can keep a balance between model performance and inference cost by decomposing the federated model. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data and propose the Split Knowledge Distillation (SplitKD) schema to avoid cross-agency serving. Empirical studies on three industrial datasets exhibit the effectiveness of ourmethods, with the median AUC over all datasets improved by 0.86% and 2.6% in the local andthe federated deployment mode respectively. Overall, our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.
BibTeX
@misc{li2022vfed,
  author = {Wenjie Li and Qiaolin Xia and Junfeng Deng and Hao Cheng and Jiangming Liu and Kouying Xue and Yong Cheng and Shu‐Tao Xia},
  title = {{VFed-SSD: Towards Practical Vertical Federated Advertising}},
  year = {2022},
  eprint = {2205.15987},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.15987},
}
Figure