If you are not paying for it, you are the product: how much do advertisers pay to reach you?

Panagiotis Papadopoulos, Pablo Rodríguez Rodríguez, Nicolas Kourtellis, Nikolaos Laoutaris

TL;DR

This paper introduces a novel methodology and a browser extension to estimate the real-time monetary value of individual users to advertisers within the Real-Time Bidding (RTB) ecosystem. By accounting for both cleartext and encrypted ad prices, they reveal that advertisers pay significantly more for encrypted price notifications, challenging previous assumptions and offering crucial transparency into the economics of online advertising.

What Problem Does This Paper Address?

The digital advertising landscape is increasingly driven by programmatic auctions (RTB), where user data is used to target ads in real-time. While the privacy implications of this data collection are widely debated, a critical question remains largely unanswered: how much do advertisers actually pay to reach a specific individual? Existing research often relies on averages (ARPU) or focuses only on cleartext ad prices, failing to capture the full picture, especially as encryption of ad prices becomes more prevalent. This paper aims to bridge this knowledge gap by providing a method to estimate the exact cost of an individual user to the ad ecosystem, even when prices are encrypted.

Methods

The authors propose a two-pronged approach:

  • Price Modeling Engine (PME): A centralized system that trains machine learning models to predict encrypted ad prices. It uses a combination of historical user data and data from controlled "probing ad-campaigns" to learn the relationship between ad features (like user location, time of day, device type, ad format, etc.) and the actual charge price. Dimensionality reduction techniques are employed to identify the most impactful features, making the probing campaigns cost-effective.
  • YourAdValue: A client-side browser extension that utilizes the PME's models. It monitors a user's network traffic, identifies RTB notifications, and calculates the cumulative cost of ads delivered to that user in real-time. For cleartext prices, it directly sums them. For encrypted prices, it uses the PME-trained model and the extracted features to estimate the price before summing it.

Datasets

  • Offline Weblogs: A year-long dataset of weblogs from 1594 volunteering mobile users, containing 373 million HTTP requests. This dataset was used to bootstrap the PME and analyze user behavior and ad traffic patterns.
  • Probing Ad-Campaigns: Two controlled, real-world ad-campaigns were conducted to generate ground-truth data for encrypted and cleartext prices. These campaigns were designed to be affordable and targeted specific ADX-DSP pairs.

Key Insights and Ideas

  • Encryption Matters (and Costs More): Contrary to previous assumptions, encrypted ad prices are significantly higher (around 1.7x) than cleartext prices. This suggests that encryption isn't just for tamper-proofing but also for hiding potentially more aggressive bidding strategies or higher perceived user value.
  • Individual User Value Varies Wildly: While the median user costs advertisers around 25 CPM annually, a small but significant portion of users (~2%) can cost advertisers orders of magnitude more (1000-10000 CPM). This highlights the highly personalized nature of ad pricing.
  • YourAdValue Empowers Users: The development of a practical, client-side tool like YourAdValue is crucial for bringing transparency to end-users, allowing them to understand their own "market value" in the ad ecosystem.
  • Feature Engineering is Key: Identifying and utilizing the right set of features (geo-temporal, user, and ad-related) is critical for accurately modeling encrypted prices, enabling cost-effective probing campaigns.

Key Results

  • The proposed methodology achieves over 82% accuracy in estimating a user's advertising worth.
  • The estimated ARPU (Average Revenue Per User) is ~55% higher when accounting for encrypted prices compared to using only cleartext prices.
  • Encrypted prices were found to be approximately 1.7 times higher than cleartext prices.
  • The median user cost is around 25 CPM annually, with up to 73% of users costing less than 100 CPM.
  • A small fraction of users (~2%) cost 10-100 times more to the ad ecosystem than the average user.
  • The developed classifier for encrypted prices achieved 82.9% accuracy (TP) and 83.5% precision using a subset of features, and 95% accuracy when including publisher information (though this was noted as potential overfitting).

Limitations and Future Work

  • Cost Model Assumption: The study assumes a Cost-Per-Impression (CPM) model for all ad charges, which might not reflect all advertising transactions.
  • Dataset Representativeness: While efforts were made to ensure representativeness, the dataset is from a single country and primarily mobile web traffic.
  • Future Work: The authors plan to make their YourAdValue prototype publicly available for community testing and exploration. They also suggest future research into the effects of retargeting and expanding the methodology to cover more diverse ad-buying models and platforms.

Summary updated

Abstract

Online advertising is progressively moving towards a programmatic model in which ads are matched to actual interests of individuals collected as they browse the web. Letting the huge debate around privacy aside, a very important question in this area, for which little is known, is: How much do advertisers pay to reach an individual? In this study, we develop a first of its kind methodology for computing exactly that -- the price paid for a web user by the ad ecosystem -- and we do that in real time. Our approach is based on tapping on the Real Time Bidding (RTB) protocol to collect cleartext and encrypted prices for winning bids paid by advertisers in order to place targeted ads. Our main technical contribution is a method for tallying winning bids even when they are encrypted. We achieve this by training a model using as ground truth prices obtained by running our own "probe" ad-campaigns. We design our methodology through a browser extension and a back-end server that provides it with fresh models for encrypted bids. We validate our methodology using a one year long trace of 1600 mobile users and demonstrate that it can estimate a user's advertising worth with more than 82% accuracy.
BibTeX
@inproceedings{papadopoulos2017if,
  author = {Panagiotis Papadopoulos and Pablo Rodríguez Rodríguez and Nicolas Kourtellis and Nikolaos Laoutaris},
  title = {{If you are not paying for it, you are the product: how much do advertisers pay to reach you?}},
  booktitle = {Internet Measurement Conference},
  year = {2017},
  url = {https://openreview.net/forum?id=kX5EKsrf1d},
}

Loading related content...

Figure