TL;DR
This paper introduces a novel methodology and a browser extension to estimate the real-time monetary value of individual users to advertisers within the Real-Time Bidding (RTB) ecosystem. By accounting for both cleartext and encrypted ad prices, they reveal that advertisers pay significantly more for encrypted price notifications, challenging previous assumptions and offering crucial transparency into the economics of online advertising.
What Problem Does This Paper Address?
The digital advertising landscape is increasingly driven by programmatic auctions (RTB), where user data is used to target ads in real-time. While the privacy implications of this data collection are widely debated, a critical question remains largely unanswered: how much do advertisers actually pay to reach a specific individual? Existing research often relies on averages (ARPU) or focuses only on cleartext ad prices, failing to capture the full picture, especially as encryption of ad prices becomes more prevalent. This paper aims to bridge this knowledge gap by providing a method to estimate the exact cost of an individual user to the ad ecosystem, even when prices are encrypted.
Methods
The authors propose a two-pronged approach:
- Price Modeling Engine (PME): A centralized system that trains machine learning models to predict encrypted ad prices. It uses a combination of historical user data and data from controlled "probing ad-campaigns" to learn the relationship between ad features (like user location, time of day, device type, ad format, etc.) and the actual charge price. Dimensionality reduction techniques are employed to identify the most impactful features, making the probing campaigns cost-effective.
- YourAdValue: A client-side browser extension that utilizes the PME's models. It monitors a user's network traffic, identifies RTB notifications, and calculates the cumulative cost of ads delivered to that user in real-time. For cleartext prices, it directly sums them. For encrypted prices, it uses the PME-trained model and the extracted features to estimate the price before summing it.
Datasets
- Offline Weblogs: A year-long dataset of weblogs from 1594 volunteering mobile users, containing 373 million HTTP requests. This dataset was used to bootstrap the PME and analyze user behavior and ad traffic patterns.
- Probing Ad-Campaigns: Two controlled, real-world ad-campaigns were conducted to generate ground-truth data for encrypted and cleartext prices. These campaigns were designed to be affordable and targeted specific ADX-DSP pairs.
Key Insights and Ideas
- Encryption Matters (and Costs More): Contrary to previous assumptions, encrypted ad prices are significantly higher (around 1.7x) than cleartext prices. This suggests that encryption isn't just for tamper-proofing but also for hiding potentially more aggressive bidding strategies or higher perceived user value.
- Individual User Value Varies Wildly: While the median user costs advertisers around 25 CPM annually, a small but significant portion of users (~2%) can cost advertisers orders of magnitude more (1000-10000 CPM). This highlights the highly personalized nature of ad pricing.
- YourAdValue Empowers Users: The development of a practical, client-side tool like YourAdValue is crucial for bringing transparency to end-users, allowing them to understand their own "market value" in the ad ecosystem.
- Feature Engineering is Key: Identifying and utilizing the right set of features (geo-temporal, user, and ad-related) is critical for accurately modeling encrypted prices, enabling cost-effective probing campaigns.
Key Results
- The proposed methodology achieves over 82% accuracy in estimating a user's advertising worth.
- The estimated ARPU (Average Revenue Per User) is ~55% higher when accounting for encrypted prices compared to using only cleartext prices.
- Encrypted prices were found to be approximately 1.7 times higher than cleartext prices.
- The median user cost is around 25 CPM annually, with up to 73% of users costing less than 100 CPM.
- A small fraction of users (~2%) cost 10-100 times more to the ad ecosystem than the average user.
- The developed classifier for encrypted prices achieved 82.9% accuracy (TP) and 83.5% precision using a subset of features, and 95% accuracy when including publisher information (though this was noted as potential overfitting).
Limitations and Future Work
- Cost Model Assumption: The study assumes a Cost-Per-Impression (CPM) model for all ad charges, which might not reflect all advertising transactions.
- Dataset Representativeness: While efforts were made to ensure representativeness, the dataset is from a single country and primarily mobile web traffic.
- Future Work: The authors plan to make their YourAdValue prototype publicly available for community testing and exploration. They also suggest future research into the effects of retargeting and expanding the methodology to cover more diverse ad-buying models and platforms.
Summary updated