Beyond content analysis: detecting targeted ads via distributed counting

Costas Iordanou, Nicolas Kourtellis, Juan Miguel Carrascosa, Claudio Soriente, Rubén Cuevas, Nikolaos Laoutaris

TL;DR

This paper introduces "eyeWnder," a novel system that uses crowdsourcing and a privacy-preserving protocol to detect targeted online advertisements in real-time. It moves beyond content analysis to identify both direct and indirect targeting, offering a scalable and user-friendly approach to ad auditing.

What Problem Does This Paper Address?

The increasing prevalence of targeted advertising raises significant privacy concerns. While regulations like GDPR and CCPA aim to protect users, it's challenging for individuals to verify if ads are being targeted based on sensitive data or in ways that violate privacy norms. Existing methods for detecting targeted ads often rely on artificial "personas" or offline analysis, limiting their real-world applicability. Furthermore, detecting "indirect targeting" – where an ad for one product is shown to a group targeted for a seemingly unrelated reason – is particularly difficult with content-based analysis. This paper addresses the need for a scalable, real-time system that empowers users to audit targeted ads, even in complex indirect scenarios, while preserving their privacy.

Methods

The core of eyeWnder is a count-based algorithm that leverages two key observations about targeted ads:

  1. "Following" behavior: Targeted ads tend to appear across multiple domains for the same user.
  2. Lower user reach: Targeted ads are typically seen by a smaller, more specific group of users compared to non-targeted ads.

The algorithm locally estimates the number of domains a user sees a specific ad in (#Domains) and globally aggregates the number of unique users who see an ad (#Users). Thresholds for these counts are dynamically estimated.

To address privacy concerns, eyeWnder employs a privacy-preserving data sharing protocol. This protocol allows the system to compute aggregate statistics (#Users) without revealing individual users' browsing history or the specific ads they've seen. It uses Count-Min Sketches (CMS) and an Oblivious Pseudo-Random Function (OPRF) to enable secure aggregation of ad impression data. The OPRF is crucial for mapping ad URLs to unique identifiers that the server can process without knowing the original URLs.

The system architecture includes browser extensions for data collection, a back-end server for aggregation and threshold computation, a database for storing metadata, and a crawler server for evaluation purposes.

Datasets

The paper describes the use of three datasets for evaluation:

  • "eyeWnder dataset": Collected over three weeks from 100 users via the FigureEight crowdsourcing platform, containing 6743 ads. This dataset was used as input for the count-based algorithm.
  • "F8 dataset": A subset of the "eyeWnder dataset" where users manually labeled ads as targeted or non-targeted. This served as a ground truth for validation.
  • "CR dataset": Collected by the crawler during the same three weeks, containing ad collections from websites where eyeWnder had classified an ad. This was used to identify static ads.

No new benchmarks were introduced; instead, existing methods and user-labeled data were used for validation.

Key Insights and Ideas

  • Beyond Content: The paper demonstrates that simple frequency counts of ad impressions across users and domains can be surprisingly effective at detecting targeted ads, even without analyzing ad content or user profiles directly. This "black box" approach makes it robust to evasion tactics.
  • Indirect Targeting is Detectable: The count-based approach is shown to be capable of detecting indirect targeting, a significant challenge for content-based methods. This is a crucial step towards a more comprehensive understanding of ad targeting practices.
  • Privacy-Preserving Crowdsourcing Works: The implementation of a privacy-preserving protocol using CMS and OPRF successfully enables large-scale data collection for ad auditing without compromising user privacy. The overhead of this protocol is shown to be manageable.
  • Low Repetition Threshold: The system can detect targeted ads with as few as 6-7 repetitions, making it practical for real-time detection.
  • High Precision is Achievable: The system achieves very low false positive rates, meaning that when eyeWnder flags an ad as targeted, it is highly likely to be so. This is critical for enabling users to report potential privacy violations with confidence.

Key Results

  • False Negatives: The algorithm achieves a false negative rate of less than 30% with just 6-7 ad repetitions, which can be further reduced to below 10% with more repetitions.
  • False Positives: The false positive rate is consistently very low, typically close to zero and up to 2% in extreme scenarios. This high precision is a key strength.
  • Live Validation: Live validation with real users showed a true positive rate of 78% and a true negative rate of 87%.
  • Privacy Protocol Overhead: The privacy-preserving protocol introduces a manageable communication and computation overhead, with the mapping of ad URLs to IDs being particularly lightweight.
  • Socio-Economic Biases: Logistic regression analysis revealed statistically significant biases in ad targeting, with women being more likely to be targeted than men, and a complex relationship between income level and targeting probability.

Limitations and Future Work

  • User Base Size: The evaluation was conducted with a relatively small number of users (around 100 for live validation), which is acknowledged as a limitation. Scaling up the user base is a primary goal for future work to improve the system's robustness and the accuracy of its evaluation.
  • Precision vs. Recall: The system was configured to maximize precision rather than recall, meaning it prioritizes correctly identifying targeted ads over finding all of them. Future work could explore heuristics to improve recall.
  • Ad Network Complexity: The paper notes that different ad delivery channels use various techniques, some of which are difficult to detect programmatically.
  • Future Work: The authors plan to scale up their user base, refine the count-based detection method and evaluation methodology, and explore more sophisticated heuristics for improved performance. They also aim to attract more users through media exposure and collaboration with data protection authorities.

Summary updated

Abstract

Being able to check whether an online advertisement has been targeted is essential for resolving privacy controversies and implementing in practice data protection regulations like GDPR, CCPA, and COPPA. In this paper we describe the design, implementation, and deployment of an advertisement auditing system called iWnder that uses crowdsourcing to reveal in real time whether a display advertisement has been targeted or not. Crowdsourcing simplifies the detection of targeted advertising, but requires reporting to a central repository the impressions seen by different users, thereby jeopardising their privacy. We break this deadlock with a privacy preserving data sharing protocol that allows iWnder to compute global statistics required to detect targeting, while keeping the advertisements seen by individual users and their browsing history private. We conduct a simulation study to explore the effect of different parameters and a live validation to demonstrate the accuracy of our approach. Unlike previous solutions, iWnder can even detect indirect targeting, i.e., marketing campaigns that promote a product or service whose description bears no semantic overlap with its targeted audience.
BibTeX
@inproceedings{iordanou2019beyond,
  author = {Costas Iordanou and Nicolas Kourtellis and Juan Miguel Carrascosa and Claudio Soriente and Rubén Cuevas and Nikolaos Laoutaris},
  title = {{Beyond content analysis: detecting targeted ads via distributed counting}},
  booktitle = {CoNEXT},
  year = {2019},
  url = {https://openreview.net/forum?id=6x3BsxWWrp},
}
Figure