TL;DR
This paper introduces "eyeWnder," a novel system that uses crowdsourcing and a privacy-preserving protocol to detect targeted online advertisements in real-time. It moves beyond content analysis to identify both direct and indirect targeting, offering a scalable and user-friendly approach to ad auditing.
What Problem Does This Paper Address?
The increasing prevalence of targeted advertising raises significant privacy concerns. While regulations like GDPR and CCPA aim to protect users, it's challenging for individuals to verify if ads are being targeted based on sensitive data or in ways that violate privacy norms. Existing methods for detecting targeted ads often rely on artificial "personas" or offline analysis, limiting their real-world applicability. Furthermore, detecting "indirect targeting" – where an ad for one product is shown to a group targeted for a seemingly unrelated reason – is particularly difficult with content-based analysis. This paper addresses the need for a scalable, real-time system that empowers users to audit targeted ads, even in complex indirect scenarios, while preserving their privacy.
Methods
The core of eyeWnder is a count-based algorithm that leverages two key observations about targeted ads:
- "Following" behavior: Targeted ads tend to appear across multiple domains for the same user.
- Lower user reach: Targeted ads are typically seen by a smaller, more specific group of users compared to non-targeted ads.
The algorithm locally estimates the number of domains a user sees a specific ad in (#Domains) and globally aggregates the number of unique users who see an ad (#Users). Thresholds for these counts are dynamically estimated.
To address privacy concerns, eyeWnder employs a privacy-preserving data sharing protocol. This protocol allows the system to compute aggregate statistics (#Users) without revealing individual users' browsing history or the specific ads they've seen. It uses Count-Min Sketches (CMS) and an Oblivious Pseudo-Random Function (OPRF) to enable secure aggregation of ad impression data. The OPRF is crucial for mapping ad URLs to unique identifiers that the server can process without knowing the original URLs.
The system architecture includes browser extensions for data collection, a back-end server for aggregation and threshold computation, a database for storing metadata, and a crawler server for evaluation purposes.
Datasets
The paper describes the use of three datasets for evaluation:
- "eyeWnder dataset": Collected over three weeks from 100 users via the FigureEight crowdsourcing platform, containing 6743 ads. This dataset was used as input for the count-based algorithm.
- "F8 dataset": A subset of the "eyeWnder dataset" where users manually labeled ads as targeted or non-targeted. This served as a ground truth for validation.
- "CR dataset": Collected by the crawler during the same three weeks, containing ad collections from websites where eyeWnder had classified an ad. This was used to identify static ads.
No new benchmarks were introduced; instead, existing methods and user-labeled data were used for validation.
Key Insights and Ideas
- Beyond Content: The paper demonstrates that simple frequency counts of ad impressions across users and domains can be surprisingly effective at detecting targeted ads, even without analyzing ad content or user profiles directly. This "black box" approach makes it robust to evasion tactics.
- Indirect Targeting is Detectable: The count-based approach is shown to be capable of detecting indirect targeting, a significant challenge for content-based methods. This is a crucial step towards a more comprehensive understanding of ad targeting practices.
- Privacy-Preserving Crowdsourcing Works: The implementation of a privacy-preserving protocol using CMS and OPRF successfully enables large-scale data collection for ad auditing without compromising user privacy. The overhead of this protocol is shown to be manageable.
- Low Repetition Threshold: The system can detect targeted ads with as few as 6-7 repetitions, making it practical for real-time detection.
- High Precision is Achievable: The system achieves very low false positive rates, meaning that when eyeWnder flags an ad as targeted, it is highly likely to be so. This is critical for enabling users to report potential privacy violations with confidence.
Key Results
- False Negatives: The algorithm achieves a false negative rate of less than 30% with just 6-7 ad repetitions, which can be further reduced to below 10% with more repetitions.
- False Positives: The false positive rate is consistently very low, typically close to zero and up to 2% in extreme scenarios. This high precision is a key strength.
- Live Validation: Live validation with real users showed a true positive rate of 78% and a true negative rate of 87%.
- Privacy Protocol Overhead: The privacy-preserving protocol introduces a manageable communication and computation overhead, with the mapping of ad URLs to IDs being particularly lightweight.
- Socio-Economic Biases: Logistic regression analysis revealed statistically significant biases in ad targeting, with women being more likely to be targeted than men, and a complex relationship between income level and targeting probability.
Limitations and Future Work
- User Base Size: The evaluation was conducted with a relatively small number of users (around 100 for live validation), which is acknowledged as a limitation. Scaling up the user base is a primary goal for future work to improve the system's robustness and the accuracy of its evaluation.
- Precision vs. Recall: The system was configured to maximize precision rather than recall, meaning it prioritizes correctly identifying targeted ads over finding all of them. Future work could explore heuristics to improve recall.
- Ad Network Complexity: The paper notes that different ad delivery channels use various techniques, some of which are difficult to detect programmatically.
- Future Work: The authors plan to scale up their user base, refine the count-based detection method and evaluation methodology, and explore more sophisticated heuristics for improved performance. They also aim to attract more users through media exposure and collaboration with data protection authorities.
Summary updated