Cookie Consent Has Disparate Impact on Estimation Accuracy

Erik Miehling, Rahul Nair, Elizabeth M. Daly, Karthikeyan Natesan Ramamurthy, Robert Nelson RedmondNeurIPS 2023 · OpenReview

In the era of GDPR and heightened privacy awareness, we have grown accustomed to the ubiquitous cookie consent banner. We often assume that by clicking "Reject All," we are successfully shielding our digital identity from the prying eyes of recommendation algorithms. However, research presented at NeurIPS 2023 suggests that this privacy-protective action might backfire in a surprising way.

The Privacy Paradox

Recommender systems rely on cookies to track user behavior and personalize content. While regulations mandate explicit consent, users do not opt in at uniform rates. Studies show that factors like age, demographic, and geographic location significantly influence whether someone clicks "Accept."

This paper demonstrates that when cookie consent rates vary across different demographics (e.g., older users opting out more frequently), the act of withholding consent itself becomes a powerful signal. Counter-intuitively, a recommender system may actually estimate the latent attributes of a "private" user more accurately than those of a user who agrees to be tracked.

The authors investigate a critical fairness question: Does demographic-dependent consent behavior create a "disparate impact" on how well a system can profile its users? If a specific group is known to rarely provide consent, then the very act of a user from that group withholding consent provides the system with a strong hint about who they are. This creates a privacy paradox where trying to remain anonymous actually makes you more predictable to the algorithm.

To isolate the effects of consent rates on learning, the researchers used an agent-based simulation framework built on RecSim.

The simulated recommendation process where an agent interacts with users who make cohort-dependent consent decisions.

The core components of their model include:

  • User Model: Users are defined by a "cohort" (demographic) and "topic affinities" (tastes). Their decision to share a cookie is dictated by a cohort-dependent probability.
  • The Agent: The recommender agent uses Bayesian updates to refine its beliefs about a user's cohort based on two signals: the cookie value (if provided) and the consent decision itself.
  • Learning Algorithm: To estimate user tastes, the agent employs a confidence-weighted online matrix factorization procedure, allowing the system to weigh its recommendations based on how certain it is about a user’s profile.

The study relies on synthetic datasets generated within this environment ($n=1000$ users, $m=200$ ads) to remove real-world noise and precisely measure the impact of varying consent rates.

Key Insights and Results

The researchers found that under demographic-dependent consent, the standard intuition that "more data equals better estimation" breaks down.

  • The Consent Signal: In a world with heterogeneous consent rates, the decision to opt out is highly informative. If 90% of Group A accepts cookies but only 10% of Group B does, a "No" vote acts as a strong statistical signal that the user belongs to Group B.
  • The Inversion Effect: For demographics with low consent rates, withholding consent can lead to lower estimation errors than providing it. Essentially, the "No" is a cleaner signal of group membership than a noisy cookie might be.
  • Amplification via Gaps: The larger the gap in consent rates between two demographics, the more the system's accuracy disparities are amplified.
Comparison of estimation errors under homogeneous (left) and heterogeneous (right) consent rates. Note the "inversion" in the right-hand plots where non-consent groups sometimes have lower error.

As shown in the heterogeneous consent scenario above, the error for the non-consent group in a low-consent population can drop faster or stay lower than the consent group, because the opt-out decision acts as a strong demographic proxy. Users from the lower-consent demographic who do provide consent often experience higher estimation errors than similar users from high-consent demographics.

Rethinking Fairness in Privacy

The authors argue for a new notion of fairness: informational consistency. A user’s privacy-protective decision should be consistent with the system’s resulting knowledge about them. Currently, systems can inadvertently "punish" privacy-seekers with higher profiling accuracy.

While the simulation uses a simplified model (e.g., treating a cookie as a static proxy for a cohort rather than a rich history of clicks), it highlights a critical flaw in how we conceptualize digital privacy. Future work will need to focus on validating these findings in real-world recommender systems and developing new fairness constraints—ensuring that algorithms respect a user's desire for privacy without turning their "No" into a high-fidelity tracking signal.

Summary updated

Abstract

Cookies are designed to enable more accurate identification and tracking of user behavior, in turn allowing for more personalized ads and better performing ad campaigns. Given the additional information that is recorded, questions related to privacy and fairness naturally arise. How does a user's consent decision influence how much the system can learn about their demographic and tastes? Is the impact of a user's consent decision on the recommender system's ability to learn about their latent attributes uniform across demographics? We investigate these questions in the context of an engagement-driven recommender system using simulation. We empirically demonstrate that when consent rates exhibit demographic-dependence, user consent has a disparate impact on the recommender agent's ability to estimate users' latent attributes. In particular, we find that when consent rates are demographic-dependent, a user disagreeing to share their cookie may counter-intuitively cause the recommender agent to know more about the user than if the user agreed to share their cookie. Furthermore, the gap in base consent rates across demographics serves as an amplifier: users from the lower consent rate demographic who agree to cookie sharing generally experience higher estimation errors than the same users from the higher consent rate demographic, and conversely for users who choose to disagree to cookie sharing, with these differences increasing in consent rate gap. We discuss the need for new notions of fairness that encourage consistency between a user's privacy decisions and the system's ability to estimate their latent attributes.
BibTeX
@inproceedings{miehling2023cookie,
  author = {Erik Miehling and Rahul Nair and Elizabeth M. Daly and Karthikeyan Natesan Ramamurthy and Robert Nelson Redmond},
  title = {{Cookie Consent Has Disparate Impact on Estimation Accuracy}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=dFtpRphNb3},
}
Figure