Algorithmic audits of algorithms, and the law

Erwan Le Merrer, Ronan Pons, Gilles Trédan

TL;DR

This paper bridges the gap between technical algorithmic auditing and the legal landscape, proposing two distinct audit archetypes: "Bobby" and "Sherlock." It clarifies the legal risks and opportunities associated with each, offering a framework for understanding how algorithmic audits can be used in legal contexts and for public disclosure.

What Problem Does This Paper Address?

As algorithmic decision-making becomes increasingly prevalent across various sectors, the need to audit these systems for fairness, bias, and compliance with regulations is paramount. However, researchers developing these audit techniques often face ambiguity regarding the legal standing of their findings and their own legal exposure. This paper addresses this "blind spot" by exploring the legal implications of algorithmic audits, aiming to structure the growing field of algorithm auditing by connecting technical approaches to legal perspectives.

Methods

The authors propose two canonical algorithmic audit archetypes:

  • Bobby Audit: This approach focuses on verifying a specific, well-defined predicate against the algorithm. It's akin to a policeman issuing tickets for parking violations. Bobby audits are generally easier to use in court because they often leverage real-world data to identify clear violations of a stated rule. However, they can be delicate due to data privacy concerns (like GDPR) and may be rejected if proper procedures aren't followed.
  • Sherlock Audit: This more elaborate and flexible approach involves creating a local model (a surrogate) of the target algorithm and then analyzing this surrogate to understand its behavior. This is compared to a detective investigating a crime, building a narrative from various clues. Sherlock audits craft their own data, making them less problematic from a data privacy standpoint. However, their findings might have lower evidential value in court, making them more suitable for whistleblowing or triggering ethical debates.

The paper also discusses the legal implications of the auditor's identity (e.g., regulatory authority vs. independent researcher) and the legal issues surrounding the data used for audits, distinguishing between pre-existing data (Bobby) and crafted data (Sherlock).

Datasets

Not discussed in the paper. The paper focuses on the methodologies and legal implications of auditing rather than specific datasets for training or evaluation.

Key Insights and Ideas

  • Two Archetypes for Auditing: The Bobby and Sherlock frameworks provide a useful categorization for understanding the technical approaches and legal implications of algorithmic audits.
  • Legal Risks Vary by Audit Type: Bobby audits, while potentially more direct for legal proceedings, carry higher risks due to the use of existing data (e.g., GDPR violations). Sherlock audits, by crafting data, mitigate these risks but may offer weaker legal evidence.
  • Auditor's Identity Matters: Whether an auditor is a regulatory authority, has a contract with the platform owner, or is an independent researcher significantly impacts their legal standing and the admissibility of their findings.
  • "Admissibility" vs. "Evidential Value": The paper highlights the crucial distinction between evidence being accepted into court (admissibility) and the weight a judge gives it (evidential value).
  • "Vetted Researchers" Status: The proposed "vetted researchers" status in Europe aims to provide legal protection and data access for academics auditing large online platforms, a significant step towards formalizing this field.
  • The "Diesel Gate" Analogy: The paper warns that the absence of proof of bias in an audited algorithm doesn't guarantee its absence, echoing the "diesel gate" scandal where systems were designed to appear compliant while circumventing regulations.

Key Results

The paper doesn't present quantitative results from specific audits but rather provides a conceptual framework and analysis of the legal landscape surrounding algorithmic audits. The key "results" are the articulation of the Bobby and Sherlock audit forms, the identification of their respective legal strengths and weaknesses, and the analysis of legal issues related to auditor identity, data usage, and proof admissibility.

Limitations and Future Work

  • Limitations of Bobby:
    • Limited by the input budget (N) when the input space (|X|) is vast, leading to statistical guarantees rather than definitive answers.
  • Difficulty in converting certain desirable behaviors into precise logical predicates.
  • Potential legal risks associated with processing existing data.
  • Limitations of Sherlock:
    • Higher cost in terms of human intervention and computational resources.
  • Weakened power due to the inherent uncertainty of surrogate models.
  • The evidential value of findings is still an open question for courts.
  • Future Work: The paper calls for continuous monitoring of auditing techniques and evolving laws. It also suggests that more advanced regulation might be needed to ensure proper impact on decision-making algorithms, especially in cases of "fairwashing." The development and application of the "vetted researchers" status is also a significant area for future observation.

Summary updated

Abstract

Algorithmic decision making is now widespread, ranging from health care allocation to more common actions such as recommendation or information ranking. The aim to audit these algorithms has grown alongside. In this paper, we focus on external audits that are conducted by interacting with the user side of the target algorithm, hence considered as a black box. Yet, the legal framework in which these audits take place is mostly ambiguous to researchers developing them: on the one hand, the legal value of the audit outcome is uncertain; on the other hand the auditors' rights and obligations are unclear. The contribution of this paper is to articulate two canonical audit forms to law, to shed light on these aspects: 1) the first audit form (we coin the Bobby audit form) checks a predicate against the algorithm, while the second (Sherlock) is more loose and opens up to multiple investigations. We find that: Bobby audits are more amenable to prosecution, yet are delicate as operating on real user data. This can lead to reject by a court (notion of admissibility). Sherlock audits craft data for their operation, most notably to build surrogates of the audited algorithm. It is mostly used for acts for whistleblowing, as even if accepted as a proof, the evidential value will be low in practice. 2) these two forms require the prior respect of a proper right to audit, granted by law or by the platform being audited; otherwise the auditor will be also prone to prosecutions regardless of the audit outcome. This article thus highlights the relation of current audits with law, in order to structure the growing field of algorithm auditing.
BibTeX
@article{merrer2024algorithmic,
  author = {Erwan Le Merrer and Ronan Pons and Gilles Trédan},
  title = {{Algorithmic audits of algorithms, and the law}},
  journal = {Ai Ethics},
  year = {2024},
  doi = {10.1007/s43681-023-00343-z},
}

Loading related content...

Figure