Abstract
Algorithmic decision making is now widespread, ranging from health care allocation to more common actions such as recommendation or information ranking. The aim to audit these algorithms has grown alongside. In this paper, we focus on external audits that are conducted by interacting with the user side of the target algorithm, hence considered as a black box. Yet, the legal framework in which these audits take place is mostly ambiguous to researchers developing them: on the one hand, the legal value of the audit outcome is uncertain; on the other hand the auditors' rights and obligations are unclear. The contribution of this paper is to articulate two canonical audit forms to law, to shed light on these aspects: 1) the first audit form (we coin the Bobby audit form) checks a predicate against the algorithm, while the second (Sherlock) is more loose and opens up to multiple investigations. We find that: Bobby audits are more amenable to prosecution, yet are delicate as operating on real user data. This can lead to reject by a court (notion of admissibility). Sherlock audits craft data for their operation, most notably to build surrogates of the audited algorithm. It is mostly used for acts for whistleblowing, as even if accepted as a proof, the evidential value will be low in practice. 2) these two forms require the prior respect of a proper right to audit, granted by law or by the platform being audited; otherwise the auditor will be also prone to prosecutions regardless of the audit outcome. This article thus highlights the relation of current audits with law, in order to structure the growing field of algorithm auditing.
Introduction
Algorithmic decision-making has permeated critical sectors of society, from healthcare allocation to information ranking. As these systems grow in complexity and influence, the need for external "black-box" audits—conducted by interacting with the user-facing side of an algorithm—has become paramount. However, researchers conducting these audits often navigate a legal "blind spot." This paper bridges the gap between computer science and procedural law, defining two canonical audit forms and analyzing their legal standing and risks.
Technical Framework: Bobby vs. Sherlock
The authors formalize the auditing landscape into two archetypal approaches:
- The Bobby Audit: Named after a beat cop, this approach checks a specific, well-defined predicate (a logical "if-then" rule) against an algorithm. For example, a Bobby audit might verify if a platform shows gambling ads to users under 13. These audits typically rely on real-world datasets and aim for "manifest evidence"—violations so obvious they require little further justification.
- The Sherlock Audit: Named after the detective, this is an exploratory approach. Instead of checking a single rule, the auditor submits various inputs to build a surrogate model—a local approximation—of the target algorithm to infer its internal logic. This often involves "crafted" or synthetic data, such as creating thousands of fake accounts to test pricing variations.
The Legal Landscape: Admissibility and Risk
The choice between a Bobby or Sherlock approach is not merely technical; it is determined by the broader context, including the nature of the audited platform and the legal powers of the auditor entity.
The framework highlights a critical trade-off:
- The Bobby Paradox: While technically simpler and more amenable to prosecution, Bobby audits are legally "delicate." Because they often process real user data, auditors risk violating privacy laws like the GDPR. If evidence is gathered "disloyally" (illegally), a judge may deem it inadmissible in civil proceedings, regardless of its accuracy.
- The Sherlock Weakness: Sherlock audits are safer for the auditor as they use synthetic data, but their results have lower probative value. Since the evidence is based on an interpolation (the surrogate model), defendants can argue the model is an inaccurate representation of the actual algorithm.
Practical Implications and Case Studies
The paper maps several high-profile audits to this framework:
- Uber Surge Pricing: Classified as a Sherlock audit, it used crafted inputs from multiple smartphones to build a regression model of Uber's pricing logic.
- XRay: A Sherlock audit that used fake Gmail accounts to correlate specific emails with served ads.
- Fairwashed Explanations: A Bobby audit that looked for specific "incoherent pairs" of explanations to prove unfairness.
For an audit to serve as "scientific proof," it must navigate the "loyalty of proof" principle. Evidence obtained through unfair means—such as scraping a site in violation of its Terms of Service—is often inadmissible unless the infringement is deemed "necessary and proportionate" to uncovering a significant harm.
Conclusion and Future Outlook
While this analysis focuses on French and EU law (including the GDPR and AI Act), the principles of necessity and proportionality are universal. A major challenge remains the "moving target" problem: platforms may "sandbox" auditors to hide their true behavior. Future work must focus on creating standardized technical representations of legal requirements, allowing judges to interpret algorithmic evidence more effectively. Ultimately, the right to audit must be clearly granted by law to protect researchers and ensure algorithmic accountability.
Summary updated