Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study

Miri Zilka, Holli Sargeant, Adrian Weller

Introduction

The integration of algorithmic tools into the criminal justice system has often been studied through the lens of specific, high-profile models like the US-based COMPAS. However, this narrow focus frequently overlooks the systemic reality of how these tools interact across an entire national infrastructure. In the United Kingdom, the deployment of algorithmic technologies—ranging from "hotspot" policing to recidivism prediction—is characterized by a fragmented landscape and a significant "transparency gap."

This paper provides a comprehensive survey of the UK’s algorithmic landscape, arguing that without robust governance, the cumulative effects of these tools can create a "feedback loop" that unfairly prejudices individuals as they move through the justice system. By mapping these tools against the criminal justice pipeline, the authors highlight how risks evolve and compound from the moment of first contact with police to final sentencing and incarceration.

A Functional Taxonomy of Algorithmic Tools

The UK criminal justice system is highly decentralized, comprising 43 autonomous police forces, the Crown Prosecution Service, the courts, and the prison and probation services. Each entity utilizes different technologies with varying levels of oversight. To analyze this complexity, the authors categorize algorithmic tools into three functional tiers:

  1. Data Infrastructure: Systems for recording and organizing raw data, such as STORM and Niche RMS. These serve as the foundation for all subsequent analysis.
  2. Data Analysis: Dashboards and statistical tools (e.g., Mapinfo, NDAS) that provide insights and visualizations without automated prediction.
  3. Risk Prediction: Software that uses historical data to predict future events. This is further divided into spatiotemporal tools (predicting where or when crime might occur, such as Predpol or iHotSpot) and individual tools (predicting the likelihood of an individual re-offending).

The Pipeline Effect and Cumulative Bias

The paper’s most critical insight is the "Pipeline Effect." Bias in the criminal justice system is not static; it is cumulative. This creates a dangerous feedback loop where early-stage policing tools influence the data used by later-stage judicial tools.

For example, if a predictive policing tool leads to over-policing in a specific neighborhood, the residents of that area will naturally accumulate more "prior arrests" in the system. When that individual later faces a court, a risk assessment tool—such as the Offender Group Reconviction Score (OGRS)—uses "number of prior arrests" as a primary feature to suggest a sentence length. Consequently, the bias inherent in early-stage policing tools "noises" the input for later-stage judicial tools, leading to harsher outcomes based on skewed historical data.

Barriers to Transparency and Accountability

The authors identify several systemic barriers that prevent effective regulation and public oversight:

  • The "Advisory" Fallacy: Agencies often defend the use of these tools by claiming they are merely "advisory" and that human practitioners make the final decision. However, the authors argue this ignores "automation bias." Practitioners often lack the time, technical expertise, or access to underlying data required to effectively challenge a machine-generated score, leading to a de facto reliance on the algorithm.
  • Proprietary Opacity: Many tools are developed by private vendors who cite "commercial confidentiality" to shield their algorithms from scrutiny. This makes it nearly impossible for public sector bodies to fulfill their equality duties or for the public to understand the logic behind life-altering decisions.
  • Inconsistent Standards: The lack of a national standard for data collection leads to a "postcode lottery." The level of algorithmic profiling an individual faces depends entirely on which of the 43 police forces arrests them, as each force may collect different "extra" data points.

Current Deployments in the UK

The survey highlights several active tools currently influencing the UK justice system:

  • Policing: The HART tool (Durham) uses machine learning to suggest whether to charge a suspect or offer a diversionary "checkpoint" program. Tools like Patrol-Wise are used for street-level crime prediction.
  • Courts: Actuarial scores like OGRS and RSR (Risk of Serious Recidivism) are standard components of pre-sentence reports used by judges to determine sentence length.
  • Prisons: The Digital Categorization Service (DCS) now suggests initial security categories for prisoners, a high-stakes decision that dictates a prisoner's daily life and future parole eligibility.

Policy Recommendations and Future Work

To address these systemic risks, the authors propose two major regulatory shifts. First, the creation of a Centralized Governance body to assist agencies in selecting tools and ensuring rigorous pre-deployment testing. Second, the implementation of Public Audits, where evaluation results are made public by default to allow for independent academic and civil society scrutiny.

While the survey is non-exhaustive due to the autonomy of local police forces, the principles of "pipeline bias" are globally applicable. Future work must focus on developing robust contestability mechanisms—ensuring that individuals have a clear legal and technical path to challenge algorithmic decisions that affect their fundamental liberty.

Summary updated

BibTeX
@inproceedings{zilka2022transparency,
  author = {Miri Zilka and Holli Sargeant and Adrian Weller},
  title = {{Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study}},
  booktitle = {AIES},
  year = {2022},
  doi = {10.1145/3514094.3534200},
}
Figure