The Ethics of Automating Legal Actors

Josef Valvoda, Alec Thompson, Ryan Cotterell, Simone Teufel

TL;DR

This position paper argues that the current "renaissance" in legal NLP is dangerously focused on automating the role of the judge. The authors contend that because judges in common law systems actively shape the law through moral and political reasoning, automating them presents insurmountable ethical risks; instead, research should pivot toward augmenting lawyers, whose roles are more amenable to AI and pose fewer systemic threats to justice.

What Problem Does This Paper Address?

The field of Legal NLP has shifted from symbolic AI (hand-coded rules) to sub-symbolic machine learning (LLMs and deep learning). While this has led to impressive benchmarks, the authors identify a fundamental flaw: most popular datasets are composed of judicial opinions.

By training models on the final word of a judge, researchers are inadvertently building "machine judges." This ignores the adversarial nature of the legal system and overlooks the fact that a judge’s role involves subjective fact-finding (reading a witness's "malice" or "honesty") and making controversial moral decisions to evolve the law. The paper addresses the technical and ethical gap between simply predicting a case outcome and actually performing the complex social role of a legal actor.

Methods

As a position piece, the paper provides a multi-disciplinary critique combining legal theory (Common Law vs. Civil Law) with NLP evaluation metrics. Key methodological contributions include:

  • Taxonomy of Automation: Distinguishing between Replacement (total AI takeover) and Augmentation (AI as a tool for experts).
  • Ethical Framework: Evaluating AI through the "pillars" of the Rule of Law (Consistency, Access, Equality, Comprehensibility) and Substantive Justice.
  • Technical Critique of ML Tasks: The authors deconstruct how tasks like "Legal Outcome Prediction" are currently operationalized, arguing they are "artificially easy" because the input facts are often written by the judge after the decision has been made, thus containing implicit clues to the outcome.
  • Proposed Modeling Shift: Moving from binary classification (violation vs. no violation) to a three-way classification that includes "claims" (what the lawyer argued) to better model the interaction between actors.

Datasets

The paper critiques and references several standard benchmarks in the field:

  • LexGLUE: A benchmark for legal language understanding.
  • COLIEE: Used for legal information retrieval and similar case matching.
  • ECtHR (European Court of Human Rights): Datasets used for outcome and claim prediction.
  • JEC-QA: A legal-domain question-answering dataset.
  • Caselaw Access Project: A primary source for large-scale judicial text.

Note: The authors highlight a significant gap—the lack of publicly available legal briefs (the documents lawyers write before a trial). They mention they are currently working on collecting and releasing such a dataset to the community.

Key Insights and Ideas

  • The "Trial Problem": Current NLP excels at text, but trials rely on multi-modal cues. A judge must decide if a witness is lying based on tone and demeanor—subjective mental states that text-only models cannot currently ground.
  • Fact Selection Bias: In judicial opinions, the "facts" are curated by the judge to justify their conclusion. Training on this data creates a "backwards-looking" model that cannot handle the "fluid aspect" of law where precedents are re-interpreted for modern social needs (e.g., Roe v. Wade).
  • Lawyers as a "Safer" Target: The authors argue that automating lawyers is ethically preferable. A lawyer's success is measured by persuasion (convincing a judge), which is a clearer metric for AI than the abstract concept of "justice." Furthermore, because lawyers are distributed across many private firms, their automation doesn't centralize political power as much as a single "State AI Judge" would.
  • Systemic Brittleness: Over-reliance on a single automated judicial system creates a "fragile" legal environment where a single algorithmic error or cyber-attack could propagate across the entire legal system instantly.

Key Results

While primarily a theoretical piece, the authors highlight specific quantitative findings to support their claims:

  • Metadata vs. Text: They cite Katz et al. (2017), who achieved 70% accuracy in predicting US Supreme Court outcomes using only metadata (date, judge names, etc.) rather than legal text, suggesting that current models might be picking up on statistical noise rather than legal reasoning.
  • Three-way Classification: The authors reference their own work (Valvoda et al., 2023) showing that including "lawyer claims" in the model significantly improves the performance and interpretability of outcome prediction tasks.
  • Adoption Rates: They note that 48% of UK law firms were already using some form of AI by 2018, primarily for augmentation tasks like due diligence and e-discovery.

Limitations and Future Work

  • Data Scarcity: The primary limitation is the lack of data representing the "voice of the lawyer" (briefs and oral arguments).
  • Multi-modality: The authors stress that "Full Automation" is impossible without models that can integrate vision and sound to handle the "Trial Problem."
  • Explainability: Current LLMs struggle to provide justifications that allow the public to exercise moral scrutiny. Future work must focus on "Recommendation 3": building models that explain reasoning in a way that is contestable by human citizens.
  • Accountability: The paper leaves open the difficult political question: if an AI makes a legal error, who is liable? The researcher, the software engineer, or the service provider?

Summary updated

Abstract

The introduction of large public legal datasets has brought about a renaissance in legal NLP. Many of these datasets are comprised of legal judgements - the product of judges deciding cases. This fact, together with the way machine learning works, means that several legal NLP models are models of judges. While some have argued for the automation of judges, in this position piece, we argue that automating the role of the judge raises difficult ethical challenges, in particular for common law legal systems. Our argument follows from the social role of the judge in actively shaping the law, rather than merely applying it. Since current NLP models come nowhere close to having the facilities necessary for this task, they should not be used to automate judges. Furthermore, even in the case the models could achieve human-level capabilities, there would still be remaining ethical concerns inherent in the automation of the legal process.
BibTeX
@article{valvoda2024ethics,
  author = {Josef Valvoda and Alec Thompson and Ryan Cotterell and Simone Teufel},
  title = {{The Ethics of Automating Legal Actors}},
  journal = {Trans. Assoc. Comput. Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00668},
}
Figure