Structured Persuasive Writing Support in Legal Education: A Model and Tool for German Legal Case Solutions

Florian Weber, Thiemo Wambsganss, Seyed Parsa Neshaei, Matthias SöllnerACL 2023 · DOI

Introduction

In German legal education, mastering the "appraisal style" (Gutachtenstil) is a fundamental requirement for students. This rigid argumentative structure is essential for solving legal cases, yet it remains one of the most significant hurdles for law students. The challenge is compounded by a high student-to-instructor ratio in law schools, which limits the availability of individual, formative feedback on practice cases.

To address this gap, researchers have developed an NLP-powered writing support tool designed to identify complex legal argumentation structures and provide real-time feedback. By introducing a new corpus of annotated student case solutions and training transformer-based models, this research demonstrates how AI can effectively support the acquisition of specialized legal writing skills.

The Appraisal Style Framework

The appraisal style requires a precise, four-step logical sequence to address legal issues. It begins with a Major Claim (raising a legal question), followed by a Definition of the legal requirements, a Subsumption (weighing the facts against those requirements), and finally a Conclusion.

The structured legal writing scheme based on the appraisal style (Gutachtenstil).

A key contribution of this research is the granular breakdown of the "Subsumption" phase. Recognizing that this is where the core of legal reasoning occurs, the researchers divided it into Premises (factual statements) and Legal Claims (conclusions drawn from those facts). This finer-grained approach allows for more nuanced feedback on the "weighing of facts," which is often the most cognitively demanding part of the process.

Methodology and Corpus Development

The researchers created a multi-stage pipeline to automate the identification of these components:

  1. Corpus Creation: A significant new resource was developed consisting of 413 German student-written case solutions, totaling over 25,000 sentences. Two native German speakers with legal backgrounds annotated the text, achieving high Inter-Annotator Agreement (Krippendorff’s $\alpha$ of 0.92 for Major Claims and 0.88 for Conclusions).
  2. Modeling: Several transformer-based models (BERT, RoBERTa, DistilBERT, and DistilRoBERTa) were trained. The system uses a three-tier classification approach: a main model for primary components, a secondary model for subsumption details, and a tertiary model to identify relationships between premises and claims.
  3. Syntactic Enhancement: The team found that incorporating simple syntactic rules, such as identifying Roman numerals or specific legal abbreviations, significantly improved model performance in the highly structured legal domain.

The resulting models were integrated into a web-based writing tool. The "Legal Argumentation Dashboard" visualizes the student's progress by highlighting different components of their argument in real-time. This allows students to see the distribution of their components and identify missing or misplaced elements in their logic.

The Legal Argumentation Dashboard providing real-time feedback and structural visualization to students.

Key Results and Insights

The technical evaluation showed that the models are highly effective, with the best-performing versions achieving F1-scores between 0.82 and 0.95 for primary structural components. Interestingly, the lighter DistilBERT model performed comparably to larger models, suggesting that massive model size is not always a prerequisite for high accuracy in specialized domain tasks.

Beyond technical metrics, a user study with 34 law students provided evidence of learning success. The "treatment" group, which received AI-powered feedback, performed significantly better in a post-test assessment ($p = 0.0404$) compared to a control group. Furthermore, students reported high levels of trust in the system, rating its usefulness at 5.07/7 and feedback accuracy at 4.95/7.

Conclusion and Future Work

This research demonstrates that intelligent writing support systems can be successfully tailored to the unique logic of legal education. While currently optimized for the German Gutachtenstil, the underlying methodology of multi-tier classification and granular subsumption analysis could potentially be adapted to other legal writing styles, such as the US-based "IRAC" method. Future work will focus on refining the detection of nuanced relationships within the subsumption phase to provide even deeper insights into the "heart" of legal persuasion.

Summary updated

BibTeX
@inproceedings{weber2023structured,
  author = {Florian Weber and Thiemo Wambsganss and Seyed Parsa Neshaei and Matthias Söllner},
  title = {{Structured Persuasive Writing Support in Legal Education: A Model and Tool for German Legal Case Solutions}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.145},
}
Figure