Abstract
Automatic summarization of legal documents is hindered by domain-specific terminology, extreme redundancy, and a lack of large-scale annotated datasets. This study introduces a multi-task learning (MTL) approach for extractive summarization that incorporates rhetorical role labeling as an auxiliary task. By utilizing Legal-BERT embeddings and a novel redundancy loss function, the proposed models effectively identify salient information while minimizing repetition. Evaluation on US Board of Veterans’ Appeals decisions shows that the hierarchical MTL model with redundancy loss outperforms standard baselines and produces summaries favored by legal experts over human-written ones.
Introduction
Legal research is a cornerstone of judicial practice, yet the sheer volume and verbosity of case law often overwhelm practitioners. Extractive summarization—selecting the most representative sentences from a document—offers a solution. However, the legal domain presents unique challenges: "low-resource" environments where expert annotations are scarce, and "extreme redundancy," where key facts and reasoning are repeated across different sections of a decision.
Methodology
The researchers developed a pipeline that transitions from sentence scoring to final selection, integrating domain knowledge at every stage.
Sentence Representation and Scoring
The model utilizes Legal-BERT, pre-trained on over 3.4 million legal decisions, to capture nuanced legal vocabulary. For scoring, Bidirectional Gated Recurrent Units (Bi-GRUs) were selected over Transformers to better handle the limited training data and the substantial length of legal texts.
Multi-Task Learning (MTL)
A primary innovation is the use of Rhetorical Role (RR) labeling as an auxiliary task. By training the model to identify the functional role of a sentence (e.g., "Finding of Fact" or "Legal Rule"), the system learns the structural patterns of legal decisions. Two architectures were evaluated: MT-Shared, where tasks share all layers, and MT-Hierarchical, where the summarization task leverages higher-level representations.
Redundancy Management
To combat verbosity, the authors implemented a two-pronged strategy:
- Implicit (RdLoss): A redundancy loss term ($L_{RD}$) penalizes the model during training for assigning high scores to semantically similar sentences.
- Explicit (MMR-Select): During the selection phase, Maximal Marginal Relevance (MMR) is used to ensure that each selected sentence provides novel information relative to those already included in the summary.
Results and Discussion
The models were tested on decisions from the US Board of Veterans’ Appeals (BVA) concerning PTSD disability claims.
Quantitative Performance
The multi-task models consistently outperformed single-task (ST) baselines. The MT-Hierarchical+RdLoss configuration achieved the highest ROUGE-L scores across most test scenarios. Notably, performance varied by case outcome, with "Remanded" cases showing the strongest results, likely due to their more predictable structural templates compared to "Denied" or "Granted" cases.
Qualitative Expert Evaluation
A blind ranking by a legal expert revealed that model-generated summaries were often preferred over human-annotated "gold standards." Specifically, the ST+RdLoss and MT-Hierarchical+RdLoss models were frequently ranked as the top choices for adequacy and overall utility, demonstrating the practical value of the redundancy loss and multi-task framework.
Conclusion
This research demonstrates that multi-task learning, combined with structural awareness through rhetorical roles and explicit redundancy penalties, can overcome the hurdles of low-resource legal summarization. Future work will explore the generalizability of these models to other legal subdomains and the potential for scaling to multi-issue decisions.
Summary updated