BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval

Yunqiu Shao, Jiaxin Mao, Yiqun Liu, Weizhi Ma, Ken Satoh, Min Zhang, Shaoping Ma

Abstract

Legal case retrieval is a critical task in common law systems, requiring the identification of "noticed cases" from vast corpora. This task is complicated by the extreme length of legal documents and the nuanced semantic relationships between them. We propose BERT-PLI, a novel retrieval framework that decomposes documents into paragraphs to capture fine-grained semantic interactions. By employing a three-stage cascade—comprising BM25 filtering, domain-specific BERT fine-tuning, and paragraph-level interaction modeling—BERT-PLI effectively handles long-form text while overcoming data scarcity. Experimental results on the COLIEE 2019 dataset demonstrate that BERT-PLI significantly outperforms traditional IR baselines and standard deep learning models.

Introduction

In legal practice, finding relevant precedents is essential for building strong arguments. However, legal documents often exceed 3,000 words, far surpassing the input limits of standard transformer models like BERT (typically 512 tokens). Furthermore, relevance in law is determined by complex situational logic and statutory alignment rather than simple keyword overlap. Existing deep learning models often fail in this domain due to their design for short-text matching and the lack of large-scale labeled legal datasets.

Methodology

The BERT-PLI framework utilizes a three-stage approach to balance efficiency and depth:

  1. Stage 1: BM25 Selection: A keyword-based retriever filters the corpus to identify the top $K$ candidate cases, reducing the search space for computationally intensive neural stages.
  2. Stage 2: BERT Fine-tuning: To address data scarcity, a BERT model is "warmed up" on a legal case entailment task. This step adapts the model to the specialized language and logic of legal texts.
  3. Stage 3: Paragraph-Level Interaction (PLI): This core component breaks cases into paragraphs and computes an interaction map for all query-candidate paragraph pairs using BERT.
The three-stage architecture of BERT-PLI, highlighting the paragraph-level interaction and aggregation components.

The interaction scores are aggregated via Max-Pooling to capture the strongest signals for each query paragraph. These signals are then processed by an RNN (LSTM or GRU) with an Attention mechanism to determine the final document-level relevance score.

Experimental Setup

Evaluation was conducted using the COLIEE 2019 benchmark. Task 1 (Legal Case Retrieval) provided 285 training and 61 test cases, while Task 2 (Legal Case Entailment) was used for the Stage 2 fine-tuning. The dataset primarily features Federal Court of Canada case law.

Results and Discussion

BERT-PLI demonstrates a substantial performance gain over existing methods. As shown in the results table, the BERT-PLI (GRU) variant achieved an F1 score of 0.5857, significantly higher than the BM25 baseline (0.4867) and short-text models like ARC-II (0.1873).

Comparison of BERT-PLI variants against traditional IR and deep learning baselines on the COLIEE 2019 dataset.

Ablation studies highlight the importance of the "pre-fine-tuning" stage. Removing Stage 2 (BERT-PLI$_{org}$) led to a drop in F1 score to approximately 0.51, confirming that domain adaptation is crucial for legal retrieval. The model's ability to process documents as sequences of semantic units allows it to maintain context that is lost in truncation-based approaches.

Conclusion

BERT-PLI addresses the challenges of legal case retrieval by modeling paragraph-level interactions. By combining traditional retrieval with deep semantic modeling and transfer learning, it sets a new standard for processing long-form legal documents. Future work will focus on improving computational efficiency and incorporating document metadata to further refine retrieval accuracy.

Summary updated

Abstract

Legal case retrieval is a specialized IR task that involves retrieving supporting cases given a query case. Compared with traditional ad-hoc text retrieval, the legal case retrieval task is more challenging since the query case is much longer and more complex than common keyword queries. Besides that, the definition of relevance between a query case and a supporting case is beyond general topical relevance and it is therefore difficult to construct a large-scale case retrieval dataset, especially one with accurate relevance judgments. To address these challenges, we propose BERT-PLI, a novel model that utilizes BERT to capture the semantic relationships at the paragraph-level and then infers the relevance between two cases by aggregating paragraph-level interactions. We fine-tune the BERT model with a relatively small-scale case law entailment dataset to adapt it to the legal scenario and employ a cascade framework to reduce the computational cost. We conduct extensive experiments on the benchmark of the relevant case retrieval task in COLIEE 2019. Experimental results demonstrate that our proposed method outperforms existing solutions.
BibTeX
@inproceedings{shao2020bert,
  author = {Yunqiu Shao and Jiaxin Mao and Yiqun Liu and Weizhi Ma and Ken Satoh and Min Zhang and Shaoping Ma},
  title = {{BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/484},
}
Figure