Long Legal Article Question Answering via Cascaded Key Segment Learning (Student Abstract)

Shugui Xie, Lin Li, Jingling Yuan, Qing Xie, Xiaohui Tao

Introduction

Legal documents present a unique challenge for Natural Language Processing (NLP) due to their length, density, and rigorous structure. Standard models like BERT often struggle with these documents because of two primary hurdles: strict input token limits (typically 512 tokens) and the necessity of maintaining "discourse coherence." In legal contexts, the meaning of a sentence is deeply intertwined with its surrounding context; extracting isolated "evidence sentences" often destroys the logical flow required for accurate legal reasoning.

To address these issues, the CALQA (Cascaded Answer-guided key segment learning) framework was proposed. CALQA is designed to digest long articles without losing the vital context of consecutive sentences by employing a "Sifter-Reader-Responder" pipeline.

The CALQA Framework

The CALQA framework consists of three cascaded modules that work together to identify and process relevant information from long legal texts.

  1. The Sifter: Instead of selecting isolated sentences, the Sifter uses a Dynamic Programming (DP) algorithm to divide an article into "segments" of consecutive sentences. It then employs an iterative, answer-guided approach to sift through these segments in a coarse-to-fine manner, selecting the most relevant "key fact segments."
  2. The Reader: Identified key segments are passed to the Reader, which utilizes co-attention and segment-level self-attention mechanisms. This allows the model to understand the relationship between the question and specific segments, as well as the inter-segment relationships.
  3. The Responder: Acting as the final decision-maker, the Responder treats the problem as a multi-label classification task. It uses semantic representations from the Reader to predict the final answer, whether it be a text span, a "yes/no" confirmation, or an "unknown" status.
The architecture of the CALQA framework, highlighting the Sifter's coarse-to-fine selection process.

Performance Evaluation

The model was evaluated using the CAIL 2021 Law MRC dataset, where over 56% of the articles exceed the standard 512-token limit. CALQA was compared against several strong baselines, including Sliding Window approaches, CogLTX, RikiNet, and MacBERT.

Comparison of CALQA against state-of-the-art baselines and ablation study results on the CAIL 2021 dataset.

As shown in the results table, CALQA achieved a Macro-F1 score of 83.1, significantly outperforming the state-of-the-art MacBERT baseline (79.8). It also reached an Exact Match (EM) score of 65.8, a notable improvement over the 61.94 achieved by the Sliding Window baseline. Ablation studies further confirmed the importance of each module; removing the Sifter or the Reader led to substantial performance drops, validating the necessity of both intelligent filtering and deep contextual reading.

Conclusion and Future Work

CALQA demonstrates that preserving discourse coherence through segment-based processing is significantly more effective for legal reasoning than traditional sentence-level extraction. While the framework is highly effective, there are areas for improvement, such as addressing the "hard truncation" that still affects a small percentage of segments and optimizing the computational overhead introduced by the iterative sifting process. Future research may explore applying this cascaded logic to even more complex tasks, such as cross-referencing multiple legal articles simultaneously.

Summary updated

Abstract

Current sentence-level evidence extraction based methods may lose the discourse coherence of legal articles since they tend to make the extracted sentences scattered over the article. To solve the problem, this paper proposes a Cascaded Answer-guided key segment learning framework for long Legal article Question Answering, namely CALQA. The framework consists of three cascaded modules: Sifter, Reader, and Responder. The Sifter transfers a long legal article into several segments and works in an answer-guided way by automatically sifting out key fact segments in a coarse-to-fine approach through multiple iterations. The Reader utilizes a set of attention mechanisms to obtain semantic representations of the question and key fact segments. Finally, considering it a multi-label classification task the Responder predicts final answers in a cascaded manner. CALQA outperforms state-of-the-art methods in CAIL 2021 Law dataset.
BibTeX
@inproceedings{xie2023long,
  author = {Shugui Xie and Lin Li and Jingling Yuan and Qing Xie and Xiaohui Tao},
  title = {{Long Legal Article Question Answering via Cascaded Key Segment Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.27042},
}
Figure