TL;DR
This paper introduces GNNRR, a novel Graph Neural Network (GNN) based approach for predicting rhetorical roles in long legal documents. By effectively modeling inter-sentence dependencies and incorporating multitask learning with label shift prediction and contrastive learning, GNNRR achieves a top-4 ranking in the SemEval-2023 Task 6.
What Problem Does This Paper Address?
Understanding legal documents is crucial for legal systems, but their sheer length and complex structure pose significant challenges for automated processing. Specifically, the task of predicting "rhetorical roles" (e.g., identifying the purpose of each sentence like "argument," "evidence," or "conclusion") in these lengthy texts is difficult because traditional methods struggle to capture long-range dependencies between sentences. Furthermore, the presence of semantically similar but distinct labels (like "ARG_PETITIONER" vs. "ARG_RESPONDENT") introduces ambiguity, hindering accurate prediction.
Methods
The GNNRR model tackles these challenges through a multi-pronged approach:
- GNN-based Context Representation: The core innovation is a Graph Neural Network (GNN) that models relationships between words and sentences. This GNN uses directed edges for sequential words within a sentence and undirected edges between words that share entities (identified by a Legal-NER model). This allows for a richer representation of the document's context than simple sequential models.
- Multitask Learning (MTL): The model jointly optimizes two tasks: rhetorical role prediction (the main task) and label shift prediction (LSP). LSP, which predicts whether adjacent sentences have different labels, acts as an auxiliary task to help the model learn better sentence representations and relationships.
- Contrastive Learning: To address label ambiguity, the authors employ Supervised Contrastive Learning (SupCon). This technique encourages the model to learn an embedding space where sentences with the same rhetorical role are pulled closer together, while sentences with different roles are pushed apart.
- Sentence Encoder: For initial sentence representation, the model leverages InLegalBERT (a BERT model fine-tuned on legal texts) combined with BiLSTM to capture contextual information. This is followed by a k-head attention pooling mechanism to generate sentence embeddings.
Datasets
The primary dataset used is from SemEval-2023 Task 6 - LegalEval Understanding Legal Texts. This dataset contains long, unstructured legal documents segmented into sentences, with each sentence labeled with one of 13 rhetorical roles (12 semantic roles + NONE). The paper does not mention introducing a new benchmark dataset.
Key Insights and Ideas
- GNNs are powerful for long-text dependencies: The success of the GNN component highlights its ability to capture complex inter-sentence relationships that are missed by purely sequential models, which is critical for lengthy legal documents.
- Multitask learning boosts performance: Using label shift prediction as an auxiliary task significantly improves the model's ability to understand sentence relationships and ultimately predict rhetorical roles more accurately.
- Contrastive learning combats ambiguity: The application of contrastive learning effectively addresses the challenge of similar-sounding labels, leading to more discriminative representations.
- Combining multiple techniques is key: The paper demonstrates that a synergistic combination of GNNs, multitask learning, and contrastive learning yields superior results compared to individual components or simpler baselines.
Key Results
The GNNRR model achieved a Micro-F1 score of 84.3% on the development set and 83.89% on the test set. This performance placed the team top 4 on the public leaderboard for the Rhetorical Roles sub-task. Ablation studies showed that each component (GNN, multitask learning, contrastive learning) contributed positively to the overall performance, with the full GNNRR model outperforming simplified versions.
Limitations and Future Work
- Limited Data: The paper notes that with only 247 documents for training, representing the document effectively is challenging.
- Entity Extraction Dependency: The GNN's undirected edges rely on entity information extracted by a separate Legal-NER model. Errors in entity extraction could propagate to the GNN.
- Future Work: While not explicitly detailed, the success of the GNN-based approach suggests further exploration into more sophisticated graph structures and attention mechanisms for long-text representation. The authors also mention the potential for their approach to be applied to other legal NLP tasks.
Summary updated