UniLR: Unleashing the Power of LLMs on Multiple Legal Tasks with a Unified Legal Retriever

Abstract

Large Language Models (LLMs) often suffer from hallucinations in specialized domains like law, where factual accuracy is critical. Retrieval-Augmented Generation (RAG) offers a solution, but the legal domain presents unique challenges: format diversity between concise law articles and verbose case descriptions, subtle legal nuances, and task fragmentation. We present UniLR (Unified Legal Retriever), a novel framework that bridges these gaps. UniLR utilizes attention supervision on "key elements" and a heterogeneous meta-information graph to create a format-agnostic, context-aware retriever. Our evaluations across multiple benchmarks show that UniLR significantly outperforms existing general and legal-specific retrievers, effectively grounding LLMs in factual legal knowledge.

Introduction

In the legal field, accuracy is paramount; a single misinterpreted word can alter a charge from "theft" to "robbery." While LLMs show promise, their tendency to hallucinate makes them risky for legal applications. Standard RAG approaches struggle with three primary hurdles in law:

1. Format Diversity: Law articles are brief and terminological, whereas legal cases are lengthy narratives. Standard retrievers often fail to match these disparate formats.
2. Subtle Nuances: Legally distinct concepts often share similar linguistic descriptions, leading to retrieval errors between "confusing" law articles.
3. Task Fragmentation: Most retrievers are specialized for a single task (e.g., case retrieval), lacking the versatility needed for complex, multi-step legal workflows.

Expand

Source, p. 1

Methodology

UniLR addresses these challenges through a two-stage process: Knowledge Mining and a specialized Model Architecture designed to capture the "essence" of legal knowledge regardless of its presentation format.

Knowledge Mining

To capture the underlying logic of legal information, UniLR extracts two types of hidden knowledge:

• Key Elements: Using a distilled legal event model (DMBERT), the system identifies critical "who, what, and where" components (subjects, objects, triggers) within legal facts.
• Meta-Information Graph: A heterogeneous graph is constructed to link knowledge nodes to their hierarchical positions in the legal system (e.g., "Property Infringement") and specific legal event schemas.

Model Architecture

The retriever, built upon the Lawformer backbone, incorporates two innovative components:

• Key Element Supervisor (KES): KES employs attention supervision by calculating the KL-divergence between the model's attention and the "ideal" attention on extracted key elements. This forces the model to prioritize legally significant tokens over narrative "fluff."
• Graph-based Knowledge Augmenter (GKA): GKA uses graph attention and convolution to integrate context from the meta-information graph, allowing the model to leverage hierarchical relationships to disambiguate similar legal concepts.

Expand

Source, p. 4

Evaluation and Results

UniLR was evaluated on four major benchmarks: CAIL2018 (Article & Charge), LAIC2021-CVG (Court View Generation), and Lawzhidao-QA.

• Retrieval Performance: UniLR achieved a Hit@3 of 84.88% on the Article Prediction task, outperforming the state-of-the-art legal retriever SAILER by over 4%.
• LLM Enhancement: When integrated with models like GPT-4 and GLM4, UniLR significantly improved downstream performance. In Legal QA, it boosted the ROUGE-L score by 12.71% compared to the next best baseline.
• Qualitative Analysis: Visualization of the embedding space indicates that UniLR creates tighter, more distinct clusters for different legal charges, demonstrating its superior ability to handle "confusing" cases that share similar language but different legal hierarchies.

Conclusion

UniLR demonstrates that by focusing on the underlying "essence" of legal events rather than surface-level formats, a unified retriever can effectively serve diverse legal tasks. By incorporating hierarchical meta-information and supervising attention on key elements, UniLR provides a robust foundation for building reliable, RAG-enhanced LegalAI systems. Future work will explore generalizing these methods to other jurisdictions and languages.

@inproceedings{li2025unilr,
  author = {Ang Li and Yiquan Wu and Yifei Liu and Ming Cai and Lizhi Qing and Shihang Wang and Yangyang Kang and Chengyuan Liu and Fei Wu and Kun Kuang},
  title = {{UniLR: Unleashing the Power of LLMs on Multiple Legal Tasks with a Unified Legal Retriever}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}