KoLEG: On-the-Fly Korean Legal Knowledge Editing with Continuous Retrieval

Jaehyung Seo, Dahyun Jung, Jaewook Lee, Yongchan Chun, D. S. Kim, Hwijung Ryu, Dong-Hoon Shin, Heuiseok Lim2025 · DOI

Large Language Models (LLMs) have become indispensable legal assistants, yet they struggle with the dynamic nature of law. In South Korea, statutes are amended or repealed an average of 12.2 times, with over 14,000 legislative amendments proposed between 2020 and 2022 alone. For static models, this volatility leads to "hallucinations" of outdated legal information. To address this, researchers from Korea University and KT Corporation developed KoLEG, an on-the-fly knowledge editing framework designed for surgical updates without the prohibitive cost of full retraining.

The Challenge of Temporal Volatility

Legal knowledge is cumulative and time-sensitive. Traditional editing methods like ROME or MEMIT often suffer from performance degradation after repeated edits. Furthermore, legal texts contain subtle nuances where a single digit change in an article number or a date can completely alter a legal outcome. Standard Retrieval-Augmented Generation (RAG) often fails to distinguish between nearly identical versions of the same law because they appear similar to standard retrievers.

The KoLEG Framework

KoLEG addresses these issues through two primary innovations: the LawEdit Retriever (LER) and an Editing-Aware Learning Strategy (EALS).

Overview of the KoLEG framework and the Knowledge Book containing legal evolution.
  1. LawEdit Retriever (LER): Optimized via contrastive learning to handle "hard negatives," LER is specifically trained to distinguish between laws that differ only by implementation period, article number, or specific clauses. It achieves a Top-1 retrieval accuracy of over 95%.
  2. Editing-Aware Learning Strategy (EALS): Utilizing LoRA (Low-Rank Adaptation), this strategy employs a multi-task loss function covering Reliability, Portability, Locality, and a novel "Locality+" loss. This allows the model to ignore irrelevant "distractor" information and autonomously decide when to apply an edit versus relying on its internal parametric knowledge.

Experimental Setup and Datasets

The study introduced the Korean Legislative Amendment Dataset, the first of its kind for this task. It features a "Knowledge Book" of 1,260 statutes with full amendment histories since 1970. The test suite includes 799 single-update samples and 345 sequential-update samples to track laws through multiple revisions.

Results and Discussion

KoLEG demonstrates state-of-the-art performance, achieving 97.7% accuracy on single updates using the KULLM3 backbone, surpassing baselines by up to 30 percentage points. Crucially, it maintains stability over sequential edits where other methods fail.

Performance comparison showing KoLEG's stability against baselines across cumulative legal updates.

As shown in the performance metrics, while traditional methods like ROME and MEMIT see sharp declines after multiple updates, KoLEG maintains performance above 95%. In practical reasoning tasks, such as matching precedents to edited statutes, KoLEG outperformed base models by 25.2%, proving it can reason with edited knowledge rather than just memorizing it. Expert validation by licensed attorneys further confirmed KoLEG's superiority, with a 96% preference rate due to its coherent and accurate legal phrasing.

Conclusion

KoLEG provides a highly efficient, zero-edit-time solution for keeping legal LLMs current. By combining specialized retrieval with an editing-aware model, it overcomes the temporal volatility inherent in the legal domain. While currently focused on the Korean statutory system, future work aims to expand this framework to include case law and multi-jurisdictional support.

Summary updated

BibTeX
@misc{seo2025koleg,
  author = {Jaehyung Seo and Dahyun Jung and Jaewook Lee and Yongchan Chun and D. S. Kim and Hwijung Ryu and Dong-Hoon Shin and Heuiseok Lim},
  title = {{KoLEG: On-the-Fly Korean Legal Knowledge Editing with Continuous Retrieval}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.489},
}
Figure