IFlyLegal: A Chinese Legal System for Consultation, Law Searching, and Document Analysis

Ziyue Wang, Baoxin Wang, Xingyi Duan, Dayong Wu, Shijin Wang, Guoping Hu, Ting LiuEMNLP 2019 · OpenReview

Abstract: Ziyue Wang, Baoxin Wang, Xingyi Duan, Dayong Wu, Shijin Wang, Guoping Hu, Ting Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations.

2019.

Introduction

The "Legal Tech" industry has historically struggled with three main issues: flexibility, diversity, and accuracy. Most existing systems are either simple "switchboards" that redirect users to human lawyers or basic Information Retrieval (IR) tools that rely on keyword matching. These systems often fail to comprehend the complex, often vague, natural language used by non-experts.

IFlyLegal addresses the need for an integrated "artificial lawyer" that can understand legal context, provide specific statutory references for arbitrary questions, and assist legal workers by automatically analyzing case descriptions to predict charges or sentences. It integrates automated legal consultation, multi-modal law searching, and judicial document analysis into a single platform.

System Architecture and Methodology

IFlyLegal is structured into three primary functional blocks, supported by a shared Natural Language Understanding (NLU) layer and a robust set of legal resources including case documents, statutes, and legal common sense.

Detailed system architecture and workflow of IFlyLegal.
  1. Consultation Block: This module uses a binary classifier to distinguish between "chit-chat" and actual legal queries. Valid queries are processed by an end-to-end QA model trained on data from legal forums. It also includes specialized modules for legal term explanation and lawyer recommendations.
  2. Law Searching Block: Beyond standard query-based search, the system introduces Natural Language Article Inference. To manage the computational expense of matching queries against thousands of statutes, it employs a two-phase "coarse-to-fine" approach. A law-level classifier first narrows the search to the top 3 most relevant Chinese laws, followed by a fine-grained matching phase using BiLSTM and fine-tuned BERT models to identify specific articles.
  3. Case Analysis Block: This block processes formal case descriptions using a 1-layer CNN for classification (civil vs. criminal) and a specialized "disconnected recurrent neural network" to predict judicial outcomes, such as criminal charges and prison terms.

Evaluation and Real-World Impact

The system demonstrates significant improvements over baseline models. The BERT-based article inference model achieved an F1-score of 89.14%, outperforming vanilla RNNs (80.64%). In human evaluations for article inference, the system achieved a precision of 76.67% at top-1 (p@1), which jumped to 90.00% when including relevant partial matches.

Example of the consultation interface in the WeChat Mini Program.

Launched as a WeChat Mini Program, IFlyLegal gathered over 28,000 page views from 2,349 unique users within its first five months. The analysis modules showed high performance, with 98.2% accuracy for criminal accusation prediction and 95.5% for civil/criminal classification.

Conclusion

IFlyLegal moves beyond simple keyword retrieval to provide nuanced legal advice and predictive case analysis. While the system is highly accurate, the authors acknowledge that neural networks often act as "black boxes." Future work will focus on explainability—providing users with the specific evidence and logic behind the model's outputs. Additionally, the team plans to develop a web-based version and explore adapting the architecture to other languages and legal systems.

Summary updated

BibTeX
@inproceedings{wang2019iflylegal,
  author = {Ziyue Wang and Baoxin Wang and Xingyi Duan and Dayong Wu and Shijin Wang and Guoping Hu and Ting Liu},
  title = {{IFlyLegal: A Chinese Legal System for Consultation, Law Searching, and Document Analysis}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=gFxD1N7bMS-},
}
Figure