Incorporating Legal Logic into Deep Learning: An Intelligent Approach to Probation Prediction

Qinghua Wang, Xu Zhang, Lingyan Yang, Rui Shao, Bonan Wang, Fang Wang, Cunquan QuIJCAI 2025 · DOI

TL;DR

This paper introduces a novel approach to predicting probation eligibility by integrating legal logic into deep learning models. By constructing a specialized dataset with legal elements and developing a multi-task learning framework, the authors achieve superior prediction accuracy and offer insights into the judicial decision-making process.

What Problem Does This Paper Address?

The paper tackles the challenge of accurately predicting probation eligibility, a crucial aspect of the judicial system that balances rehabilitation with public safety. While AI has been increasingly adopted in judicial assistance systems (IJAS) to improve efficiency, existing methods often rely on data-driven approaches that overlook the nuanced legal logic underpinning judicial decisions. Specifically, current IJAS lack dedicated methods for probation prediction, and research on the factors influencing probation eligibility is limited. This gap means that AI systems might not fully grasp the complexities of legal reasoning, potentially leading to less accurate or interpretable predictions. The authors highlight that probation eligibility requires a comprehensive analysis of both the criminal's circumstances and their remorse, which traditional data-driven models may not adequately capture.

Methods

The authors propose a three-stage approach to integrate legal logic into deep learning for probation prediction:

  1. Dataset Construction: They created a specialized probation dataset that includes not only factual descriptions of cases but also "Probation Legal Elements" (PLEs). These PLEs are derived from legal interpretations and are crucial for understanding the underlying legal logic.
  2. Model Development (MT-DT): They developed the "Multi-Task Dual-Theory Probation Prediction Model" (MT-DT). This model is grounded in the legal logic of probation and the "Dual-Track Theory of Punishment." The MT-DT model operates in a multi-task framework, treating the prediction of prerequisites (like sentence length) as an auxiliary task to the main task of predicting probation eligibility. This allows for simultaneous optimization of both aspects.
  3. Legal Interpretation Encoding: A key innovation is the "Legal-Enhanced Sequence Generator," which encodes the legal interpretation of PLEs. This allows the model to better simulate the judicial decision-making process by explicitly incorporating legal reasoning.

The MT-DT model uses a two-stage prediction process. First, it predicts if the criminal's sentence meets the prerequisite of three years or less in prison. If it does, it then uses the text sequence of PLEs as input for the second stage to predict probation eligibility. The model's training involves a combined loss function that balances the auxiliary and main tasks.

Datasets

The authors constructed a new dataset for probation prediction. This dataset comprises 29,105 judgment documents from China, specifically focusing on intentional injury cases with a single defendant and charge, spanning from 2011 to 2021. Of these, 8,351 cases (28.69%) involved probation. The dataset was split into training (80%), validation (10%), and testing (10%) sets.

Key Insights and Ideas

  • Legal Logic is Crucial: The paper strongly emphasizes that simply using data-driven methods is insufficient for accurate legal prediction. Incorporating explicit legal logic, such as PLEs, significantly improves model performance and interpretability.
  • Dual-Track Theory of Punishment: The model's design is informed by the "Dual-Track Theory of Punishment," which considers both retributive and preventive factors. This theoretical grounding helps in a more balanced and comprehensive assessment of probation eligibility.
  • Addressing Error Amplification: The MT-DT model's multi-task approach, where the prerequisite prediction is an auxiliary task, effectively mitigates the "error amplification" problem seen in cascaded models. This is crucial because errors in the initial stage can disproportionately affect the final prediction.
  • PLEs Enhance Prediction: The ablation studies demonstrate that using PLEs (especially as text sequences) significantly improves prediction accuracy compared to using only fact descriptions. This highlights the value of structured legal information.

Key Results

The MT-DT model demonstrated superior performance compared to baseline models.

  • Overall Accuracy: The MT-DT model achieved an accuracy of 88.15% for the main probation prediction task (Task 2), outperforming previous models.
  • Comparison with Baselines: Table 3 shows that MT-DT (88.15% ACC) significantly outperforms various neural network models (e.g., TextCNN at 81.70%) and pre-trained language models (e.g., Lawformer at 84.42%).
  • Ablation Study: Using the text sequence of PLEs (variant C) achieved the highest accuracy (88.15%), outperforming using only fact descriptions (variant A, 82.30%) and fact descriptions with vector embeddings of PLEs (variant B, 83.42%).
  • Auxiliary Task Impact: Figure 6 shows that the MT-DT model's performance stabilizes in the range of 86% to 88% when the auxiliary task loss is incorporated, significantly surpassing baseline performance.

Limitations and Future Work

  • Limitations: The paper acknowledges the class imbalance in the dataset, which can lead to suboptimal model performance, especially in cascaded training. The accuracy of the prerequisite prediction task (Task 1) not reaching 100% can lead to error amplification in Task 2.
  • Future Work: The authors plan to conduct a deeper investigation into factors influencing probation sentence lengths and systematically analyze underlying legal principles. They also aim to expand their dataset to include a broader range of charges and incorporate more sophisticated legal reasoning frameworks to enhance the model's robustness and generalizability.

Summary updated

BibTeX
@inproceedings{wang2025incorporating,
  author = {Qinghua Wang and Xu Zhang and Lingyan Yang and Rui Shao and Bonan Wang and Fang Wang and Cunquan Qu},
  title = {{Incorporating Legal Logic into Deep Learning: An Intelligent Approach to Probation Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/868},
}
Figure