Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning

Wenbin Hu, Haoran Li, Huihao Jing, Qi Hu, Ziqian Zeng, Sirui Han, Heli Xu, Tianshu Chu, Peizhao Hu, Yangqiu SongarXiv 2025 · DOI · arXiv

Introduction

Current Large Language Model (LLM) safety and privacy measures often rely on shallow pattern matching, which frequently fails in complex, real-world scenarios. Privacy and safety are inherently context-dependent; for instance, sharing a medical record is a violation when sent to an advertiser but a necessity when sent to a doctor with patient consent. Existing "guardrails" are reactive and brittle, leading to "over-refusal" of harmless prompts or "under-protection" of sensitive data. Furthermore, these models are rarely aligned with established legal standards like the GDPR, creating systemic compliance risks. This paper introduces Context Reasoner, a framework that utilizes Contextual Integrity (CI) theory and Reinforcement Learning (RL) to move from simple pattern matching to sophisticated contextual reasoning.

Methodology

The authors propose a multi-stage pipeline to integrate legal reasoning into the model's internal processing. The framework is built on four key pillars:

  1. Contextual Integrity (CI) Formalization: Adopting CI theory, privacy is defined as the "appropriate flow of information," characterized by five parameters: Subject, Sender, Recipient, Information Type, and Transmission Principle.
  2. Hierarchical Regulation Structure: Legal documents (GDPR, HIPAA, etc.) are structuralized into a machine-readable hierarchy of Chapters, Articles, and Points, allowing the model to cite specific rules.
  3. Cold Starting with Distillation: High-quality "Chain of Thought" (CoT) trajectories are generated using DeepSeek-R1 as a teacher. These trajectories demonstrate how to analyze cases using the CI framework.
  4. Reinforcement Learning (PPO): Proximal Policy Optimization (PPO) is applied with a rule-based binary reward (+1 for correct legal conclusion, 0 otherwise), incentivizing the model to refine its internal reasoning.
Overview of the Context Reasoner framework, including hierarchical regulation structure, case database, cold starting, and RL alignment.

Experimental Setup

The study utilizes a mix of specialized legal datasets and general reasoning benchmarks:

  • PrivaCI-Bench: 6,348 real-world legal cases annotated for GDPR, HIPAA, and EU AI Act compliance.
  • Contextual Understanding MCQs: 5,844 multiple-choice questions designed to test the identification of information flow roles (e.g., "Who is the sender?").
  • General Benchmarks: MMLU (general knowledge), LegalBench (US law), LawBench (Chinese law), and TruthfulQA were used to ensure general intelligence was maintained.

Results and Discussion

The best-performing model, OpenThinker-7B-PPO, demonstrated significant improvements across all metrics.

The model achieved an average accuracy of 88.29% across safety and privacy benchmarks, representing a +17.64% improvement over the Qwen2.5-7B-Instruct baseline. Notably, the 7B model outperformed the much larger DeepSeek-R1 (671B) on several specific legal compliance tasks.

Comparison of model performance across GDPR, HIPAA, and AI ACT benchmarks, showing significant improvements for the proposed models.

General Intelligence Gains

A key finding is that training for legal reasoning acts as a "rising tide" for general intelligence. The model saw a +2.05% boost on MMLU and a massive +8.98% boost on LegalBench. This suggests that the logic required for legal parsing translates effectively to other domains.

The Importance of Cold Starting

The authors found that a "warm-up" phase using Supervised Fine-Tuning (SFT) on high-quality reasoning chains is essential. Without this cold start, RL alone is insufficient for the model to grasp the complex structure of legal analysis.

Conclusion

Context Reasoner demonstrates that safety and privacy compliance can be significantly improved by moving from pattern matching to contextual reasoning. By aligning models with established legal frameworks through CI theory and RL, developers can create more robust and compliant AI systems. Future work will address regulatory conflicts between different legal jurisdictions and the scarcity of data for emerging regulations like the EU AI Act.

The code and models are available at https://github.com/HKUST-KnowComp/ContextReasoner.

Summary updated

Abstract

While Large Language Models (LLMs) exhibit remarkable capabilities, they also introduce significant safety and privacy risks. Current mitigation strategies often fail to preserve contextual reasoning capabilities in risky scenarios. Instead, they rely heavily on sensitive pattern matching to protect LLMs, which limits the scope. Furthermore, they overlook established safety and privacy standards, leading to systemic risks for legal compliance. To address these gaps, we formulate safety and privacy issues into contextualized compliance problems following the Contextual Integrity (CI) theory. Under the CI framework, we align our model with three critical regulatory standards: GDPR, EU AI Act, and HIPAA. Specifically, we employ reinforcement learning (RL) with a rule-based reward to incentivize contextual reasoning capabilities while enhancing compliance with safety and privacy norms. Through extensive experiments, we demonstrate that our method not only significantly enhances legal compliance (achieving a +8.58% accuracy improvement in safety/privacy benchmarks) but also further improves general reasoning capability. For OpenThinker-7B, a strong reasoning model that significantly outperforms its base model Qwen2.5-7B-Instruct across diverse subjects, our method enhances its general reasoning capabilities, with +2.05% and +8.98% accuracy improvement on the MMLU and LegalBench benchmark, respectively.
BibTeX
@misc{hu2025context,
  author = {Wenbin Hu and Haoran Li and Huihao Jing and Qi Hu and Ziqian Zeng and Sirui Han and Heli Xu and Tianshu Chu and Peizhao Hu and Yangqiu Song},
  title = {{Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning}},
  year = {2025},
  eprint = {2505.14585},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.14585},
}
Figure