Abstract
As Large Language Models (LLMs) are increasingly deployed in knowledge-intensive fields like law, their reasoning capabilities come under scrutiny. It remains unclear whether LLMs make judgments based on domain knowledge and logic or merely on surface-level patterns. This paper introduces J&H, an evaluation framework designed to probe the robustness of LLMs through legal knowledge-injection attacks. By targeting the components of legal syllogism—major premise, minor premise, and conclusion—we demonstrate that current LLMs, including domain-specific ones, are highly vulnerable to misleading information. Our findings suggest that LLMs often prioritize word patterns over deductive logic, posing significant risks for the "LLM-as-judge" paradigm.
Introduction: The "LLM-as-Judge" Dilemma
In the legal domain, judicial reasoning typically follows a strict syllogism: a major premise (the law), a minor premise (the facts of the case), and a conclusion (the judgment). While LLMs have demonstrated proficiency in legal examinations, it is uncertain if they are truly performing deductive reasoning or simply engaging in sophisticated pattern matching. If an LLM's judgment hinges on specific keywords rather than legal logic, it becomes susceptible to "knowledge-injection attacks"—where irrelevant or slightly incorrect legal information is introduced via prompts or retrieval systems (RAG). The J&H framework aims to determine whether LLMs act as "Justice" (logic-driven) or "Hellions" (correlation-driven).
The J&H Framework
The J&H framework systematically attacks each stage of the legal syllogism to evaluate model robustness.
1. Major Premise Attacks
These attacks target the legal statutes. Models are provided with incorrect statutes via RAG or mentions of "similar crimes" to see if they can still identify the correct law based on the facts.
2. Minor Premise Attacks
These target the factual basis of the case through three methods:
- Word Attack: Replacing legal terms with synonyms.
- Element Attack: Inserting adversarial factual elements from similar but distinct crimes.
- Narration Attack: Adding "flavor text" (e.g., "it was a fine day" vs. "it was a dark night") to test if semantic "vibes" alter the verdict.
3. Conclusion Attacks
These introduce irrelevant logical chains, such as:
- Expert Opinion Attack: Inserting sentences like "A judge thought the defendant committed [Crime X]."
- Previous Behavior Attack: Mentioning the defendant's past criminal record, which should not legally influence the current judgment.
Experimental Setup and Datasets
The study utilized two primary Chinese legal datasets: LEVEN (Legal Event Detection) and CAIL2018 (Judgment Prediction). Labels were updated to align with current Chinese criminal law, and legal experts were employed to annotate "similar crimes" and "legal synonyms" to ensure the attacks were professionally plausible. The researchers also tested defense strategies including RAG, Chain-of-Thought (CoT), and few-shot prompting.
Key Insights and Results
The researchers measured the Performance Drop Ratio (PDR)—the decrease in accuracy when an attack is present.
- High Vulnerability: Most adversarial attacks resulted in a PDR greater than 0, with many exceeding 30%.
- Conclusion Attacks are Most Effective: Attacks at the conclusion level, particularly the "Expert Opinion" attack, were the most damaging. For instance, LLaMA3’s accuracy dropped by over 44% when a "judge's opinion" was injected.
- The Fragility of CoT: Surprisingly, Chain-of-Thought prompting sometimes decreased robustness. If a model is successfully "attacked" early in its reasoning chain, it may confidently hallucinate an entire logical path to an incorrect conclusion.
- Specialization is Not a Panacea: While legal-specific models like Farui showed more robustness than general models (GPT-3.5, LLaMA3), they still suffered significant performance drops under conclusion-level attacks.
- "Lost in the Middle": Noise inserted into the middle of a prompt generally had less impact than noise at the beginning or end.
Conclusion
Current prompt-level mitigations like CoT or RAG are insufficient to ensure the logical grounding of LLMs in the legal domain. The J&H framework reveals that LLMs struggle with the nuances of domain-specific concepts and are easily misled by irrelevant context. Future work should focus on integrating domain-specific knowledge and explicit reasoning chains directly into the pre-training and fine-tuning stages to develop a truly robust "LLM-as-judge."
Summary updated