TL;DR
The J&H framework tests whether large language models (LLMs) make legal judgments through genuine deductive reasoning or merely rely on surface-level correlations. By injecting subtle, knowledge-level perturbations into the major premises, factual minor premises, and conclusions of legal cases, the study shows that leading general and domain-specific LLMs are highly brittle. Strikingly, popular prompt-level fixes like Chain-of-Thought (COT) or few-shot examples fail to restore robustness, revealing that prompt engineering cannot substitute for sound domain reasoning.
What Problem Does This Paper Address?
Deploying LLMs as autonomous agents or assistants in high-stakes, knowledge-intensive domains (such as law or medicine) assumes that models reason using verified domain logic. In criminal law, a human judge uses classical deductive syllogism:
- Major Premise: Applicable statutory law.
- Minor Premise: Legally relevant case facts.
- Conclusion: Guilt or specific charge.
Human experts easily ignore irrelevant distractions, such as stylistic prose, minor factual typos, or biased lay opinions. If an LLM acts as an "LLM-as-judge," does it maintain logical causal inference, or is it swayed by superficial text cues? Prior robustness benchmarks mostly focused on character- or token-level noise; this work investigates whether LLMs can withstand attacks explicitly crafted to corrupt domain logic.
Methods
The authors introduced J&H (named after Jekyll & Hyde, representing Justice & Hellion), an adversarial evaluation framework targeting each stage of legal syllogistic deduction:
- Major Premise Attacks: Misleading legal statutes are injected into the prompt context via simulated RAG (retrieval-augmented generation), or confusing similar crime definitions are introduced directly into the instructions to test if the model can still identify the correct applicable statute.
- Minor Premise Attacks: Case facts are modified without changing the underlying legal reality. This includes:
- Word Attacks: Replacing legal terms or ordinary words with legal/common synonyms.
- Element Attacks: Appending adversarial factual or statutory elements borrowed from legally similar crimes.
- Narration Attacks: Introducing environmental narrative color (e.g., "It was a stormy day" or "It's a dark night, a perfect time for murder").
- Conclusion Level Attacks: Adding logically irrelevant assertions directly before asking for the verdict, including:
- Previous Behavior Attacks: Mentioning unrelated past offenses by the defendant (which Chinese criminal law excludes from current guilt determination).
- Expert Opinion Attacks: Inserting subjective opinions attributing the act to a similar crime from characters spanning laypersons and pupils to lawyers and judges.
The authors also tested the spatial sensitivity of attacks (inserting noise at the beginning, middle, or end of prompts) and evaluated three standard defenses: RAG, Chain-of-Thought (COT), and Few-shot prompting.
Datasets
The authors evaluated the framework on two Chinese criminal law datasets:
- CAIL2018: 15,806 case documents spanning 184 criminal charges.
- LEVEN: 3,323 legal event detection documents covering 61 criminal charges.
Cases were formatted into four-option multiple-choice questions (one ground-truth charge and three confusingly similar counter-options annotated by ten law school graduate students).
Key Insights and Ideas
- LLMs crumble most at the conclusion stage: Irrelevant outside opinions and past criminal history caused the largest performance drops. When an injected sentence claimed an authority figure (lawyer or judge) believed the defendant committed an alternate crime, models frequently abandoned the correct verdict.
- Authority bias is real for LLMs: Injected opinions attributed to judges or lawyers degraded accuracy significantly more than opinions attributed to laypersons or children.
- Chain-of-Thought can backfire: Explicitly instructing models to reason step-by-step through the four elements of criminal law (subject, subjective aspect, objective aspect, object) frequently degraded robustness. LLMs hallucinated flawed legal logic chains that validated the injected distractor.
- The "Lost in the Middle" effect applies to attacks: Adversarial statements injected into the beginning or end of prompts caused the largest drop in accuracy; adversarial noise hidden in the middle had a substantially reduced effect.
- Domain-specific pre-training helps, but not enough: The legal LLM (Farui) outperformed general models (e.g., GPT-3.5-turbo, LLaMA-3), but remained susceptible to knowledge injection attacks.
Key Results
Experiments evaluated five models: GPT-3.5-turbo, Baichuan2-7B-chat, ChatGLM3-6B, LLaMA-3 (Chinese fine-tuned), and Farui (Alibaba's legal LLM), measured by Performance Drop Ratio ($\text{PDR} = 1 - \frac{\text{AttackAcc}}{\text{OriginalAcc}}$):
- General Vulnerability: Adversarial attacks induced massive degradation across all models, with PDRs frequently exceeding 30% to 40%.
- Major Premise Attacks: Supplying provisions for a similar, incorrect crime via RAG caused LLaMA-3’s accuracy to drop from 67.9% to 47.1% (a 30.63% PDR).
- Minor Premise Attacks: In LEVEN, inserting statutory elements from similar crimes dropped Baichuan2's accuracy from 77.7% to 53.4% (31.27% PDR) and LLaMA-3's from 67.9% to 43.0% (36.67% PDR).
- Conclusion Attacks: On LEVEN, attributing an incorrect charge to a "judge" dropped Farui's accuracy from 84.9% to 55.5% (34.63% PDR) and LLaMA-3's to 37.9% (44.18% PDR).
- Defense Limitations: In element attack scenarios, COT increased PDR for Baichuan2 (provisional attack PDR worsened from 31.27% to 33.42%) and Farui (12.13% to 15.14%).
Limitations and Future Work
- Language and Legal Jurisdiction: The experiments focus on statutory Chinese criminal law, leaving common-law and multi-lingual jurisdictions unexamined.
- Prompt Mitigations Are Insufficient: The paper shows that prompting tricks cannot patch foundational reasoning deficits.
- Future Work: The authors advocate moving beyond prompt-level defenses by directly integrating domain-specific deductive logic chains into the model pre-training and supervised fine-tuning phases. Additionally, they suggest applying the J&H syllogistic attack framework to other critical domains, such as medical diagnosis.
Summary updated