Modeling Motivated Reasoning in Law: evaluating Strategic Role Conditioning in LLM Summarization

Eunjung Cho, Alexander Hoyle, Yang TianUniversität 2025

TL;DR

This paper investigates how Large Language Models (LLMs) adapt their legal summaries based on specified stakeholder roles, revealing that LLMs exhibit "motivated reasoning" by selectively including information that favors the assigned role. This highlights the need for role-aware evaluation frameworks to ensure the reliability of LLM-generated legal summaries in high-stakes contexts.

What Problem Does This Paper Address?

The increasing use of LLMs for generating tailored summaries, particularly in legal settings, raises concerns about "motivated reasoning." This phenomenon, well-documented in human legal professionals, describes the tendency to interpret facts and arguments in ways that support one's institutional goals or beliefs. When LLMs are prompted to summarize legal decisions from the perspective of different legal roles (e.g., judge, prosecutor, defense attorney), there's a risk that these summaries might strategically frame information, omitting crucial countervailing facts or arguments to align with the stakeholder's position. This could lead to biased outputs that appear objective, posing challenges for legal practice where accuracy and impartiality are paramount. The paper aims to systematically evaluate this role-conditioning behavior in LLM legal summarization and develop a framework to detect such motivated reasoning.

Methods

The authors developed a systematic framework for evaluating role-conditioned legal summarization. This involved:

  1. Prompt Engineering: Crafting specific prompts in German for LLMs to generate summaries from the perspective of various legal roles (judge, prosecutor, defense attorney, plaintiff attorney, respondent attorney) and a neutral "no role" baseline.
  2. LLM Summarization: Using two LLMs (OpenAI's GPT-4o and Llama 3.3 70B-Instruct) to generate summaries of Swiss Federal Supreme Court (SFSC) decisions.
  3. Evaluation Pipeline:
    • Fact and Reasoning Extraction: Identifying all legally relevant factual statements and legal reasoning elements from the original court opinions.
  4. Favorability Classification: Labeling each extracted element as favorable, unfavorable, or neutral to a focal stakeholder (defense attorney for criminal cases, plaintiff attorney for private law cases).
  5. Inclusion Matching: Determining whether each extracted element was included in the generated summaries.
  6. Metric Computation: Calculating inclusion rates and a "favorability bias score" (difference between inclusion rates of favorable and unfavorable facts).
  7. LLM-as-a-Judge Evaluation: Employing GPT-4o and DeepSeek-R1-Distill-Llama-70B as evaluators to assess fact and reasoning inclusion and favorability bias.
  8. Human Evaluation: Conducting a smaller-scale human evaluation with legal professionals to validate the LLM-based assessments.

Datasets

  • Swiss Federal Supreme Court (SFSC) Decisions: The study utilized a dataset of 200 randomly selected SFSC opinions, equally split between private and criminal law cases.
  • Swissblawg Summaries: Expert-written summaries from Swissblawg, a prominent Swiss law blog, were used as a neutral baseline for evaluation.

Key Insights and Ideas

  • LLMs Exhibit "Motivated Reasoning" in Legal Summaries: The core finding is that LLMs, when prompted to adopt specific legal roles, exhibit selective inclusion of facts and reasoning that align with the stakeholder's perspective, mirroring human motivated reasoning.
  • Adversarial Roles Show Stronger Bias: Defense attorneys and plaintiff/respondent attorneys consistently produced summaries with higher favorability bias compared to neutral roles like judges or the "no role" baseline.
  • Fact Inclusion is More Selective Than Reasoning Inclusion: Adversarial roles tended to include fewer facts overall, indicating selective omission, while reasoning inclusion was more variable and context-sensitive.
  • "No Role" Summaries Can Still Show Bias: Interestingly, "no role" summaries sometimes included more content favorable to the focal stakeholder than stakeholder-specific summaries, suggesting that even without explicit instructions, LLMs might implicitly align with parties seeking relief.
  • Stakeholder Drift Bias: The study raises concerns about "stakeholder drift bias," where LLMs might infer user roles from interaction history and adapt outputs accordingly, even without explicit role conditioning.
  • Limitations of Standard Metrics: The findings underscore the inadequacy of traditional overlap metrics like ROUGE for evaluating role-conditioned summaries, necessitating role-aware benchmarks.

Key Results

  • Fact Inclusion: Adversarial roles (defense, respondent attorneys) included fewer facts than neutral roles (judge, "no role"), indicating selective omission.
  • Favorability Bias: Defense attorney summaries showed positive fact bias (e.g., 0.041 for DeepSeek), while prosecutor summaries were more neutral (e.g., 0.019 for DeepSeek). Reasoning favorability was generally negative across roles but less biased for defense attorneys.
  • Lexical Similarity: Adversarial roles showed higher semantic similarity (SBERT) with lower lexical overlap (ROUGE-2), suggesting reframing without losing core meaning. Institutional roles (prosecutor, judge) showed closer lexical reproduction.
  • Human Evaluation Agreement: Human judgments largely replicated LLM-as-a-Judge trends, with moderate agreement levels for inclusion and favorability, lending credence to the LLM-based findings.

Limitations and Future Work

  • Prompt Ambiguity: The use of abstract role descriptions in prompts might have introduced interpretive ambiguity, potentially flattening strategic nuances. Future work could explore case-specific prompting.
  • Granularity and Context Sensitivity: While role is a strong predictor, case-level differences also contribute to variance. Future research could investigate how doctrinal areas or normative stakes interact with role conditioning.
  • LLM-as-a-Judge Reliability: While the LLM-as-a-Judge approach showed promise, its broader reliability in high-stakes domains warrants further investigation, especially concerning hallucination and instruction sensitivity.
  • Realistic Workflows: Future research should embed LLMs in realistic legal workflows to capture how professionals prompt, revise, and negotiate ambiguity over time.
  • Subtler Rhetorical Shifts: Exploring rhetorical shifts beyond fact/reasoning inclusion and favorability (e.g., certainty language, hedging) is suggested.
  • User Studies: Post-task user studies could provide deeper insights into how legal actors assess reliability, perceive prompt sensitivity, and attribute responsibility.
  • Mitigation Strategies: Investigating prompting-based interventions and interface-level solutions (e.g., visualizing omitted content) is proposed to address bias amplification.

Summary updated

Abstract

Large Language Models (LLMs) are increasingly used to generate user-tailored summaries, adapting outputs to specific stakeholders. In legal contexts, this raises important questions about motivated reasoning — how models strategically frame information to align with a stakeholder’s position within the legal system. Building on theories of legal realism and recent trends in legal practice, we investigate how LLMs respond to prompts conditioned on different legal roles (e.g., judges, prosecutors, attorneys) when summarizing judicial decisions. We introduce an evaluation framework grounded in legal fact and reasoning inclusion, also considering favorability towards stakeholders. Our results show that even when prompts include balancing instructions, models exhibit selective inclusion patterns that reflect role-consistent perspectives. These findings raise broader concerns about how similar alignment may emerge as LLMs begin to infer user roles from prior interactions or context, even without explicit role instructions. Our results underscore the need for role-aware evaluation of LLM summarization behavior in high-stakes legal settings.
BibTeX
@misc{cho2025modeling,
  author = {Eunjung Cho and Alexander Hoyle and Yang Tian},
  title = {{Modeling Motivated Reasoning in Law: evaluating Strategic Role Conditioning in LLM Summarization}},
  year = {2025},
  eprint = {2509.00529},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.00529},
}
Figure