Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution

Calvin Bao, Connor Baumler, Hal Daumé, Marine Carpuat2025 · DOI

Abstract

Despite growing interest in explainable NLP, it remains unclear how explanation strategies shape user behavior in tasks like authorship identification, where relevant textual features may be difficult for lay users to pinpoint. To support their analysis of text style, we consider two explanation types: example-based style rewrites and feature-based rationales, generated using a LLM-based pipeline. We measured how explanations impact user behavior in a controlled study (n=95) where participants completed authorship identification tasks with our types of assistance. While no explanation type improved overall task accuracy, fine-grained reliance patterns revealed that rewrites supported appropriate reliance, whereas presenting both explanation types increased AI overreliance, minimizing participant self-reliance. We find that participants exhibiting better reliance behaviors had focused explanation needs, contrasting with the diffused preferences of those who overrelied on AI, or incorrectly self-relied. These findings highlight the need for adaptive explanation systems that tailor support based on specific user reliance behaviors.

The Challenge of AI Reliance in Subjective Tasks

As AI becomes a standard "co-pilot" for decision-making, researchers are grappling with the reliance problem: how do we get humans to trust AI when it’s right but override it when it’s wrong?

Most studies on AI reliance focus on "verifiable" tasks, like factual Q&A, where a user can eventually check the truth. This paper tackles authorship attribution—identifying who wrote a specific text based on subtle stylistic markers. This task is inherently interpretive and subjective. Because there is no "ground truth" easily accessible to the user, they must rely heavily on how the AI frames its reasoning. The authors investigate whether different types of explanations—specifically feature-based rationales versus example-based rewrites—help or hinder a user's ability to calibrate their trust.

Experimental Design

The researchers conducted a controlled online study with 95 participants. The task was a binary choice: given a "Mystery Post" and two samples from different authors, which author wrote the mystery text?

The core task: participants compare a 'Mystery Text' against known writing samples from two candidate authors.

The study used a two-phase design:

  1. Phase 1 (Human-only): Participants make a choice based on their own analysis.
  2. Phase 2 (AI-assisted): Participants see an AI prediction and are randomly assigned to one of four explanation conditions:
  3. Prediction-only: Just the AI's guess.
  4. Stylistic Rationales: A feature-based explanation listing stylistic similarities (e.g., "Both use passive voice").
  5. Author-style Rewrites: An example-based explanation where the AI rewrites the Mystery Post in the style of each candidate author.
  6. Dual Support: Both rationales and rewrites combined.

The study utilized the PAN 2023 Style Change Detection dataset, ensuring participants had to rely on stylistic cues (syntax, punctuation, vocabulary) rather than just topical keywords.

Results: The Paradox of More Explanations

The study revealed several counter-intuitive findings regarding how explanations affect user behavior:

  • The Confidence Trap: Every type of AI support significantly boosted user confidence, but this didn't translate into higher accuracy.
  • The "Short-Circuit" Effect: When users were given the "Dual" treatment (both rationales and rewrites), their perceived mental workload dropped. While this sounds like a win for UX, it actually led to the lowest "Relative Self-Reliance." Users essentially stopped double-checking the AI and started "short-cutting" their own reasoning process.
The Dual Support condition provides both stylistic rewrites and feature-based rationales, which was found to increase user overreliance.
  • Rewrites Promote Critical Thinking: Example-based rewrites (counterfactuals) were the most effective at helping users maintain "appropriate reliance." They provided enough friction to keep the user engaged, leading to the highest resistance against incorrect AI suggestions.
  • Overreliance in Dual Support: In the Dual Support condition, accuracy dropped most sharply when the AI was incorrect. The average accuracy decrease was statistically significant ($t = -4.23, p = 0.0003$).

Discussion: Towards Adaptive AI Support

The findings suggest that "more is not always better" in AI explanations. Participants exhibiting better reliance behaviors had focused explanation needs, contrasting with the diffused preferences of those who overrelied on AI.

Future work should explore adaptive explanation systems. Instead of a "one-size-fits-all" approach, an AI could sense a user's reliance pattern and tailor the explanation—perhaps providing more "friction" (like rewrites) if the user is showing signs of blind trust, or more "support" (like rationales) if the user is struggling to find any signal in the noise.

Summary updated

Abstract

Despite growing interest in explainable NLP, it remains unclear how explanation strategies shape user behavior in tasks like authorship identification, where relevant textual features may be difficult for lay users to pinpoint.To support their analysis of text style, we consider two explanation types: example-based style rewrites and feature-based rationales, generated using a LLM-based pipeline.We measured how explanations impact user behavior in a controlled study (n=95) where participants completed authorship identification tasks with our types of assistance.While no explanation type improved overall task accuracy, fine-grained reliance patterns (Schemmer et al., 2023) revealed that rewrites supported appropriate reliance, whereas presenting both explanation types increased AI overreliance, minimizing participant self-reliance.We find that participants exhibiting better reliance behaviors had focused explanation needs, contrasting with the diffused preferences of those who overrelied on AI, or incorrectly self-relied.These findings highlight the need for adaptive explanation systems that tailor support based on specific user reliance behaviors.
BibTeX
@misc{bao2025whos,
  author = {Calvin Bao and Connor Baumler and Hal Daumé and Marine Carpuat},
  title = {{Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.1380},
}
Figure