A Layer-wise Study of How Style Rewriting Breaks the Calibration of Zero-Shot LLM Fake-News Detectors

Prabhjot Singh, Praney Goyal, Raksha SharmaarXiv 2026 · arXiv

TL;DR

Style rewriting—changing the tone of a news article without altering its facts—can trick Large Language Models (LLMs) into becoming "confidently wrong" about fake news. This study reveals that while LLMs normally have a late-stage "self-correction" mechanism that keeps their confidence in check, sensational or emotional styles break this process, causing models to perform worse than random guessing while remaining highly certain of their incorrect answers.

What Problem Does This Paper Address?

As LLMs are increasingly deployed as zero-shot fake-news detectors, we often rely on their stated confidence as a safety signal. If a model is unsure, we might flag the article for human review. However, recent "style attacks" have shown that rewriting a fake article to sound authoritative can tank a detector's accuracy.

The authors of this paper ask a critical follow-up question: What happens to the model's confidence during these attacks? If accuracy drops but confidence remains high, the model becomes a "confidently wrong" judge—the most dangerous regime for deployment because it bypasses human oversight. The researchers use mechanistic interpretability tools to look "under the hood" and see how this overconfidence forms layer-by-layer.

Methods

The researchers evaluated three popular instruction-tuned model families: LLaMA-3-8B, Qwen2.5-7B, and Mistral-7B. They employed a technique called the logit lens, which allows them to "read" the model's internal state after every single transformer layer. By projecting the residual stream through the model's final output layer (the unembedding), they could track how the prediction and the confidence (calibration) evolved as the data moved through the network.

They tested the models in a zero-shot setting, asking them to classify articles as "real" or "fake." To simulate attacks, they used GPT-3.5 to rewrite articles into four distinct styles:

  1. Reliable styles: Objective and Neutral.
  2. Unreliable styles: Sensational and Emotionally Triggering.

Datasets

The study utilized three established fake-news benchmarks:

  • PolitiFact (90 test samples)
  • GossipCop (1,584 test samples)
  • LUN (1,500 test samples)

Each dataset included original articles and four adversarial "reframings" per article, designed to disguise fake news as reputable sources (e.g., CNN style) or real news as tabloids (e.g., National Enquirer style).

Key Insights and Ideas

  • The "Honest Judge" Pattern: On original, unmanipulated articles, LLMs are surprisingly honest. They typically settle on an answer in the middle layers, become briefly overconfident, and then—crucially—perform a late-layer calibration where they dial back their confidence to match their actual accuracy.
  • Upper-Layer Fragility: Style attacks specifically break this self-correction phase. The failure is co-located in the final layers where the model usually fixes its overconfidence.
  • Systematic Inversion: Under the strongest style attacks, models don't just become "uncertain"; they actually perform worse than random guessing (accuracy < 0.5) while maintaining confidence levels near 95%. They are effectively "tricked" into a systematic inversion of the truth.
  • Style as a Switch: The damage is highly selective. Neutral or objective rewrites leave the model's calibration intact. However, sensational or emotional styles act like a switch that collapses the model's ability to separate fact from fiction, likely because the "alarmist" register makes all articles look "fake-like" to the model's internal heuristics.

Key Results

The quantitative results show a stark contrast between original and adversarial performance:

  • Accuracy Collapse: Final-layer accuracy dropped to between 56% and 69% of the original values across all models.
  • Calibration Error (ECE): The Expected Calibration Error rose sharply by +0.23 to +0.35. A perfectly calibrated model has an ECE of 0; these jumps indicate massive overconfidence.
  • Robustness Gradient: While Mistral-7B was the most robust of the three, it still exhibited the same qualitative pattern of confident error.
This figure shows that while original tests (teal) have high accuracy and low calibration error, adversarial attacks (coral) drive accuracy toward chance while calibration error (ECE) spikes.

As seen in the layer-wise analysis, the "gap" between original and adversarial performance opens up specifically in the final third of the network:

Layer-wise evolution shows accuracy (top) saturating for original text but stalling for adversarial text, while ECE (bottom) fails to drop in the final layers under attack.

Limitations and Future Work

The authors acknowledge a few key limitations:

  • OOD vs. Veracity: It is currently unclear if sensational rewrites break "veracity judgment" specifically, or if they are simply "out-of-distribution" (OOD) data that would degrade the model's performance on any task (like topic classification).
  • Single Generator Bias: All style rewrites were generated by GPT-3.5. It’s possible that the "sensational" style of one specific model is particularly homogeneous, making the attack easier.
  • Future Directions: The researchers suggest a "same-articles control task" to separate general OOD degradation from veracity-specific fragility. They also note that an LLM's confidence should never be used as a "decision gate" for content that might have been restyled.

Summary updated

Abstract

Large language models (LLMs) are increasingly asked to judge whether a news article is true, and their stated confidence is often taken at face value. We test what happens to that confidence when an article's writing style, but not its facts, is manipulated, as in recent \emph{style attacks} on fake-news detectors. Tracing how the prediction and its confidence form layer by layer inside the network, we study three model families (LLaMA-3-8B, Qwen2.5-7B, Mistral-7B) on three fake-news benchmarks, and find the same three patterns in every model--dataset pairing. First, on the original articles the models are competent and largely honest judges: they settle on an answer, briefly become overconfident, and then correct their confidence in the final layers. Second, style-rewritten articles break this self-correction: the models become far less accurate, under the strongest attack, worse than random guessing, yet remain just as confident, so they are \emph{confidently wrong} precisely when a warning is most needed. Third, nearly all of the damage comes from rewriting into sensational or emotionally charged styles, while sober, neutral rewrites leave the models' honesty largely intact. The practical implication is that an LLM's confidence about veracity cannot be trusted as a safety signal when an article's style may have been manipulated.
BibTeX
@misc{singh2026layer,
  author = {Prabhjot Singh and Praney Goyal and Raksha Sharma},
  title = {{A Layer-wise Study of How Style Rewriting Breaks the Calibration of Zero-Shot LLM Fake-News Detectors}},
  year = {2026},
  eprint = {2611.11239},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11239},
}
Figure