Style-Blind by Construction: Invertible Role Factoring for Misinformation and Hallucination Detection

Fake news detectors often succeed for the wrong reason: they learn that sensational, emotional or informal writing tends to be false, rather than checking what the text actually claims. They therefore break when someone rewrites a false story in a calm, newsroom voice, which a language model can now do for free. This proposal builds a detector whose decision depends only on who did what to whom, and that cannot see writing style because of how it is wired, not because it was trained on enough examples to ignore it. The detector maps each sentence into a latent code through an invertible network, meaning a learned transformation that can be run backwards to recover its input exactly. That code is split into coordinates for each semantic role (the predicate, its agent, its object), plus a residual block that holds everything else, including style. The classifier reads only the role coordinates.

Invertibility matters here for one reason. It makes the split auditable. Because nothing is thrown away, the style information must end up somewhere, and we can check that it ended up in the residual. We do this by decoding sentences with the residual swapped, and by testing whether the role coordinates predict style. A standard encoder trained to ignore style gives no such check: style may have been removed, or it may be hidden in the same coordinates the classifier uses. The same role-level code also addresses a second problem. False details in real misinformation, and in model hallucinations, usually sit in a single argument of an otherwise true sentence, so a detector that compares claim and evidence role by role can say which argument is unsupported.

Core Claim

A claim-versus-evidence verifier that reads only the role-content block of an invertible, role-factored sentence latent will stay accurate under style-rewriting attacks it never saw in training. Detectors made robust by training on rewritten examples should lose accuracy on new attack families. The same verifier should also localize false arguments in deceptive news and in model-generated text better than whole-sentence classifiers.

The claim is falsifiable in a sharp way. If a conventionally trained detector, given the same budget of rewritten training examples, matches our detector on held-out attack families, then the architectural guarantee buys nothing, and the proposal fails on its main point.

Where This Sits

The motivating evidence comes from the fake news literature. Automatic Fake News Detection: Are current models "fact-checking" or "gut-checking"? shows that BERT-style detectors on MultiFC exploit emotional and stylistic cues. Crude removal of those cues, by filtering words by part of speech or dropping stopwords, raises macro F1, but at the cost of discarding rhetorical signal. Fake News in Sheep's Clothing shows that rewrites by a language model in a different style are a practical attack on such detectors. We take both papers as the problem statement. Our response is different from either: we neither delete style tokens nor rely only on training with attacked examples. Instead, we route style into coordinates the classifier cannot read.

Granularity is motivated by two further results. MisSpans shows that real misinformation mixes true clauses with localized false details, and that 15 language models score below 0.5 F1 at finding and typing those spans. Localizing Factual Inconsistencies in Attributable Text Generation finds that 27% of hallucination errors sit inside a single predicate, where some arguments are supported and others are not. Its QASEMCONSISTENCY method decomposes sentences into question-answer pairs about each predicate using an external 3B T5 parser, and reaches Spearman ρ = 0.73 with human judgments. We borrow its predicate-argument view of errors. We differ in learning a continuous role-factored code rather than running an external question-answering pipeline at test time.

The method comes from Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks. That work trains a flow-based invertible network on top of a BERT-GPT2 sentence autoencoder, with cluster supervision from argument-role annotations. It reports reconstruction "invertibility ratios often above 90%" and better role separability than the Optimus variational autoencoder. It was evaluated only on clean scientific explanations (WorldTree, EntailmentBank) and never used for veracity. We reuse its two-stage design. We add an explicit residual block and train it to absorb style, and we move the model onto noisy, deceptive news text with weak role labels from an off-the-shelf semantic role labeler. The broader claim that invertible networks can classify well without discarding input information comes from i-RevNet. The idea of pulling decision boundaries back to input space for exact attribution comes from Decision Explanation and Feature Importance for Invertible Networks.

Two papers already connect hallucination and fake news. Both differ from us in kind. PHPFND uses the locations where a language model hallucinates while processing a news item as a signal of deceptive regions. It relies on event-tuple extraction, retrieval and iterative prompting, and reports 42% and 46% error-rate reductions on hallucination-inducing samples of Weibo21 and Pheme. Is LLMs Hallucination Usable? generates deliberately wrong "negative reasoning" with an iterative prompting loop, trains a dual-BERT teacher on news-reasoning pairs, and distills it into a text-only student, NRFE-D. Neither paper makes any claim about invariance to style, and neither yields role-level localization. We compare against NRFE-D directly.

One caveat constrains the design. Only the invertible network is exactly bijective. The autoencoder beneath it is lossy, so the "nothing is discarded" guarantee holds between the autoencoder latent and the factored code, not between the text and the code. We therefore measure autoencoder reconstruction on news text explicitly, and report results separately for sentences it reconstructs well and poorly.

Research Program

The system has four parts.

First, a BERT-GPT2 sentence autoencoder is pre-trained on news and claim sentences from PolitiFact, LIAR, MultiFC and the MisSpans source stories.

Second, an invertible flow maps the autoencoder latent $u$ to a factored code $z = f(u) = (z_{\text{pred}}, z_{\text{arg0}}, z_{\text{arg1}}, z_{\text{mod}}, z_{\text{res}})$. The first four blocks hold role content (predicate, agent, patient, and time/place modifiers). The last block, $z_{\text{res}}$, is the residual. The role blocks are trained with the cluster objective of the explanation-space paper, using weak labels from semantic role labeling. To push style into the residual, we train on pairs of sentences produced by a language model that differ only in style (formal versus sensational, neutral versus emotional). The objective requires their role blocks to match and lets their residuals differ. An adversary also tries to predict a style label from the role blocks, and the flow is trained to defeat it.

Third, a verifier encodes a claim and its evidence sentences. It compares them block by block, using only role blocks, and outputs a label for each role plus an aggregate label. Because each role block is tied to a role span from semantic role labeling, the per-role scores become span predictions in MisSpans format.

Fourth, an audit procedure exploits invertibility. For any claim, we swap its residual with that of a stylistically different sentence and decode through $f^{-1}$ and the GPT-2 decoder. We then check two things: that the decoded text kept its role content, and that the verifier output is unchanged. The latter holds exactly by construction. The open question is whether the decoded text really is the same claim in a different style.

The analysis asks three questions. Does the residual carry the style information, which we measure by probing accuracy for formality and emotion from each block? Do the role blocks carry the factual content, which we measure by verification accuracy when only role blocks are used? Where does the factoring leak? The audit's decoded rewrites are scored by human raters on a sample, with a natural language inference model as an automatic proxy.

On the hallucination side, the same verifier is applied to model-written summaries and biographies. The source document or reference serves as evidence, and the verifier flags role-level unsupported arguments. This tests whether a representation trained on deceptive human text transfers to fluent machine fabrication. That transfer is plausible, since both look like a correct sentence frame with one wrong filler, such as the entity and role swaps documented in Léo Milecki.

Baselines

The naive reference point is a RoBERTa or BERT claim-plus-evidence classifier, fine-tuned on each benchmark with no style handling. The "gut-checking" paper used this baseline and showed it relies on affect. It sets the floor for clean accuracy and the expected size of the accuracy drop under attack.

The strongest existing detector on our datasets is NRFE-D from Is LLMs Hallucination Usable?. It reports accuracy of 0.857 on PolitiFact and 0.971 on Twitter-16, 3.2% above standard BERT on PolitiFact. We will rerun it from the described recipe on our splits rather than copy numbers, since our attacked test sets are new. Beating it means matching its clean accuracy within about one point while losing substantially less accuracy on held-out style attacks. We do not expect to beat it on clean accuracy and will not claim to.

The most direct competitor on the robustness question is a style-augmented detector. We follow the attack-and-retrain approach of Fake News in Sheep's Clothing, training a RoBERTa detector on the same style-paired data our flow sees. No reported result was found on Lacuna for that paper, so the comparison is entirely on our runs. Fairness here is the crux. Both systems get the identical set of rewritten training pairs, the same backbone size, the same number of tuning trials, and the same compute. Both are then tested on attack families excluded from training, for example training on formality and emotion rewrites and testing on sensationalizing, hedging, and persona rewrites. Our detector beats it if, averaged over the held-out families, it loses at least half as much accuracy relative to clean data as the baseline loses, with confidence intervals from five seeds.

We also include the part-of-speech and stopword filtering baselines from the gut-checking paper, which reports gains on MultiFC but no numbers on Lacuna. For localization, we compare against QASEMCONSISTENCY (ρ = 0.73 with human judgments) on its CLIFF, FActScore and Verifiability data. On MisSpans, the reference is the reported range of below 0.5 F1 for prompted language models, together with a fine-tuned token tagger described below.

Two ablations isolate the key ingredient. The first replaces the invertible flow with Optimus plus a learned linear split into the same blocks, trained with the same losses. This tests whether exact invertibility, rather than the factoring losses, produces the robustness and the trustworthy audit. The second keeps the invertible flow but lets the verifier read the residual block too. This tests whether excluding the residual is what does the work.

Why not just…?

Why not just train a standard detector on many style-rewritten examples? This is the real threat. If rewrites are cheap, a standard encoder may learn invariance to style that is good enough. Our bet is that augmentation teaches invariance to the rewrites it has seen, while routing style into a block the classifier cannot read generalizes to new ones. That bet is decided by the held-out attack-family experiment under equal data. If the augmented baseline matches us there, we will report that the architectural route offers only the audit capability. We will then reframe the work as a diagnostic tool rather than a better detector.

Why not just use an autoencoder like Optimus and split its latent space, without invertibility? The explanation-space paper already reports that Optimus separates roles less cleanly than the supervised invertible network. Its decoder, however, can drop or invent content, so a role-preserving audit through it cannot tell leakage apart from decoder error. The Optimus ablation measures whether this costs robustness, or only auditability. If it costs neither, the invertibility claim is weakened and we will say so.

Why not just run a semantic role labeler and an entailment model per argument, as QASEMCONSISTENCY does? For localization alone, this pipeline is strong and may beat us; ρ = 0.73 is a serious bar. Its weakness for our question is that the per-argument entailment model is itself a standard encoder that can be swayed by style. We test this by running the pipeline on the attacked test sets. If it proves equally style-robust, the contribution narrows to cost: one forward pass versus a 3B parser plus a question-answering model.

Why not just fine-tune a token tagger on MisSpans? A RoBERTa span tagger trained on MisSpans labels is the obvious localization baseline and will likely outperform prompted language models. We run it with the same training data. Our claim is not that role blocks beat it on clean MisSpans, but that it degrades more when the surrounding sentences are restyled. We test this by applying held-out style rewrites to MisSpans fake stories while preserving the gold false spans.

Decisive Experiments

The central experiment is the held-out style-attack study on PolitiFact, LIAR, and MultiFC, using claims plus evidence. All systems share training rewrites from two attack families and are tested clean and under three unseen families. We report accuracy and macro F1 drop, five seeds.

The supporting checks come next. Probing must show that formality and emotion are predictable from $z_{\text{res}}$ and near chance from the role blocks. If not, the mechanism has not worked, and any robustness gain is unexplained. Human raters must judge that most residual-swap decodes preserve the claim. Autoencoder reconstruction must be reported on news text, and results stratified by it. If the gain vanishes on well-reconstructed sentences, the effect is an artifact.

Localization is tested on MisSpans span identification and typing F1, clean and restyled, against the token tagger and the prompted-model range. It is also tested on CLIFF, FActScore and Verifiability, as Spearman correlation against QASEMCONSISTENCY's 0.73. Finally, we report clean accuracy on PolitiFact and Twitter-16 against our rerun of NRFE-D, to show the style blindness was not bought with a large loss in ordinary performance.

The project succeeds if role-only reading roughly halves the accuracy drop on unseen style attacks relative to equal-data augmentation, while staying within a point of NRFE-D on clean data. It fails cleanly if the augmentation baseline or the Optimus ablation closes that gap.