Retrieval-augmented generation (RAG) means letting a language model look up passages from a document collection and read them before it answers. It is usually sold as a cure for misinformation. One well-documented case shows it doing the opposite. Health misinformation often cites a real study and then stretches it: a drug that killed cancer cells in a dish becomes "a cure for cancer." When a RAG system retrieves the real finding, the "kernel of truth," the model tends to accept the stretched conclusion along with it. Grounding Fallacies Misrepresenting Scientific Publications in Evidence reports that Llama 3 labeled such fallacious claims TRUE 0.0% of the time from memory alone, and 44.4% of the time once shown the retrieved kernel passage. The same paper finds that retrievers are good at finding the kernel and poor at finding the passages that undermine the claim, such as "this was an in-vitro study" or "the cohort was mice." Those passages share few words with the claim.
We propose to fix this by changing what the model reads when it decides, not by adding more text. The system asks two separate questions in two separate contexts. First, is the cited fact real? Second, does the conclusion follow, given the study's actual scope? The second question is answered by a judge that never sees the persuasive kernel passage. It sees only scope facts retrieved by queries built from the specific reasoning gap each fallacy type exploits. Examples are "what population was studied?", "was the design able to support a causal claim?", and "what dose or setting was tested?" The prediction is that most of the 44.4% collapse comes from two causes. The kernel passage sits next to the verdict, and the undermining evidence never gets retrieved. Each cause can be removed and measured separately.
Core Claim
The failure described by Glockner et al. is a context-composition problem. It is not mainly a knowledge problem, since the model rejects the claims closed-book, and not mainly a detection problem. The work of Direction 2 on context windowing, that more context can hurt and that what fills the window matters more than how much there is, has so far been applied to turn counts and token budgets. Here we apply it to the argumentative role of each passage. Two components are proposed. The first is gap-directed retrieval: search queries generated from fallacy-specific checklists, so that they target a paper's methods and limitations rather than the claim's wording. The second is verdict isolation: the inference judgment is made in a context that contains the claim and the retrieved scope facts, but not the kernel passage. The claim is testable in a way that could refute it. If simply adding the gold undermining passages next to the kernel removes most of the overruling, isolation is unnecessary and the proposal reduces to a retrieval contribution. We treat that outcome as a real possibility and design for it.
Relation to Prior Work
Glockner et al. supply the problem, the benchmark (MISCIPLUS: 114 fallacious arguments grounded in 100 biomedical publications), the three subtasks (find the kernel, find undermining passages, reconstruct the fallacy), and the motivating failure. They benchmark BM25, BioBERT and INSTRUCTOR retrieval with a standard single-pass RAG reader. They propose no mechanism for retrieving undermining context and do not vary what the reader sees. Both of those are what we add.
Our methods come from three places. Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs breaks each of 232 fallacy classes into yes/no checking steps and raises Claude-Sonnet-4 from 42.2% to 62.9% accuracy on short, closed-book snippets. We reuse the idea of procedural checks, but as retrieval queries against the source paper rather than as questions the model answers from memory. The same paper constrains our design: sorting a fallacy into a hierarchy first caused errors that later steps could not undo. We therefore do not classify the fallacy before retrieving. We issue the checks for all candidate fallacy types in the MISCIPLUS inventory and let the evidence decide. When Splitting Makes Stronger argues and shows that splitting a task into sub-tasks solved in separate, smaller contexts stops early errors from contaminating later ones. It reports, for example, that GPT-3.5 with divide-and-conquer beat GPT-4 with direct prompting on SciFact. We borrow the isolation principle but make a different claim. What hurts in our setting is persuasive, correct evidence, not the model's own intermediate mistakes, and the split is dictated by the premise-versus-inference structure of the argument rather than by generic task decomposition.
The context-management literature both motivates and constrains the design. Towards Adaptive Context Management for Intelligent Conversational Question Answering compresses distant history down to named entities, and CORAL finds that LLM summarization of history gives the best citation precision (31.1%) in multi-turn RAG. Both treat relevance as topical. Our case is the reverse. The most topically relevant passage, the kernel, is the harmful one. The qualifiers that entity extraction or abstractive summarization would drop ("in mice", "at 100× the clinical dose") are exactly what the judge needs. So the judge receives short, verbatim, qualifier-preserving scope snippets, never a summary.
Finally, Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation raises GPT-4 on COVID-19 fallacy data from 0.25 to 0.76 F1 by having the model generate counterarguments and explanations. It relies on the model's own knowledge. It is our strongest prompting-only competitor, and we test whether it survives when a kernel passage is present.
Research Program
The pipeline has three stages, all run with the same backbone model. In the premise stage, the claim's cited fact is checked against the top-ranked kernel passages, using the Subtask 1 retrieval setup from Glockner et al. The output is only a label (supported, unsupported, or misquoted). The passage text is not carried forward. In the gap-directed retrieval stage, a fixed library of scope questions is compiled once, by hand, from the AID-LF-style decision steps for each fallacy type in the MISCIPLUS inventory: composition, causal oversimplification, impossible expectations, and so on. At test time the model fills these questions in with the claim's entities (for example, "In what organism or cell line was [drug] tested?"). Each filled question retrieves passages from the source paper with the same retrievers Glockner et al. used, so that retrieval gains are attributable to the queries and not to a new retriever. In the verdict stage, a fresh context receives the claim, the premise label, and the top scope snippets, verbatim and qualifier-intact. It returns a judgment on whether the conclusion follows, together with a short reconstruction of the gap (Subtask 3).
Three analyses accompany the build. The first measures how the TRUE rate on fallacious claims changes as kernel passages are added to or removed from the judge's context, holding the undermining evidence fixed. The second identifies which fallacy types gain most from gap-directed queries. Composition and population gaps should gain, and gaps about missing effect sizes perhaps less. The third checks whether the judge over-rejects when the scope facts are benign.
This last check requires a control set, and building it is part of the work. A pipeline that says "false" to everything would score perfectly on MISCIPLUS, which contains only fallacious arguments. The 0.0% closed-book TRUE rate is itself uninterpretable without controls. For each of the 100 publications we will write, with two annotators, one or two faithful claims that make the paper's finding no stronger than its evidence supports. Agreement will be reported as Cohen's κ, following the annotation practice of Illuminating Logical Fallacies with the CAMPFIRE Corpus. Every headline number is then reported as balanced accuracy over fallacious and faithful claims, alongside the overruling rate. CAMPFIRE's finding is relevant here too: GPT-4o's overall F1 of 0.89–0.90 hides a fallacy-class F1 of 0.05, which is exactly the illusion that imbalanced evaluation produces.
Baselines
The naive reference points are two. The first is closed-book judgment by the same model, the setting where Glockner et al. report a 0.0% TRUE rate for Llama 3. We rerun it on the balanced set to see whether that number reflects skepticism or knowledge. The second is standard RAG: the claim plus the top-k claim-retrieved passages in one context, which is Glockner et al.'s setup and reported a 44.4% TRUE rate on fallacious claims with Llama 3. Beating standard RAG means lowering that rate substantially while keeping accuracy on faithful claims within a few points of standard RAG. Standard RAG should be good on faithful claims, because the kernel supports them.
The strongest competitor is long-context whole-paper reading. The full source publication is placed in the context and the model judges the claim. This is a real threat, because MISCIPLUS arguments come from single papers that fit in modern context windows, so the undermining passages are present by construction. It removes the retrieval problem entirely. The only question left is whether the model uses the limitations it can see or is still pulled toward the kernel. The evidence on redundant context in Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?, and on attention spreading over long inputs in Focus and Dilution, suggests it may be pulled, but that is a hypothesis. We find no reported result on Lacuna for whole-paper long-context judgment on MISCIPLUS.
The strongest prompting competitor is the counterargument-and-explanation method of Jeong et al., applied on top of standard RAG. It is reported at 0.76 F1 on COVID-19 fallacy classification without retrieval, and we find no reported result on Lacuna for it under kernel-present RAG. We also run generic divide-and-conquer prompting in the style of Zhang et al.: the model decomposes the claim into sub-claims on its own and checks each in a separate context, using claim-driven retrieval. This isolates whether our gains come from generic splitting or from splitting along the premise/inference line with gap-directed queries.
Two ablations isolate the key ingredients. Gap retrieval without isolation puts the gap-retrieved snippets into the same context as the kernel. Isolation with claim-driven retrieval keeps the kernel out of the judge's context but fills it with ordinary claim-retrieved passages. An oracle condition replaces retrieved scope snippets with the gold undermining passages from MISCIPLUS, both with and without the kernel present. This gives the ceiling for retrieval and the cleanest test of isolation.
Fairness is enforced by matching the total number of model calls. Our pipeline uses roughly three calls plus one retrieval per scope query, so single-context baselines receive self-consistency voting over the same number of samples. All methods share the same retriever indices, the same top-k budget of retrieved tokens (except long-context, which is given the whole paper), and the same prompt-tuning budget: one small development split of 20 publications, held out from all reported numbers.
Why not just…?
Why not just give the model the whole paper? This is the alternative most likely to work. If whole-paper long-context reading matches our pipeline on balanced accuracy, the retrieval component is unnecessary for single-paper claims. The contribution would then shrink to the isolation finding, or vanish if long-context also shows low overruling. The experiment below settles it directly. Our expectation that it falls short rests on indirect evidence: redundant context derails models even on problems they can solve (Yang et al.), and topically relevant text dominates attention. Neither result was measured on this task. Even in the losing case, whole-paper reading does not extend to the realistic setting where the source paper must first be found among many, so the retrieval question survives in that form.
Why not just prompt the model to be skeptical or to generate counterarguments? The method of Jeong et al. shows large gains when the model argues against the claim, but its counterarguments come from the model's own knowledge. The kernel-of-truth failure is precisely the case where retrieved evidence overrides that knowledge. If counterargument prompting over standard RAG closes most of the gap on fallacious claims without hurting faithful ones, our pipeline is not worth its extra calls. We report that comparison as a primary result, not a footnote.
Why not just retrieve the undermining passages better and keep one context? This is exactly the oracle ablation. If gold undermining passages placed beside the kernel bring the overruling rate near the isolated condition, then retrieval was the whole problem and isolation adds nothing. In that case we would reframe the work around gap-directed retrieval for Subtask 2. The file gives no reported retrieval number on Lacuna for Subtask 2, only that retrievers "perform poorly" there, so our Subtask 2 results would be the first numbers for this approach.
Why not just drop retrieval, since closed-book already rejects every fallacious claim? A 0.0% TRUE rate on an all-fallacious set is also what a model that distrusts every health claim would produce. The faithful control set tests this. We expect closed-book accuracy on faithful claims to be poor, which is why retrieval is needed at all.
Proposed Experiments
The main experiment runs every baseline and our pipeline on MISCIPLUS plus the faithful controls, with Llama 3 (for continuity with Glockner et al.) and one other open model. We report the overruling rate on fallacious claims, accuracy on faithful claims, and balanced accuracy, with paired bootstrap confidence intervals. With about 114 fallacious and 100–200 faithful claims, only differences of roughly ten points or more will be detectable, and we say so in advance. Retrieval is scored separately. Subtask 1 MRR is reported against the 0.664 ceiling from Glockner et al. to confirm we have not broken kernel finding. Subtask 2 MRR and recall@5 are reported for claim-driven versus gap-directed queries. Subtask 3 reconstructions are rated by two annotators for whether they name the correct gap.
The decisive check is the 2×2 oracle design: kernel present or absent in the judge's context, crossed with gold undermining passages present or absent. It answers whether presence of the kernel harms judgment independently of missing evidence. If the kernel-present, undermining-present cell still shows a much higher TRUE rate than the kernel-absent, undermining-present cell, verdict isolation is justified. If the two cells match, it is not.
A second check compares whole-paper long context against our pipeline. A third is a harm check on SciFact, where claims are ordinary supported or refuted statements. It confirms that removing the kernel from the judge's context does not hurt verification of claims that are simply true. The divide-and-conquer paper reports that its approach helps on SciFact, but we find no raw number reported on Lacuna for that result, so we rerun the setup ourselves.
A result that would count as success is an overruling rate well below standard RAG's 44.4% (Llama 3), faithful-claim accuracy no more than a few points below standard RAG, and a win over both whole-paper reading and counterargument prompting on balanced accuracy under matched compute. Failure on any one of these tells us which of the two components, if either, deserves the credit.
References
- Grounding Fallacies Misrepresenting Scientific Publications in Evidence
- Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
- When Splitting Makes Stronger: A Theoretical and Empirical Analysis of Divide-and-Conquer Prompting in LLMs
- Towards Adaptive Context Management for Intelligent Conversational Question Answering
- CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation
- Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation
- Illuminating Logical Fallacies with the CAMPFIRE Corpus
- Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
- Focus and Dilution: The Multi-stage Learning Process of Attention