When a language model answers a question from retrieved documents, it can fail in at least three different ways. A passage may be off-topic. A passage may be on-topic but false or misleading. Or the retrieved set may answer one part of a multi-part question and say nothing about another. Current defenses against "noisy context" collapse all three into one relevance score and filter on it: rerankers, compressors, pruning of retrieved paths, and extract-then-answer training. A misleading passage, however, is by construction highly relevant, and a passage that covers half the question also scores well on relevance. One-dimensional filtering therefore cannot catch the failures that produce the most confident fabrications. This proposal borrows the central principle of faceted classification from library science, which describes items along independent axes rather than placing them in a single hierarchy. Before the model reasons, each retrieved sentence is labeled on separate, independently computed binary axes: which part of the question it covers, whether it is corroborated or contested by the other evidence and by the model's own prior answer, and whether it asserts or hedges. The reader then answers or abstains separately for each part of the question.
The bet is mechanistic. The "rationalization trap" happens when a reasoning model builds a logical bridge to justify a misleading premise. If the facet labels are fixed by a separate pass that never sees the candidate answer, the reader has nothing to rationalize toward, because conflict is flagged before the model commits. The proposal stands or falls on one ablation: orthogonal facets must beat the same signals collapsed into a single score, on exactly the noise types where the axes disagree.
Core Claim
Retrieved-context noise has at least two dimensions that relevance filters conflate: topical relevance and evidential support. Missing evidence is a third failure, and it is defined per requirement of the question rather than per document. We claim that representing each retrieved sentence on separate binary facets brings three benefits. The facets are requirement coverage, corroboration status, and assertion strength. They must be computed independently of answer generation. The three benefits are as follows. First, the approach should keep the gains of relevance filtering on off-topic noise. Second, it should recover most of the accuracy lost to misleading snippets, including for step-by-step "thinking" models. Third, it should turn the all-or-nothing abstention decision into per-requirement abstention, so that partially supported questions get partial answers instead of fabricated whole ones. Each failure type should be fixed by its own facet and not by the others. That noise-type-by-facet interaction is the novel, falsifiable prediction.
Why This Is Not Already Done
The closest work applies facets on the wrong side of the pipeline or along the wrong axis. RichRAG splits the user's query into sub-aspects and retrieves for each one to improve coverage of long-form answers. It never labels the retrieved evidence and never measures noise or hallucination. We reuse its decomposition step but apply facets to the evidence and use them for abstention, not ranking. Ext2Gen trains the generator with preference optimization to extract relevant sentences before answering, using up to 25 distractors. Its sentences are flat and its distractors are off-topic, so it addresses the relevance axis only. CompAct compresses documents into a running summary with an early-stop token, and Superposition Prompting processes documents in parallel paths and prunes low-saliency ones. Both keep or drop text by one salience notion, and neither can tell a relevant falsehood from a relevant truth. Earlier faceted classification of scientific text predates generative models. Jochim and Schütze's citation classifier decomposes citation function into four orthogonal binary axes so that the classes do not collapse into a "neutral" catch-all. Kobayashi et al.'s discourse-facet search indexes Objective/Method/Result separately. Both supply the design principle we reuse, but neither feeds facets to a generator.
Three papers supply the evidence that motivates this work. RGB shows ChatGPT rejects unanswerable contexts only about 45% of the time and corrects injected counterfactual errors only 57.14% of the time. Thinking in a Crowd shows Qwen3-8B dropping from 96.22% to 77.03% with two misleading snippets when thinking is on. That is worse than the clean baseline, and it barely beats thinking-off at 74.87%. CRAG reports straightforward retrieval-augmented generation (RAG) at about 44% accuracy and a truthfulness score of around 13.4% for GPT-4 Turbo on its Task 3. Two papers constrain the design. Jochim and Schütze find that the evaluative facet (confirm vs. negate) is by far the hardest to label automatically, well below human agreement. Our corroboration facet is that kind of facet, so its labeler quality is the main risk. The dataset-measurement position paper warns against treating downstream performance as proof that a categorization is valid, which obliges us to validate facet labels directly against human annotation.
Research Program
The pipeline has three stages, and the separation between them is the point. In stage one, a requirement decomposer lists the atomic requirements a complete answer must satisfy. For "which director of film X was born in country Y," the requirements are the director of X and that director's birthplace. This uses the RichRAG-style decomposition prompt, but the outputs are constraints, not topics. In stage two, a facet labeler processes each retrieved sentence on its own and assigns three binary labels.
The coverage facet records whether the sentence entails a value for requirement $r_j$. It is computed per requirement with an entailment model, so a sentence can cover several requirements or none.
The corroboration facet records whether the sentence's claimed value for $r_j$ agrees with the other covering sentences and with the reader's closed-book answer, which is generated beforehand and treated as one more hidden "witness." It is computed by pairwise entailment and contradiction among the covering sentences. This follows the internal-consistency intuition of Agrawal et al..
The assertion facet records whether the sentence states its claim flatly or hedges it. It targets the overconfidence pattern described by Tang et al.. We treat it as the least justified facet and measure its marginal value separately.
Crucially, the labeler never sees a candidate answer from the reader's open-book pass.
In stage three, the reader receives the context grouped by requirement, with each sentence tagged by its facet labels. It is instructed to answer requirement $r_j$ only from covered, uncontested sentences, to report a conflict when the coverage set for $r_j$ is contested, and to abstain on $r_j$ when nothing covers it. The final answer composes the requirement answers. When a requirement abstains, the system states which part is unsupported rather than guessing.
We build two versions. The zero-training version uses a prompted labeler and a frozen reader, so it plugs into any model, as CompAct does. The trained version distills the labeler into a small classifier on silver labels. The silver labels come from benchmarks whose construction tells us which passages are gold, distractor, counterfactual, or misleading, which gives free supervision on RGB, SciAux, and HotpotQA-derived sets. Following the measurement-theory warning, 500 sentences per benchmark are double-annotated by humans on all three facets. We report agreement and labeler F1 per facet before any downstream number is interpreted.
Baselines
The naive reference point is plain concatenated top-$k$ retrieval with the reader's default prompt. On RGB this corresponds to the setting behind ChatGPT's roughly 45% rejection rate, and on CRAG to the "straightforward RAG" setting reported at about 44% accuracy. We run it with identical retrieval outputs for every method.
The prompting baseline is distraction-aware answer selection from Improving Zero-shot Reader by Reducing Distractions. It adds an explicit "return unanswerable" instruction and calibrates answers by the query likelihood given each document. That paper reports a 64% average exact-match increase across its open-domain benchmarks. It is the strongest zero-training competitor because it targets exactly the abstention failure we target. We give its prompt the same per-requirement wording our reader receives, so that only the facet labels differ.
The strongest existing method is Ext2Gen. Its R2 variant reports an average QA score of 0.557 against 0.473 for the default Llama3.1-8B-Instruct on its own noisy-chunk evaluation, and it reports beating RECOMP, CompAct, and EXIT. No reported result was found on Lacuna for Ext2Gen on RGB negative rejection, on RGB counterfactual robustness, or on SciAux. We will therefore run it ourselves on Llama3.1-8B-Instruct with its released training recipe, and give our trained labeler the same number of GPU-hours and the same source data. Beating it means two things. We must match its score on its own noisy-chunk test set within one point, because our method should not lose on off-topic noise. And we must exceed it by a clear margin on RGB counterfactual and SciAux misleading conditions. CompAct and Superposition Prompting are run as additional relevance-axis baselines on MuSiQue and HotpotQA, where both report results. On HotpotQA, CompAct reports a 7.0 F1 gain over RECOMP.
The key ablation is the collapsed-score variant. It uses the same labeler outputs, but a learned linear combination merges coverage, corroboration, and assertion into one score per sentence. Sentences are kept or dropped on that score, and the reader receives untagged text. This isolates the claim that orthogonality, not extra signal, does the work. A second ablation runs the same three-facet procedure inline in the reader's own chain of thought ("label each sentence, then answer"). It isolates the claim that labels must be fixed before and apart from answer generation.
Why not just…?
Why not just tell the model to abstain and to check for contradictions? This is the cheapest alternative, and DAS shows prompting does real work on off-topic noise. But RGB shows that instruction-level abstention leaves ChatGPT rejecting only about 45% of unanswerable cases and correcting only 57.14% of counterfactual errors. Prompting also gives no mechanism for partial abstention. Our prompting baseline uses the identical per-requirement instruction minus the labels, so any gap is attributable to the labels themselves.
Why not just train extraction harder, for example Ext2Gen with misleading distractors added? This is a real threat, because a strong extractor trained on counterfactual negatives might learn the corroboration distinction implicitly. We will run exactly this variant, Ext2Gen retrained with RGB-style counterfactual chunks in its preference pairs, at the same compute. If it matches facet labeling on misleading noise, the contribution narrows to per-requirement abstention, and we would report that honestly. We expect it still to fail on partial-negative questions. Extraction picks sentences to keep, which tells the reader nothing about which requirement went unsupported.
Why not let a reasoning model sort the evidence itself? SciAux is direct evidence against unconstrained reasoning, since thinking-on underperforms the clean baseline by about 19 points under two misleading snippets. A structured self-labeling prompt is a stronger version of this alternative, and that is precisely our inline ablation. If inline labeling matches decoupled labeling, the "label before you reason" mechanism is wrong and the method reduces to a structured prompt.
Why not one combined relevance-plus-contradiction score? This is the collapsed-score ablation. On purely off-topic noise it should tie our method. It should lose on misleading noise and partial-negative questions, because a single threshold cannot simultaneously keep a relevant contested sentence for conflict reporting and keep it out of the answer.
Proposed Experiments
Readers are Qwen3-8B with thinking on and off, and Llama3.1-8B-Instruct. All methods share the same retriever outputs. We evaluate four noise regimes.
The first is off-topic noise: RGB noise-robustness at increasing noise ratios, plus Ext2Gen's distractor setting.
The second is misleading noise: RGB counterfactual robustness, and SciAux with zero, one, and two misleading snippets, using the SciAux dataset. The reference numbers to beat are 77.03% (thinking on) and 74.87% (thinking off) under two misleading snippets.
The third is full negatives: RGB negative rejection.
The fourth is partial negatives, which we construct ourselves. We take HotpotQA, MuSiQue, and RGB information-integration questions, delete the gold support for exactly one hop, and score whether the system answers the supported hops while abstaining on the deleted one. RGB reports integration accuracy peaking at 60–67% and falling to 34% under noise, which gives the non-deleted reference. End-to-end real-world performance is measured on CRAG with its hallucination-penalizing truthfulness score, against the reported 13.4% for straightforward RAG and up to 51% for the best industry system.
Metrics are accuracy or exact match, rejection rate, counterfactual error-correction rate, per-requirement abstention precision and recall on the partial-negative set, CRAG truthfulness, and reader output length. Output length is our proxy for rationalization, since SciAux observed longer outputs under misleading context.
The decisive check is the interaction pattern across the four regimes and three conditions: full facets, collapsed score, and inline self-labeling. The hypothesis survives only if three things hold. The collapsed score ties full facets on off-topic noise yet trails them on misleading and partial-negative regimes. Removing the corroboration facet specifically erases the misleading-noise gain, and removing coverage specifically erases the partial-negative gain. Decoupled labeling beats inline labeling for the thinking-on reader on SciAux. If the collapsed score ties everywhere, orthogonality buys nothing and the idea should be abandoned. If corroboration-labeler F1 against human labels falls near the roughly 58–68 macro-F1 range Jochim and Schütze report for automatic facet classifiers, while downstream gains still appear, we will attribute them cautiously and report per-facet error analysis. A secondary, optional analysis asks whether calibrating abstention thresholds per requirement, rather than per answer, shrinks the conservative prediction sets reported by TRAC on Natural Questions: 20.4 unique answers at 0.90 coverage.
References
- RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- CompAct: Compressing Retrieved Documents Actively for Question Answering
- Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
- Towards a Generic and Flexible Citation Classifier Based on a Faceted Classification Scheme
- Scientific Article Search System Based on Discourse Facet Representation
- Benchmarking Large Language Models in Retrieval-Augmented Generation
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- CRAG - Comprehensive RAG Benchmark
- Position: Measure Dataset Diversity, Don't Just Claim It
- Do Language Models Know When They're Hallucinating References?
- Evaluating large language models on medical evidence summarization
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- TRAC: Trustworthy Retrieval Augmented Chatbot