Retrieval-augmented generation (RAG) answers a question by first fetching passages from a document collection and then asking a language model to write an answer from them. The fetching step usually ranks passages by how close their vector embeddings are to the question's embedding. That works when the right passage is about the right things. It fails in a specific and dangerous way when a passage mentions exactly the right things in the wrong relation. "Russia sold Alaska to the United States" and "The United States sold Alaska to Russia" contain the same entities and the same verb, and they will sit almost on top of each other in embedding space. The generator then has to sort them out, and it may write the wrong one with full confidence.
Linguistics has a well-studied tool for this problem. Semantic role mapping turns surface grammar (subject, object, "by"-phrase) into roles such as agent and patient. Under that mapping, "the bread was cut by the chef" and "the chef cut the bread" come out identical, while "the bread cut the chef" does not. This note proposes to attach these role structures to every indexed sentence and to use them twice. First, a role-consistency score reranks retrieved evidence. Second, a role-binding check verifies the generated answer against that evidence. Both pieces are evaluated on stress tests built in matched pairs. In each pair, one variant swaps roles, so the right answer is to reject it. The other only changes voice or phrasing, so the right answer is to accept it. The pairing lets us tell whether roles add anything that entity matching, cross-encoders, or learned evidence extractors do not already provide.
Core Claim
The claim is that explicit, voice-normalized predicate-argument roles help in two places. They detect and suppress evidence that is lexically and entity-wise on-topic but role-inverted. They also catch generated answers that swap roles relative to the evidence. Both can be done at lower query-time cost than an LLM reranker, with no regression on ordinary questions, because the score falls back to dense similarity whenever role parsing is unavailable.
What is new is the direction of information flow and where it is applied. LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models uses retrieval of PropBank frame descriptions, together with LoRA fine-tuning, to make LLMs into strong role labelers. Its information flows only one way, from retrieval to role labeling. We reuse that parser as an indexing tool, so role labeling serves retrieval instead. Using Semantic Roles to Improve Question Answering showed that matching FrameNet roles between question and answer sentences beat syntax-only extractive QA by 4–7 points on TREC. It was capped by FrameNet covering only about 35% of questions, and it predates dense retrieval and generative readers. We replace hand-built frames with a learned, broad-coverage PropBank parser. We also move role matching from answer extraction into reranking and answer verification.
Research Program
The first step is indexing. Each sentence in the collection is parsed into predicate-argument frames using the LLM role labeler from LLMs Can Also Do Well!. Each argument filler is linked to a canonical entity using the coreference-resolved entity table of SlimRAG. That paper's ablation shows coreference matters: accuracy on HotpotQA falls from 57.41% to 40.98% without it.
PropBank roles are defined per verb, so they cannot by themselves align converse predicates such as buy and sell. To bridge them, we attach Dowty-style proto-role properties to each argument: volition, causation, and change of state. The Semantic Proto-Role Linking Model found a single coherent proto-agent cluster, but found that proto-patient is a composite of distinct subtypes. That finding constrains the design. We match agent-side arguments strictly and patient-side arguments softly. A small LLM call maps the question's predicate onto the frames of the candidate sentence. This call happens at query time only, and only for the question.
The second step is reranking. The question is parsed into a frame with one open slot, the role being asked about. A candidate chunk $c$ for question $q$ receives
$$s(q,c) = \cos(e_q, e_c) + \lambda \, g(q,c)\, R(q,c),$$
where $e_q$ and $e_c$ are dense embeddings, and $\lambda$ is tuned on a development split. The gate $g(q,c) \in \{0,1\}$ is 1 only when both $q$ and some sentence in $c$ received a confident parse of a matching predicate. The role-consistency term $R(q,c)$ is the best score over sentence frames in $c$. Each bound question argument that appears in the same role in the frame adds $+1$. Each bound argument that appears in a conflicting role adds $-\mu$, with the penalty $\mu$ also tuned on development data.
The gate is how we address the coverage problem that limited the 2007 work. When the parse fails, the method reduces exactly to the dense baseline rather than breaking. Semantic role labeling tools for biomedical question answering found that question-to-snippet structure matching ranged from 2% to 72% depending on the tool. That finding motivates the gate, and it is also why gate-firing rate is reported as a first-class quantity.
The third step is answer verification. The generator's answer sentence is parsed. Its frame must be entailed by some retained evidence frame with the same role bindings. A role swap triggers one regeneration, with the conflicting frame shown to the generator. This targets the binding failure that Does CLIP Bind Concepts? documents in vision-language models. On that benchmark, accuracy drops from about 92% on single objects to about 31% on two-object binding and to chance on relations. Those numbers come from images, not text RAG, so we treat them only as motivation and measure the text-side rate directly. Current evidence extractors such as SEER and Ext2Gen select sentences by learned preference over relevance and overlap. Neither checks role binding, which is the specific gap the verifier fills.
Baselines
The naive reference point is vanilla dense RAG: the same embedding model, the same top-k, and the same generator, with no reranking. SlimRAG reports 50.78% accuracy for vanilla RAG on HotpotQA. We will reproduce it in our setup rather than copy it.
The strongest entity-level competitor is SlimRAG itself, which scores chunks by combining vector similarity with entity coverage and reports 57.41% HotpotQA accuracy. On role-swap pairs it should tie by construction, because both members of a pair cover the same entities. That is exactly the comparison that isolates roles. We run it with our corpus, embeddings, and generator. Beating it means a statistically significant win on the swap set, with no loss beyond noise on the original HotpotQA questions.
The strongest learned-extraction competitor is Ext2Gen. It reports an average score of 0.557, against 0.473 for the default Llama3.1-8b-instruct, under noisy retrieval on HotpotQA, MS-MARCO, and NQ Aligning Extraction and Generation. SEER is a second extractor, which shrinks input length 9.25x. We use the released or reproduced extractors on the same backbone and the same retrieved sets. Our verifier is added on top of them as well as compared against them, since the two methods are complementary.
The most dangerous competitor is a cross-encoder or LLM reranker. These models read the question and passage jointly, so they may resolve role inversion implicitly. No reported result was found on Lacuna for cross-encoder or LLM rerankers on HotpotQA or on role-inverted distractors. We will therefore run a cross-encoder reranker and a pointwise LLM reranker at the same top-k. The LLM reranker uses the same backbone as our parser, and we report query-time LLM tokens and latency for all methods.
We include two ablations. The first, an unlabeled ablation, keeps argument segmentation but drops role labels and matches argument sets as bags. It isolates whether the role label carries the gain. The second, a triple ablation, replaces role frames with subject-relation-object triples extracted by the same LLM, the representation PathRAG traverses. It isolates voice normalization and proto-role alignment.
Why not just…?
Why not just use entity linking and coreference, as SlimRAG and ChainRAG do? Entity methods establish which entities a passage mentions, not what each one did. ChainRAG repairs vague multi-hop sub-questions by rewriting them around resolved entities. It reports roughly 60% relative F1 gain over naive RAG on MuSiQue, which shows the value of entity grounding for the problem it targets. On a role-swapped distractor, however, entity coverage is identical, so these methods cannot separate the pair. The swap-set comparison against SlimRAG tests this directly. If SlimRAG already separates pairs through embedding side effects, the claim fails.
Why not just use a cross-encoder or LLM reranker? This is the real threat. A model that reads question and passage together may handle "bought by" versus "sold to" without explicit roles. The proposal does not assume otherwise. If cross-encoders match our swap-set accuracy, the remaining argument for roles is cost and auditability. Role frames are computed once at index time, and a rejected passage comes with a named conflicting role. We would then report that roles add little for reranking and focus the contribution on answer verification. Rerankers do not address generation-time swaps, but that too must be measured. The frozen-LLM role-labeling F1 of 10–21% reported in LLMs Can Also Do Well! suggests that implicit role handling in LLMs is weaker than assumed. That is a statement about explicit labeling, not about reranking, so it is suggestive rather than decisive.
Why not just use knowledge-graph RAG triples? Triples are close to roles, but graph pipelines carry heavy index overhead. SlimRAG reports a relative index token use above 56 for graph baselines, against 16.31 for itself, and 33.06% GraphRAG accuracy on HotpotQA. Triples also inherit whatever voice the extractor happened to produce. The triple ablation keeps our sentence-anchored, ungraphed setup and swaps only the representation. That separates the cost argument from the normalization argument.
Why not just train an evidence extractor on swap examples? This is plausible. We include an Ext2Gen variant whose preference data includes our synthetic swaps. If it generalizes to held-out natural swaps, it is a cheaper path. CLIP-FT's collapse to near 0% on unseen compositions in Does CLIP Bind Concepts? is one reason to doubt it, but the test is whether the extractor generalizes across predicates it was not trained on. We hold out entire predicates to measure exactly that.
Proposed Experiments
The first experiment is a prevalence measurement on unmodified data. In HotpotQA full-wiki and MuSiQue/2WikiMultiHopQA retrievals, we count how often the top-k contains a passage that shares the gold passage's predicate and entities but binds them differently. We also count how often the generator's error is a role swap. This number decides how much the method can move standard metrics. A low prevalence would mean the contribution is robustness, not average accuracy, and we will say so.
The second experiment uses the HotpotQA distractor setting HotpotQA, whose reported baseline answer F1 is 58.28, against 91.40 for humans. For each question we add a role-swapped copy of a gold supporting sentence and, separately, a voice-alternated paraphrase of it. Swapped copies are written by an LLM and filtered by annotators for fluency, so that only the role differs. Metrics are pairwise retrieval accuracy (gold ranked above swap), paraphrase retention, answer EM/F1, supporting-fact F1, and the rate of role-swap hallucinations in generated answers.
The paired design is the decisive check. A method that simply penalizes surface change will reject paraphrases, and that cost will show up as lost paraphrase retention. A method that only tracks entities will tie on swaps. Only role awareness should win both. The same construction is applied to TREC QA converse-predicate questions, the setting of the 2007 work. It is also applied to BioASQ, where natural directional relations such as one protein inhibiting another make swaps clinically meaningful. Finally, on unmodified HotpotQA and MuSiQue, we require no significant regression against vanilla RAG and SlimRAG, while reporting gate-firing rate, index cost, and query-time tokens for every system.
The claim survives if three things hold. Role reranking must beat SlimRAG and match or beat the cross-encoder on swap pairs at lower query cost, while keeping paraphrases. The unlabeled ablation must lose most of the gain. The verifier must cut role-swap hallucinations on top of Ext2Gen.
References
- LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models
- Using Semantic Roles to Improve Question Answering
- SlimRAG: Retrieval without Graphs via Entity-Aware Context Selection
- The Semantic Proto-Role Linking Model
- Semantic role labeling tools for biomedical question answering: a study of selected tools on the BioASQ datasets
- Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
- SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Paths
- Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering