Causally Filtered Slot Plans for Faithful Narrative Explanations

When a language model explains a decision in fluent prose, the prose can be wrong in two separate ways. It can state facts that are false. It can also state facts that are true but that played no part in the decision, so a plausible story stands in for the real reason. Recent work on grounding explanations in knowledge graphs (structured databases of facts stored as subject–relation–object triples) mostly targets the first failure: claims are checked against the graph after generation, or generation is restricted to edges that exist. The second failure is harder to see and arguably more harmful. A true but irrelevant justification makes a wrong answer look reasonable, and that defeats the main thing a human-facing explanation is for: helping the reader decide when to trust the model.

This note proposes an explainer that is only allowed to say what the model used. First it builds a slot plan, an ordered skeleton of the explanation in which each content slot is bound to one piece of evidence that we have shown, by intervention, actually changes the model's prediction. A language model then writes freely around these slots. A checker splits the finished text into atomic claims and rejects any factual claim that cannot be traced to a slot. When the graph lacks a needed fact, a slot can instead be bound to a retrieved text passage, but only if the same intervention test passes. The template supplies the guarantee, the knowledge graph and retrieval supply the evidence, and the explanation utility gap is the outcome we measure: we test whether explanations restricted in this way help people catch the model's errors better than unconstrained narratives.

Core Claim

Our central claim is that the main reason grounded natural-language explanations fail users is the inclusion of true but causally inactive content, and that factual hallucination matters less. Requiring every factual claim to pass an intervention test, while leaving the connecting language unconstrained, should keep most of the fluency gains of narrative explanation and recover the faithfulness of symbolic paths. Most importantly, it should raise how often human readers correctly reject wrong model answers.

To make "causally active" precise: let $G$ be the graph context the predictor sees for an input $x$, let $E_c \subseteq G$ be the edges supporting a candidate claim $c$, and let $p(\hat{y}\mid x, G)$ be the predictor's probability for its chosen answer $\hat{y}$. We call $c$ active when

$$\Delta(c) = p(\hat{y}\mid x, G) - p(\hat{y}\mid x, G \setminus E_c) > \delta,$$

where $\delta$ is a threshold set on held-out data. Only active claims may fill content slots. A verifier that checks truth alone cannot distinguish an active claim from a "bystander" claim, and that inability is the gap this note targets.

Relation to Prior Work

Several papers motivate the proposal by documenting the gap. Towards Faithful Knowledge Graph Explanation Through Deep Alignment in Commonsense Question Answering (LKDA) shows that in joint language-model/graph systems the graph often acts as a passive bystander. The highlighted paths had little effect on the answer: vanilla GreaseLM's LM-KG fidelity was 40.7% on CommonsenseQA, and LKDA's alignment training raised it to 98.0%. LKDA, however, stops at symbolic paths. Path-based Explanation for Knowledge Graph Completion (Power-Link) similarly extracts faithful connected paths, which 64.7% of participants preferred, but it never verbalizes them. On the human side, On Behalf of the Stakeholders and Hamilton et al. argue that formally faithful outputs are often unusable for people outside the development team. We take faithful paths as given and study what happens when they are turned into prose. Neither line of work addresses that step.

Other work supplies methods that we reuse. LKDA's alignment training produces a predictor whose graph paths matter, which is a precondition for our intervention test to be meaningful. MedRAGChecker decomposes long-form outputs into atomic claims and verifies them with text entailment plus knowledge-graph scoring; we reuse its decomposition as our enforcement checker. It checks truth, though, not causal activity, and it never generates or repairs text. Re-KGR repairs hallucinated entities after generation using graph lookup. That is one candidate enforcement mechanism, but its free rewriting pass can introduce new errors. Look-ahead graph walks restrict decoding to existing edges, which is the other candidate enforcement mechanism. Social-science framings from iLLuMinaTE and the narrative roles of Biran et al. give us the slot types, such as "contrastive evidence" and "missing evidence". iLLuMinaTE writes from these framings without any grounding.

The template literature constrains the design. The frozen-slot architecture in Bridging the Compression-Precision Paradox guarantees measurement accuracy by copying values into fixed slots, at a cost to naturalness. The Bi-LSTM fashion explainer shows how repetitive fully templated explanations become. MUTANT and FVQA 2.0 show that rigid templates hide failures. We therefore fix only which facts appear, never the surface wording.

The closest generation system is KnowRec, which grounds recommendation explanations in DBpedia subgraphs. It raised entity coverage to about 68% from about 10% for review-based baselines. Coverage, however, rewards mentioning graph facts whether or not the rating model used them, and that is exactly the incentive we remove.

Research Program

The system has four stages. The predictor is GreaseLM trained with LKDA alignment on CommonsenseQA and OpenBookQA. We chose it because its graph paths are known to influence its answers, so deleting edges is a meaningful intervention. For each test item, we enumerate candidate claims from the top-scoring ConceptNet paths and compute $\Delta(c)$ by edge deletion. Deletion requires one extra forward pass per claim, and we cap the search at the top paths.

Second, a planner assigns active claims to typed slots: main supporting evidence, contrastive evidence (a fact that lowered the probability of the runner-up answer, measured by the same intervention applied to that answer), and evidence the model lacked. When the graph has no edge for a relation that a retriever surfaces from text, we turn the retrieved sentence into a candidate triple. We add it to $G$ and keep it only if adding it changes the prediction. This is how retrieval enters: it patches graph incompleteness under the same causal test rather than bypassing the test. We study incompleteness by deleting held-out ConceptNet edges, following the protocol of Evaluating Knowledge Graph Based RAG under Knowledge Incompleteness.

Third, a language model writes the explanation from the slot plan with no wording constraints. Fourth, an enforcement pass applies MedRAGChecker-style claim decomposition and deletes or regenerates any sentence containing a factual claim that matches no slot. We compare this verify-and-repair enforcement with constrained decoding over slot entities as an ablation, because the proposal's claim concerns which facts are admitted, not how admission is enforced.

As a second domain we use Movie KG-Exp, which tests whether the method transfers to recommendation explanations. There the intervention deletes attribute edges from KnowRec's item subgraph and measures the change in predicted rating.

Baselines

The naive reference point is the predictor's answer shown with no explanation. Every human-utility gain is measured against it, since an explanation that does not beat silence at helping users catch errors has no operational value.

The strongest symbolic competitor is the LKDA path shown raw, as in "bird → lives in → nest", with the same top paths our planner sees. It is maximally faithful by construction, so it isolates whether verbalization adds anything. We also include a rigid-template verbalization of the same active claims in the style of the Bi-LSTM fashion explainer, which isolates the value of free wording.

The strongest narrative competitor is an iLLuMinaTE-style prompt. It gives the same writer model the same graph neighborhood and the same social-science framing instructions but no causal filtering and no enforcement. A second version adds MedRAGChecker-style truth verification with repair, which yields explanations that are true but not filtered for activity. This truth-verified narrative is the baseline the proposal most needs to beat, because it already removes hallucination, so any remaining gap must come from causal activity.

The key ablation keeps everything else fixed and replaces "active claims" with "true claims ranked by path score". It is the same pipeline with $\delta = 0$. All narrative systems use the same writer model, decoding settings, prompt-tuning budget (the same number of development-set iterations), and graph context.

For reference numbers, the file reports LKDA's fidelity of 98.0% on CommonsenseQA and accuracy of 82.4% on OpenBookQA, both for the predictor; KnowRec's BLEU-4 of 10.71 and entity coverage of about 68% on Movie KG-Exp; and iLLuMinaTE's 89.52% human preference with 94.68% GPT-4o fidelity agreement on its own MOOC data. I found no reported result on Lacuna for human error detection or claim-level causal activity of natural-language explanations on CommonsenseQA, OpenBookQA, or Movie KG-Exp, so we must measure all baselines ourselves. We count the proposal as beating the truth-verified narrative if it shows a higher error-detection rate on items the model gets wrong, confidence intervals not overlapping, with no more than a small loss on comprehension ratings.

Why not just…?

Why not just prompt a strong LLM with the graph paths in context? This is the most likely real threat. Writer models may naturally stay close to the evidence they are given. The evidence against it in the file is indirect. LKDA's 40.7% fidelity shows that available paths are often not used ones, and a writer model has no way to tell the two apart. The in-context baseline above tests this directly. If its rate of inactive claims is already low and its error-detection matches ours, the filtering step is unnecessary and we will report that.

Why not just verify claims against the graph and repair, as MedRAGChecker and Re-KGR do? These tools catch false claims, and the file shows the gain from graph signals: 69.8% versus 63.4% expert alignment. A true bystander fact still passes them. The truth-verified narrative baseline is built to settle this question. If filtering by $\Delta(c)$ adds nothing over truth verification, the central claim is wrong.

Why not just use rigid templates over the active claims? Templates already guarantee containment, and the EEG work shows they can be precise. They are included as a baseline. The real question is whether free wording improves comprehension and error detection or only fluency ratings. If templates match our system on error detection, the conclusion is that the causal filter, not narrative, carries the value. That would still be a useful result, and we would recommend templates.

Why not just show raw LKDA paths? They are faithful, and Power-Link found that people prefer connected paths. Lay readers may still struggle with relation labels, as the stakeholder literature argues. The raw-path condition measures this, and if raw paths match narratives on error detection, verbalization is unjustified for this task.

Proposed Experiments

The automatic evaluation reports, for each system, three quantities: the fraction of decomposed claims that are supported by the graph or a retrieved passage, the fraction of claims that are causally active under $\Delta(c) > \delta$, and the fraction of active claims from the slot plan that the text covers. These are reported both on the full graph and under 10–50% edge deletion, which tests whether retrieval-backed slots preserve coverage when the graph is incomplete. PASTA's learned scorer is vision-only and trained on crowd ratings (Benchmarking XAI Explanations with Human-Aligned Evaluations), so we cannot use it here and rely on direct human measurement.

The decisive check is a human study on CommonsenseQA and OpenBookQA items, sampled so that half are items the model answered incorrectly. Participants see the question, the model's answer, and one explanation condition. They decide whether to accept the answer and rate clarity. The primary outcome is the correct-rejection rate on wrong answers, and the secondary outcome is acceptance of correct answers.

Our prediction is that unconstrained and truth-verified narratives will lower correct rejection relative to no explanation, because plausible true facts rationalize mistakes, and that the causally filtered system will raise it. Our system matching the truth-verified narrative would falsify the core claim. Raw paths or templates matching it would falsify the case for free narrative. A replication on Movie KG-Exp, where participants judge whether a recommendation fits a stated user preference, tests whether the effect holds outside multiple-choice commonsense questions.