Typed Candidate Tries as the Arena for Knowledge Conflicts

When a retrieval-augmented language model answers "Which country is X located in?", two things can go wrong at once. The model can ignore the retrieved passage in favor of what it memorized during pretraining, or it can adopt a wrong passage too readily. And when a decoding trick pushes it away from its memorized answer, it can land on something that is not even a country, such as a city, a film, or a string that names no real entity. Current conflict-resolution methods reweight the model's scores over its entire vocabulary of tens of thousands of tokens, so they have no way to see or prevent that last failure. Current constrained-decoding methods force outputs into a list of valid entity names, but they do not ask whether that list should favor the passage or the model's memory.

We propose to resolve parametric-contextual conflicts inside a small, typed candidate set rather than over the open vocabulary. For an entity-valued question, we build a prefix tree (a trie: a tree of token sequences that the decoder is allowed to follow) over the union of entities mentioned in the context and entities the model proposes without context. We then keep only those whose ontological type fits the relation, for example only countries for located_in. Parametric and contextual beliefs are compared as two distributions over this same finite set, and the decision to trust the context is made from their uncertainties measured on that set. The two-sentence version is this: compare "what the model believes without the passage" and "what it believes with the passage" only over answers that are type-valid real entities, and let the relative confidence on that restricted set decide whom to trust. We expect this to remove a class of errors that open-vocabulary contrastive decoding produces. We also expect it to make the trust decision better calibrated in noisy retrieval, because entropy over ten valid candidates carries a different and cleaner signal than entropy over 128k tokens.

Core Claim

The claim has three testable parts. First, a non-trivial share of the errors made by open-vocabulary conflict methods are escape errors: answers that are type-invalid or out-of-catalog, produced when the method suppresses the parametric answer and mass flows elsewhere. No paper in our file measures this share. Second, the adaptive trust weight of Adaptive Contrastive Decoding (ACD), which compares prediction entropy with and without context, is computed over the full vocabulary. When computed instead over the typed candidate set, it separates "context is informative" from "context is a distractor" more reliably. Third, a type violation inside the context is itself evidence that the context is corrupt. When a passage asserts a film as someone's birthplace, the constraint should lower trust in the passage, not merely filter the output. That third point turns semantic constraints from an output filter into an input-reliability signal. We have not seen it made in any of the work below.

Formally, let $S$ be the typed candidate set for query $x$ with context $c$. Let $p(e \mid x)$ and $p(e \mid c, x)$ be the model's sequence probabilities for candidate entity $e$ without and with context, both renormalized over $S$. Let $H_S(\cdot)$ denote entropy over $S$. We score each candidate as

$$s(e) = (1+\alpha_S)\log p(e \mid c, x) - \alpha_S \log p(e \mid x), \qquad \alpha_S = \frac{H_S(p(\cdot \mid x))}{H_S(p(\cdot \mid x)) + H_S(p(\cdot \mid c, x))},$$

which is ACD's weighting with both the entropies and the contrast moved onto $S$. We add one ingredient: if the context's asserted answer has no type-valid reading, it is excluded from $S$, and $\alpha_S$ is shrunk toward zero so that the model falls back on memory.

Position Relative to Prior Work

Three conflict methods motivate the proposal and supply its core arithmetic. Context-aware Decoding (CAD) subtracts context-free logits from contextual logits with a fixed weight. ACD makes that weight entropy-adaptive and reports doubling CAD's exact match on noisy TriviaQA. CoRect patches deep feed-forward layers that overwrite the context answer. All three decode over the open vocabulary. We keep ACD's weighting idea but change the space it operates on, and we add a type-based reliability signal that none of them has. FastMem instead fine-tunes the last feed-forward block on the prompt at test time. It is a strong competitor on context-faithfulness, but it also leaves the output unconstrained.

On the constrained-decoding side, Autoregressive Entity Retrieval (GENRE) supplies the trie mechanism, masking tokens so outputs are valid Wikipedia titles. It enforces only lexical validity, not relational type. The closest existing work is Retrieval-Constrained Decoding (RCD), which builds a trie from knowledge-base entities retrieved for a question and shows large gains in measured parametric recall on YAGO-QA and KAMEL. RCD differs from us in three ways. It has no context passage that conflicts with memory. Its trie is not filtered by the relation's type. And it contains no mechanism for deciding between context and memory. Our method is essentially RCD's typed-trie output space combined with ACD's adjudication, plus the type-as-reliability signal. Text2Event constrains schema tokens with a trie and arguments to source spans, but it treats the source as authoritative. It is the exact design we must beat on conflict benchmarks, because a context-span trie trivially favors the passage. KCTS keeps generations grounded using tree search guided by a hallucination classifier. It is expensive and type-agnostic, and it targets long-form grounding rather than entity answers, so we treat it as related but not a primary baseline. Relation-Constrained Decoding softly enforces relational constraints with a per-step external classifier. We instead apply types once, at candidate-set construction.

The typing machinery comes from knowledge-graph completion. Type-Constrained Representation Learning showed large gains from restricting candidates to a relation's domain and range. It also introduced the local closed-world assumption, which infers a relation's valid types from the entities seen with it and so handles missing schemas. CAKE prunes concept-violating candidates before ranking. Both constrain us as well as help us: they warn that incomplete taxonomies cause over-filtering, which is exactly the failure our type filter could introduce.

Research Program

The first stage is diagnostic and could stand alone. We run CAD, ACD and FastMem on entity-valued conflict data and classify every error into one of four kinds: stubborn (the memorized answer despite a correct context), gullible (the context answer when the context is wrong), type-invalid, or out-of-catalog. To do this, we link each output to Wikidata with a GENRE-style linker and check its type against the relation's range. If escape errors turn out to be a negligible fraction, the output-constraint half of the proposal loses its motivation. In that case the project narrows to the reliability-signal question.

The second stage builds the typed candidate trie. The set $S$ has three sources. The first is context mentions linked to the knowledge base. The second is the model's top-$k$ context-free generations, linked the same way. The third, optionally, is RCD-style knowledge-base retrieval for the question. Range types come from Wikidata property constraints where they exist, and from local-closed-world pseudo-types otherwise. A context span that cannot be linked to the knowledge base is kept in $S$ with an "untyped" flag rather than dropped, because that is what an updated or emerging entity looks like. Candidates share token prefixes in the trie, so scoring all of them under both conditions costs two prompt passes plus short batched continuations, which is comparable to CAD's doubled cost.

The third stage tests the reliability signal. We construct a type-corrupted variant of NQ-Swap in which the swapped-in entity has the wrong type, alongside the original same-type swaps. On type-corrupted items the correct behavior is to distrust the context. On same-type swaps it is to follow the context. A method that uses types only as an output filter cannot do both, and one that uses them to modulate $\alpha_S$ can.

Baselines

The naive reference is greedy decoding with the passage in the prompt, and the second naive reference is the same decoding with an instruction stating the required answer type. Both use the same model and prompt. FastMem reports 59.1% NQ-Swap accuracy for plain Llama 3-8B-Instruct, and our reproduction should land near that number before any comparison is trusted.

The strongest existing method for the noisy-context regime is ACD, which reports 42.06 exact match on the noisy TriviaQA subset with Llama 2 7B, against 24.75 for CAD (ACD). For pure conflict faithfulness, the strongest number in the file is FastMem's 71.6% on NQ-Swap with Llama 3-8B-Instruct (FastMem). CAD's NQ-Swap gain is reported only as "nearly 3x" on LLaMA-30B, and CoRect's only as "~10%", so we have no absolute values to compare against for those two. We run all of them ourselves on identical models, contexts and answer normalization, and each receives the same hyperparameter budget, tuned on the same held-out split: CAD's $\alpha$, FastMem's learning rate and number of steps, and our top-$k$ for parametric candidates. Beating ACD means higher exact match on noisy TriviaQA at the same model size. Beating FastMem means higher NQ-Swap accuracy without per-query fine-tuning. We also require that neither baseline wins on the other's home benchmark.

For recall without conflict, RCD is the baseline. It reports Llama-3.1-70B F1 rising from 32.3% to 46.0% on YAGO-QA and from 8.7% to 32.0% on KAMEL (RCD). Our method run with no context should not fall below RCD. If it does, the type filter is over-pruning.

Four ablations isolate the ingredients. The first is RCD's trie with the passage added to the prompt, but with no contrast and no typing. The second adds typing but uses a fixed contrast weight. The third uses the typed trie but computes ACD's weight over the full vocabulary, which isolates the claim that entropy should be measured on $S$. The fourth removes the type-as-reliability shrinkage, which isolates the third claim.

Why not just…?

Why not just run ACD and snap its output to the nearest type-valid entity afterward? Post-hoc repair can fix a malformed string. It cannot recover the right answer once decoding has committed to a wrong entity, and snapping a city to a country by string similarity is often wrong. The stage-one error taxonomy tells us how many ACD errors are escape errors. We also run ACD with post-hoc linking and type filtering as an explicit baseline. If it matches our method, the in-decoding constraint adds nothing, and we will say so.

Why not just restrict answers to spans that appear in the context, as Text2Event does? This is the real threat on NQ-Swap, where the swapped answer always appears verbatim in the passage, so a context-span trie may match or beat every conflict method there. We report it on NQ-Swap honestly. It should fail in two regimes: noisy retrieval, where the passage contains same-type distractors or lacks the answer entirely, and the type-corrupted swaps, where following the passage is wrong. The claim that our method is worthwhile rests on those regimes, not on NQ-Swap.

Why not just tell the model the answer type in the prompt? It is cheap and works through a black-box interface. It is the naive-plus-instruction baseline above. If instruction alone removes type-invalid errors, the output-constraint claim reduces to the other two claims: better-calibrated trust from entropy on $S$, and types as a reliability signal. Those two are tested by the third and fourth ablations regardless.

Proposed Experiments

We use Llama 2 7B, to match ACD's reported numbers, Llama 3-8B-Instruct, to match FastMem, and Llama-3.1-70B, to match RCD. The benchmarks are NQ-Swap, the noisy and gold TriviaQA and NQ subsets used by ACD, PopQA, and YAGO-QA and KAMEL with conflicting passages injected by swapping the gold entity for a same-type or wrong-type entity. The metrics are exact match or F1, the fraction of type-invalid and out-of-catalog outputs, stubborn and gullible rates reported separately, and wall-clock cost per query relative to CAD.

Three checks decide the outcome. The first is whether escape errors make up a meaningful share of ACD and CAD errors. The second is whether computing the trust weight on $S$ beats full-vocabulary weighting on noisy TriviaQA with everything else held fixed. The third is whether, on the type-corrupted swaps, our method keeps the correct memorized answer while still following the context on same-type swaps, which no filter-only baseline can do. If only the third check succeeds, the contribution is a cheap corruption detector for retrieval-augmented answering, and that alone would justify the work.