Conversational Queries as Mediated Alignment: Typed-Path Expansion That Bounds Noise

When a user in a search conversation asks "who produced it?", the system must work out what "it" refers to. It must also supply words that the relevant document actually uses. Current systems handle the second job by letting a language model guess an answer and append it to the query. That guess adds useful vocabulary when it is right and misleading text when it is wrong. This proposal takes the fix from an older field, ontology alignment, which is the task of matching concepts across two differently-worded knowledge bases. Its most reliable technique is mediated alignment. Both sides are first mapped ("anchored") to a large third knowledge graph, the mediator. New matches are then derived by walking typed relations inside it. A 2006 study found that walking mixed relation types without restriction increased the number of matches but introduced nonsense links. Imposing a fixed order on relation types kept the gain in matches at full precision (Aleksovski et al., 2006).

The core claim is that this "mixing trap" and the known sensitivity of language models to irrelevant context are the same failure seen from two sides. Each irrelevant term an expansion admits is a distractor for the reader model downstream. We therefore cast each conversational turn as a small alignment problem. The entities active in the dialogue so far form one side, the retrieval corpus the other, and a typed knowledge graph (Wikidata) is the mediator. The system resolves references and chooses expansion terms only through relation-type sequences that were learned, from training data, to be precise. The testable prediction has three parts. Plain knowledge-graph neighborhood expansion should raise retrieval recall but lower answer accuracy, which is a conversational version of the mixing trap. Restricting expansion to learned typed paths should keep the recall gain without that loss. Accuracy should also fall more slowly than hallucination-based expansion when irrelevant dialogue turns are injected.

Why this gap, and what is already done

The closest existing work is Expanding Query Answers on Medical Knowledge Bases. It maps colloquial terms in a conversational assistant onto SNOMED CT and relaxes failed queries with fixed directional penalties, and it reaches Precision@10 of 90.51% against 75.55% for an information-content baseline. It motivates this proposal but does not cover it. That system works on a single turn whose intent a separate language interface has already extracted. Its penalties are hand-set, and it neither resolves references across turns nor measures how expansion affects a neural reader.

On the conversational side, ConvGQR combines a rewriter with a generator of hypothetical answers used as expansion terms. It beats human rewrites on QReCC (MRR 42.0 vs 38.4 with the ANCE dense retriever) and beats T5QR on CAsT-20 (46.5 vs 42.3). Its authors acknowledge that expansions which diverge from the user's intent derail retrieval. Our expansion source is structured and auditable where ConvGQR's is parametric guessing.

The noise literature constrains the design. Cho et al., 2023 show that zero-shot reader accuracy collapses as more distractor documents are added. PRCA shows that compressing retrieved context helps on TopiOCQA (+9%), but its adapter is tied to one generator through reinforcement-learning rewards. Both of these clean up context after retrieval. We control what goes into the query in the first place, which is a separate lever and can be combined with either.

The method we reuse is the anchor-then-derive procedure and the ordered-relation restriction from Aleksovski et al., 2006. We replace their hand-chosen ordering (part-of before is-a) with one estimated from data. We also borrow the idea from MultiOM that ontology structure, specifically which classes are disjoint, can identify which candidates are unsafe.

Research Program

The system keeps a dialogue entity state: the set of mediator nodes anchored from earlier turns and their answers. Anchoring uses an off-the-shelf entity linker into Wikidata, with confidences.

For the current turn, candidate referents and expansion entities are nodes reachable from the state by paths of length at most three. Each path is described by its sequence of relation types, for example performer → record label.

The system learns which relation-type sequences are admissible from the training splits of TopiOCQA and QReCC. For each sequence, it measures the precision with which paths of that type reach entities that appear in the gold passage. It keeps only sequences above a precision threshold chosen on validation data. This is a direct, data-driven analogue of the Exp 5 ordering.

Admitted entities contribute their labels and aliases as expansion terms, together with a rewrite of the anaphor to the chosen referent. The final query, a rewrite plus bounded expansion, goes to an unchanged retriever. A turn-relevance score in the style of Ríssola et al., 2019 decays or drops anchors from obsolete topics. This keeps topic switches from pulling expansion toward stale entities.

One risk is specific to TopiOCQA. Its topic switches were built by following Wikipedia hyperlinks. The mediator is therefore restricted to Wikidata's typed relations, and the hyperlink graph is excluded, so the path grammar cannot simply recover how the dataset was built. We will report how often gold topic shifts correspond to an admissible Wikidata path. A low rate would itself be a finding about where mediation can help.

The analysis centers on a noise curve. We inject 0 to 6 off-topic turns, taken from other conversations, into dialogue histories. For each method we plot retrieval MRR and reader F1 against the number of injected turns, and we compare slopes. Expansion precision is also reported directly: the fraction of added terms that occur in the gold passage.

Baselines

The naive reference points come from existing numbers. The first is feeding the full history to DPR with a Fusion-in-Decoder reader, which reaches F1 55.8 on TopiOCQA (Adlakha et al., 2022), with humans at 70.0. The second is human-written rewrites, at 38.4 MRR on QReCC. We rerun both with our retriever and reader checkpoints so the numbers are directly comparable.

The strongest existing method is ConvGQR, because it is the published system that already solves the same "rewriting is not enough, add vocabulary" problem. Its reported results are MRR 42.0 on QReCC with ANCE and 46.5 zero-shot on CAsT-20. We run it with the same ANCE retriever, the same training data and the same hyperparameter-search budget as our system. Beating it means higher MRR on QReCC and CAsT-20 at equal retriever. Separately, it means a smaller drop in reader F1 on the injected-noise curve. Matching it on clean MRR while degrading less under noise would still support the claim.

ConvDR (Yu et al., 2021) is the strongest method that uses no text at all. It is reported as 9% over prior rewriting systems on CAsT-19; the file gives no absolute score, so we will reproduce it on CAsT-19 and CAsT-20 under the same teacher. A large LLM rewriter prompted to produce standalone queries is also included. No reported result was found on Lacuna for this on these benchmarks, so we run it ourselves with the same prompt budget across all conditions.

Two ablations isolate the key ingredient. The first is flat expansion: link entities in the history and append all one-hop Wikidata neighbors with no path grammar. The second is unrestricted paths: all sequences up to length three, which is the conversational mixing-trap condition.

Why not just…?

Why not just have a strong LLM rewrite the query? This is the most serious threat. The file offers only indirect evidence against it. Closed-book GPT-3 reaches just 31.8 F1 on TopiOCQA. Wang et al., 2023 find that asking LLMs to judge their own knowledge is the weakest strategy. Neither result tests rewriting directly. We therefore run the LLM rewriter as a baseline, and we also test the combination of LLM rewrite plus typed-path expansion. If the combination adds nothing on topic-shift turns or under injected noise, the mediator is not earning its cost, and we will say so.

Why not just filter ConvGQR's hallucinated expansions, for example with distraction-aware scoring? Filtering after generation can remove wrong terms, but it cannot supply terms that the language model never produced. Cho et al., 2023 also show that the model's confidence is unreliable under noise, with 0.98 assigned to a wrong answer. We will run ConvGQR with query-likelihood reranking of its expansions and compare its expansion precision with ours.

Why not just link entities and append their descriptions? If this flat expansion matches the typed-path method, the learned grammar is unnecessary. That outcome would falsify the central claim. The flat-expansion ablation settles it directly.

Why not a dense conversational encoder and no symbols? ConvDR is fast, at 17.4 ms per query, and it filters history implicitly. It still has no way to bring in vocabulary it never saw, and it offers no inspectable reason for a retrieval. The comparison on the topic-shift subset and the noise curve will show whether that matters in practice.

Proposed Experiments

Retrieval is evaluated on QReCC and TREC CAsT-19/20 using MRR, Recall@100 and NDCG@3, with ANCE as the fixed retriever. End-to-end answering is evaluated on TopiOCQA with DPR and Fusion-in-Decoder, using F1 and EM. Results are broken down into topic-shift turns and same-topic turns.

The decisive check is a three-way ordering on the noise curve: no expansion, unrestricted-path expansion and typed-path expansion. The hypothesis requires three things. Unrestricted expansion must beat no expansion on Recall@100 while losing reader F1 as distractor turns grow. Typed-path expansion must keep the recall gain with a flatter F1 slope than both unrestricted expansion and ConvGQR. Typed-path expansion must also beat flat entity expansion. If the unrestricted condition shows no precision cost, the analogy to the mixing trap fails and the grammar is unnecessary. If typed paths lose to the LLM rewriter even when the two are combined, mediated expansion has no role in open-domain conversation. We would then narrow the claim to specialized-vocabulary settings like that of Lei et al., 2020.