Spelling Is a Reference Problem: Conversational Query Rewriting for Unstandardized Dialects

Conversational search systems turn a context-dependent follow-up such as "how many people live there?" into a self-contained query, "population of Munich," that an ordinary search engine can answer. Every such system assumes the user spells words the same way the index does. In unstandardized dialects that assumption fails. A Bavarian speaker may write Minga, an Alemannic speaker Münche, and the same person may switch spellings across turns. Arabic users often type in Arabizi, Latin letters plus digits such as 3 for ع, where a single token can stand for several words. In these settings the system has two jobs at once. It must work out what "there" refers to, and it must decide which spelling of that referent will actually match the documents.

We propose that these two jobs are one problem, and that the dialogue history solves both. The earlier turns are where the referent was introduced, and they are also the best local evidence for which word an ambiguous spelling stands for. Human annotators already work this way: transliterating Egyptian Arabizi chat required full conversational context to settle ambiguous characters and numerals (Transliteration of Arabizi into Arabic Orthography). We build a query rewriter that resolves references and spelling together. It is trained with retrieval feedback rather than gold rewrites, which do not exist for these dialects, and it chooses the surface form of each entity that matches the target index. We also build the first multi-turn dialectal conversational search benchmark, so that this joint approach can be tested against pipelines that normalize each turn on its own.

Core Claim

A conversational rewriter that sees the whole dialogue when it chooses spellings will beat systems that normalize each turn in isolation and then resolve references. The gain should be concentrated in one kind of turn: turns where the earlier conversation is what disambiguates the spelling, or where the referent's spelling in the history differs from the index's spelling. If the gains instead spread evenly across all turns, the joint framing is wrong and the improvement is just better normalization. That prediction is what makes the claim testable rather than a generic "dialect-robust conversational search" contribution.

What Exists, and What It Leaves Open

The conversational side supplies the mechanism. Pointer-generator rewriters copy entities from the dialogue history into a standalone query, which lets single-turn systems run unchanged downstream (Scaling Multi-Domain Dialogue State Tracking via Query Reformulation). In a dialect, though, copying faithfully reproduces whatever spelling the user typed, and that spelling may not match the index.

Later rewriters were trained against retrieval outcomes rather than human rewrites. AdaCQR aligns one rewriter to both a keyword-matching retriever (BM25) and a dense retriever, which compares learned vector representations (ANCE). Two other papers drop the need for gold passages or gold rewrites altogether: Breaking the Reference Bottleneck and References Indeed Matter?. We reuse this retrieval-feedback training directly, because dialects have no human-written rewrites to imitate.

ConvGQR adds a second ingredient: it generates a guessed answer and appends it to the query to bridge vocabulary gaps. This is a risk in our setting. EDRAC found that strong models drift from dialect into Modern Standard Arabic. GPT-5.4 fell from 1st to joint 5th place once scoring used a dialect-aware metric. Generated expansions may therefore come out in the standard language and miss a dialectal index.

The dense alternative, ConvDR, encodes the raw history into a vector, distilled from a single-turn retriever that sees human rewrites. It avoids generation errors and runs about 50x faster than generative rewriters.

The orthography side constrains the design. Cross-Dialect Information Retrieval introduced the WikiDIR benchmark of seven German dialects and found three relevant things. BM25 loses nearly half its effectiveness on queries containing dialectal entity variants. Zero-shot Llama-3-8B re-ranking scored 0.723 nDCG@10, below BM25 at 0.737. Translating documents into Standard German before indexing raised BM25 by +0.29 on average. Both zero-shot multilingual encoders and translation bridges are therefore suspect, and normalization helps but has only been measured in single-turn retrieval.

A Conventional Orthography for Algerian Arabic shows that many dialect words, including Berber, French and Turkish loanwords, have no clean standard-language target. Normalizing toward the standard language can therefore destroy the entity a user is asking about.

The closest prior system that crosses both directions is CL-QR. It rewrites Spanish, French and Italian assistant queries by pivoting through English and then translating back. Our proposal differs in three ways: it has no translation pivot, it targets languages with no stable orthography, and it uses the history to choose spellings rather than only to resolve intent. The dialect-understanding work in MaiNLP at VarDial 2025 is single-utterance. We borrow its noise augmentation, which trains on artificially perturbed text, as a baseline ingredient.

Research Program

The benchmark. No multi-turn dialectal retrieval benchmark exists, so we build one with two tracks.

The German track uses the WikiDIR dialect Wikipedias as the document collection. We group its query–article pairs into topical sessions of four to eight turns. Native speakers then write follow-up turns in their dialect that use pronouns, elided noun phrases and topic shifts, with spelling left free. The LLM-drafted, speaker-verified pipeline that EDRAC used is acceptable for first drafts. Speakers also record the referent of each context-dependent turn.

The Arabic track uses EDRAC's 499 passages as the collection. EDRAC has about ten QA pairs per passage, which we chain into conversations. Speakers rewrite each question into context-dependent form and produce both an Arabic-script version and a free Arabizi version.

Every turn is tagged on three properties. The first is whether it needs reference resolution. The second is whether the referent's surface form in the history differs from the form used in the relevant document. The third is whether it contains a spelling that is ambiguous without the history; annotators flag these following the categories in the Arabizi transliteration study. Labels for relevant passages come directly from construction, so evaluation needs no rewrite references.

The model. A multilingual sequence-to-sequence rewriter reads the full history and the current turn and outputs a standalone query. Before decoding, each entity-like span in the history and the current turn gets a set of candidate surface forms from three sources. One is the user's own spellings across earlier turns. Another is the index's vocabulary neighbors under a character-level similarity measure. The third is an off-the-shelf normalizer's output. The decoder can copy from the history, copy from this candidate set, or generate freely. Choosing among candidates is part of the output sequence, so it is trained by the same signal as everything else.

Training follows the retrieval-feedback approach of AdaCQR and the reference-free preference methods. We sample several rewrites per turn, score each by the rank of the relevant passage under both BM25 and a dialect-fine-tuned dense retriever, and apply Direct Preference Optimization (DPO, which trains a model to prefer higher-scored outputs over lower-scored ones). The user-feedback variant of this training appears in Improving Contextual Query Rewrite through User-preference Feedback Learning. Answer-style expansion as in ConvGQR is an optional, separately ablated component, because EDRAC gives reason to fear it will drift into the standard language.

Baselines

The naive reference point is the raw current turn sent to BM25 and to the dense retriever with no history and no normalization. It sets the floor and shows how much of the task depends on context at all. Next to it we run the raw concatenated history, which shows how much context helps without any rewriting.

The strongest single-turn dialect method is fine-tuned ColBERTv2, a retriever that matches query and document token by token. It reports 0.785 nDCG@10 on single-turn WikiDIR (Cross-Dialect Information Retrieval). We fine-tune it on the same dialect training data and feed it the concatenated history. Beating it means higher nDCG@10 and MRR on context-dependent turns of our German track.

The strongest conversational rewriter is AdaCQR. It reports 52.4 MRR on QReCC with BM25 and 32.8 MRR on TopiOCQA with ANCE (AdaCQR). Those are English figures. There is no reported result found on Lacuna for any conversational rewriter on dialectal data, so all comparisons on our benchmark are run by us. We train AdaCQR on our training conversations with the same backbone size, the same retrievers, the same number of sampled rewrites per turn and the same tuning budget as our model.

The pipeline baseline normalizes each turn independently and then runs AdaCQR on the normalized dialogue. The normalizer is LLM translation into the standard language for German and a transliteration model trained on the Arabizi corpus for Arabic. The dense baseline is ConvDR fine-tuned on the dialect conversations with spelling-noise augmentation.

Two ablations isolate the key ingredient. In the first, the spelling candidate set is built from the current turn only, with history used for reference resolution but not for spelling. In the second, candidates come only from the normalizer. The proposal counts as successful only if it beats the pipeline and the context-free-spelling ablation on the turns tagged "history disambiguates spelling," with a paired significance test over turns.

Why not just…?

Why not just normalize every turn first and then run an existing rewriter? This is the most natural design, and we run it as a baseline. Two pieces of evidence suggest it will fall short on exactly the turns that matter. In the Arabizi corpus, about 12% of human disagreements came from character and numeral ambiguity that annotators settled by reading the conversation. The Algerian orthography work shows that many loanwords have no standard target, so normalization can erase the entity. The context-free-spelling ablation measures this directly. If the pipeline matches our model on the history-disambiguated turns, the core claim fails.

Why not just translate each turn into the standard language with an LLM? Translating documents before indexing gave BM25 +0.29 nDCG@10 on WikiDIR, so this is a real threat for the German track. Against it, the same paper found zero-shot Llama-3-8B below BM25 on dialect text, and EDRAC documents drift toward the standard language. Adding a generation step also stacks latency on top of the rewriter, which ConvDR's speed comparison shows is costly. We run both translate-then-rewrite and translate-the-index variants. If either matches our model on Arabic Arabizi conversations, we will report that the joint model is unnecessary there.

Why not just fine-tune a dense conversational encoder with spelling-noise augmentation? This may be the strongest competitor. It never commits to a spelling, so it cannot make an unrecoverable error. We think it is vulnerable on long dialogues, where unfamiliar dialect tokens pile up in the history. The WikiDIR results suggest the underlying encoders are weak on dialect vocabulary to begin with. Our turn-depth analysis below tests this vulnerability. If the dense encoder holds up at all turn depths, a rewriter is only worth its cost when the system must call an external keyword search engine.

Proposed Experiments and Decisive Checks

The main evaluation reports nDCG@10, MRR and Recall@100 for every system on both tracks, split by the three turn tags. The first decisive check is whether our model's advantage over the pipeline and over the context-free-spelling ablation is concentrated in turns where history disambiguates spelling or where history and index spellings differ. The second is a degradation curve, plotting retrieval quality against turn depth for the rewriters and for ConvDR. This tests whether dense encoders degrade faster under dialect variation than explicit rewriters do, a question raised by the contrast between ConvDR's robustness and the dialect failures in WikiDIR.

The third check concerns expansion. We measure the fraction of expansion tokens that are standard-language forms absent from the dialect index, scored with a dialect-aware metric of the kind EDRAC recommends. We expect ConvGQR-style expansion to help on the Arabic-script index and hurt on dialect-only collections. The fourth check repeats the evaluation with a randomly shuffled history. If results are unchanged, the model is not using context for spelling at all.

All results are reported per dialect, because the WikiDIR results suggest effects will differ sharply between, for example, North Frisian and Bavarian.