TL;DR
Existing unsupervised domain adaptation methods for Machine Reading Comprehension (MRC) generate synthetic question-answer (QA) pairs that are frequently mismatched—a challenge known as noisy correspondence. This paper introduces RMRC (Robust MRC), a framework that leverages document-associated multi-turn dialogues to extract realistic QA pairs and uses a reinforced self-training loop where MRC feedback continuously optimizes question selection in the target domain.
What Problem Does This Paper Address?
When transferring an MRC reader from a high-resource source domain to an unannotated target domain, conventional approaches train a question generator on source data and synthesize pseudo-QA pairs from target documents. However, this introduces severe noisy correspondence (NC)—where generated questions and candidate answer spans are misaligned or irrelevant to each other.
Unlike standard label noise (incorrect class tags on independent samples), NC stems from relationship errors between paired data points. In target-domain MRC, it occurs because:
- Target documents contain raw text rather than natural questions, prompting generators to hallucinate irrelevant questions.
- The QA generator itself suffers from domain shift when applied directly to new domains without fine-tuning.
Methods
To resolve noisy correspondence, RMRC leverages both target documents and real-world document-associated dialogues (e.g., customer service chats) across two alternating stages:
[Document + Dialogue]
│
├──> Answer Extractor (AE) ──> Filters irrelevant chats (Eq. 3) ──> Candidate Answers
│
└──> Question Selector (QS) ──> Contextual question fusing (Eq. 6) ─> Pseudo QA Pairs
│
┌────────────────────────────────────────────────┘
▼
[MRC Model] ── (Calculates F1 Reward)
│ │
(Cross-Entropy Loss) (REINFORCE Policy Gradient)
▼ ▼
Update MRC Update QS
- Answer Extractor (AE): Extracts candidate $n$-gram spans from target documents and aligns them with answer turns in the dialogue using BERT embeddings. An empirical similarity threshold ($\gamma$) filters out irrelevant chit-chat (e.g., greetings).
- Question Selector (QS): Examines up to $\tau = 16$ previous conversational turns before a given answer. To overcome discontinuous chat patterns (e.g., a user asking multiple questions before receiving an answer), it fuses the top-$\kappa$ most relevant question turns into a consolidated query.
- Reinforced Self-Training for QS: Because question selection involves a discrete
argmaxoperation, gradients cannot backpropagate directly from the MRC model to the QS. RMRC treats the QS as an agent and uses REINFORCE policy gradient: the F1 score of the MRC model's answer prediction acts as the training reward (centered against a baseline reward $r_b$ to reduce variance). As the MRC model improves, its feedback guides the QS to select better questions, mitigating the domain shift of the QA construction module.
Datasets
- Pre-training Sources: SQuAD for English; a combination of CMRC, DRCD, and DuReader for Chinese.
- d-QuAC and d-CoQA: Synthetically perturbed variants of the conversational datasets QuAC and CoQA. To simulate noisy, discontinuous dialog interactions, questions are randomly shifted 1 to 5 rounds ahead of their corresponding answers.
- Alipay Dataset: A real-world Chinese customer service dataset containing 1,526 dialogues tied to marketing activity documents and 3,813 human-annotated test QA pairs.
Key Insights and Ideas
- Dialogues bridge the natural question gap: Real dialogues contain natural human inquiries, eliminating the need to hallucinate questions from scratch using purely generative language models.
- Multi-round question fusing handles dialogue chaos: Natural conversations are rarely strictly turn-by-turn. Selecting and concatenating the top-$\kappa$ related candidate questions resolves conversational discontinuity.
- Closed-loop adaptation beats fixed pseudo-labeling: Conventional pipelines generate static pseudo-QA sets once. Framing the MRC reader's F1 score as a reward allows the question selector to dynamically adapt to target domain semantics without human supervision.
Key Results
RMRC consistently outperforms prior domain adaptation techniques (such as UQACT, CASe, and AdaMRC):
- Synthetic Noise Benchmarks (d-QuAC & d-CoQA):
- On the full d-QuAC benchmark, RMRC reaches 8.27 Exact Match (EM) / 24.46 F1, outperforming the best baseline (CASe at 6.79 EM / 19.33 F1).
- On d-CoQA, RMRC achieves 20.97 EM / 49.35 F1 (vs. CASe's 17.01 EM / 43.27 F1).
- In low-resource regimes (only 1,000 target training instances), RMRC maintains a sizable lead (d-QuAC: 5.67 vs. 4.12 EM; d-CoQA: 16.08 vs. 12.63 EM).
- Real-World Noise (Alipay Dataset):
- RMRC achieves 9.28 EM / 55.67 F1, substantially surpassing CASe (5.75 EM / 44.28 F1) and the source-only BERT model (1.81 EM / 32.62 F1).
- Ablations: Removing the reinforced QS training causes a performance drop of ~3.95 F1 on the Alipay dataset, while omitting answer filtering drops F1 by over 20 points, underscoring the severity of conversational noise.
Limitations and Future Work
- Dialogue Dependency: While the authors show RMRC can filter purely generated questions (Table 4), the framework achieves its strongest gains when paired with human conversation logs, which may not exist for all target domains.
- Reward Sensitivity: As shown in parameter sensitivity experiments, performance is sensitive to the baseline reward $r_b$ and the question fusion parameter $\kappa$; an improperly tuned reward can discourage valid predictions and degrade QS guidance.
- Computational Cost: Alternating between fine-tuning the reader and running policy-gradient updates across multiple candidate conversational turns introduces higher training latency than single-pass generation pipelines.
Summary updated