Introduction
Conversational Question Answering (CQA) presents a unique challenge compared to traditional single-turn machine reading comprehension (MRC). In CQA, questions are often interdependent; a model must understand the context of previous turns to resolve coreferences and ellipses. For instance, answering "Is she carrying something?" requires knowing who "she" refers to from earlier in the dialogue.
Most existing CQA models address this by concatenating previous questions and answers into the current input. However, this approach is often superficial and redundant, leading to a massive memory footprint and potential confusion for the model as the conversation grows. This paper introduces the Gated Recurrent Memory Reader (GRMR), a novel architecture designed to model historical memory deeply and selectively while maintaining a lean storage profile.
Methodology: The Gated Recurrent Memory Reader
The GRMR framework reimagines CQA as a recurrent sequence-to-sequence task. Instead of simply matching text, it "generates" extractive answers by predicting start and end pointers within the passage turn by turn.
The architecture consists of two primary components:
- Passage Encoder: Encodes the input text into an intermediate representation.
- Question Decoder with Gated Memory: This is the core innovation. Unlike models that re-encode history every turn, GRMR maintains a hidden state that evolves. It uses a Gated Memory Layer (similar to GRU/LSTM mechanisms) to decide which parts of the passage and previous questions remain relevant. This allows the model to "forget" distractions and focus on the current conversational thread.
The model also employs an interaction layer with attention mechanisms to align the current question with the passage, followed by a pointer network decoder to identify the answer span.
Experimental Results
The researchers evaluated GRMR on the CoQA (Conversational Question Answering) dataset, which includes 127k questions across diverse domains.
Performance and Efficiency
GRMR achieved an F1 score of 66.3 on the CoQA development set, outperforming other non-pretrained baselines such as DrQA+PGNet (66.2). While it does not yet match the performance of massive pre-trained models like BERT, it demonstrates the power of efficient memory management.
The most significant advantage of GRMR is its storage efficiency. By avoiding the redundant concatenation of history, GRMR maintains a constant, lean profile.
As shown in the data, traditional models require up to 4x more storage space than GRMR when handling a 3-turn context. This makes GRMR particularly suitable for deployment in resource-constrained environments.
Ablation Study
Ablation experiments confirmed the importance of the gated memory mechanism. Removing the gated memory caused the F1 score to drop significantly from 66.3 to 58.8, proving that the recurrent memory—not just the base encoder—is doing the heavy lifting for conversational understanding.
Conclusion and Future Work
The Gated Recurrent Memory Reader offers a smarter, more efficient way to handle the complexities of conversational QA. By treating history as a selective, evolving memory rather than a growing text block, it achieves competitive performance with a fraction of the storage requirements. Future work could involve integrating pre-trained Transformer encoders to close the performance gap with state-of-the-art models and extending the framework to handle generative (non-extractive) answers.
Summary updated