TL;DR
This paper introduces a novel approach to interactive machine reading comprehension (iMRC) by leveraging dynamic knowledge graphs as an agent's memory. By building and updating these graphs on-the-fly, the agent can better navigate partially observable environments and significantly improve its ability to answer questions.
What Problem Does This Paper Address?
Traditional machine reading comprehension (MRC) tasks often present the entire knowledge source (e.g., a document) upfront, allowing models to learn by pattern matching. However, real-world scenarios, like searching the internet, involve interacting with information sources that are only partially observable. Interactive Machine Reading Comprehension (iMRC) aims to address this by simulating environments where an agent must sequentially gather information to answer a question. A key challenge in iMRC is how to effectively manage and utilize the information gathered over multiple interaction steps, especially when the environment itself provides limited history (e.g., only the current sentence). This paper tackles this challenge by proposing knowledge graphs as a structured memory mechanism to aid iMRC agents.
Methods
The core contribution is the integration of dynamic knowledge graphs into an iMRC agent's architecture. The authors propose four types of graphs to represent information at different levels:
- Word Co-occurrence: Connects words that appear in the same sentence.
- Relative Position: Captures word order and distance within sentences.
- Semantic Role Labeling (SRL): Uses parser-based information to link predicates, arguments, and modifiers.
- Continuous Belief (Trainable): A data-driven approach where a neural network learns to maintain a graph representing the agent's belief state.
These graphs are dynamically updated at each interaction step. A Graph Neural Network (GNN), specifically a multi-layer Relational Graph Convolutional Network (R-GCN), is used to encode these graph representations. This graph encoding is then combined with text representations (from the current observation and question) using a context-query attention mechanism. Finally, a recurrent component (GRU) is employed to further incorporate historical information.
Datasets
The experiments were conducted on the iSQuAD dataset, which is an interactive version of the SQuAD dataset. iSQuAD reformulates SQuAD examples into interactive environments by occluding document sentences and defining commands for agents to reveal them.
Key Insights and Ideas
- Graphs as Memory: The central hypothesis is that graph representations act as powerful inductive biases, serving as an effective memory mechanism for agents in partially observable environments.
- Dynamic Graph Construction: The ability to dynamically build and update graphs based on incoming information is crucial for iMRC, allowing the agent's memory to evolve with its interaction.
- Complementary Modalities: Text and graph representations, even when derived from the same text, offer complementary information. Removing text modality significantly degrades performance, suggesting that the attention mechanism effectively fuses these distinct sources.
- Ensembling Benefits: Combining multiple graph types through ensembling leads to substantial performance gains, highlighting that different graph structures capture different, valuable aspects of the information.
Key Results
- Significant Performance Boost: The proposed graph-enhanced agents consistently outperform the baseline iMRC agent across various settings, achieving relative improvements of over 9% on average.
- Effectiveness of Different Graph Types: While all graph types offer improvements, relative position and SRL graphs showed a slight advantage, though the trend wasn't strongly significant.
- Ensemble Dominance: The ensemble of all four graph types achieved the best performance, even surpassing the baseline in challenging "Hard Mode + vocab" settings.
- Robustness with Memory Slots: The benefits of graph representations persist even when the environment provides explicit memory slots (caching recent sentences), suggesting graphs offer a stronger inductive bias than simple history buffering.
- Text Modality is Necessary: Ablation studies show that removing the text modality entirely, relying solely on graphs, leads to performance degradation, underscoring the importance of both text and graph information.
Limitations and Future Work
- Hard Mode + Vocab Challenge: The paper notes that agents perform poorly on "Hard Mode + vocab" games, indicating limitations of RL algorithms like deep Q-learning when faced with extremely large action spaces. This could potentially be mitigated by pre-training strategies.
- Action Space Expansion: The authors suggest that future work could explore expanding the action space, for example, allowing the agent to query multi-word expressions with Ctrl+F, which would require careful consideration of RL training strategies.
- Cooperative Memory Mechanisms: The paper suggests that multiple memory mechanisms (memory slots, graphs, RNNs) are not redundant but rather work cooperatively, hinting at future research in combining these effectively.
- Potential for Undesirable Strategies: As with many RL systems, the agent is optimized solely for the final metric, which could lead to undesirable behaviors. The authors acknowledge the need for careful consideration, especially if action spaces become more complex.
Summary updated