Scene Restoring for Narrative Machine Reading Comprehension

Zhixing Tian, Yuanzhe Zhang, Kang Liu, Jun Zhao, Yantao Jia, Zhicheng ShengEMNLP 2020 · OpenReview

When you read a story, you don’t just process a sequence of words; you build a "mental movie." If a character’s face is covered in blood, you immediately infer they are hurt, might be in pain, and likely need medical help—even if the text doesn't explicitly say so. This process, known as scene restoration, is a hallmark of human intelligence.

In the paper "Scene Restoring for Narrative Machine Reading Comprehension," researchers propose a way to give machines this same "imagination" to better understand complex narratives. By building a scene graph using external commonsense knowledge and a novel encoding network, the model achieves state-of-the-art performance on narrative reasoning benchmarks.

An example of scene restoration where a narrative about an injured man is enriched with commonsense inferences.

The Challenge: Narrative vs. Fact

Standard Machine Reading Comprehension (MRC) models, like BERT or RoBERTa, are incredibly good at finding patterns in text. However, they often struggle with narrative passages, which are defined by a chain of events rather than isolated facts.

The gap lies in "prior knowledge." Humans use their understanding of the world to fill in the blanks between sentences. For a machine, the sentence "Nearby, a young man was dialing his phone" is just a string of tokens. For a human, it’s a potential call for an ambulance. Previous attempts to bridge this gap used word-centric knowledge bases (like ConceptNet), which help define what things are but fail to capture the dynamics of what happens.

Methodology

The proposed framework consists of three main stages designed to mimic human scene restoration:

  1. Knowledge Obtaining: Instead of looking up dictionary definitions, the model uses Atomic, an event-based knowledge graph. They use a pre-trained model called Comet to generate commonsense inferences for every event in the story, such as the character's intent (xIntend), the effect on the character (xEffect), and their attributes (xAttr).
  2. 3D Scene Graph Building: The model constructs a complex graph with three types of nodes: events, person roles, and event knowledge.
  3. Inner-event graphs: Capture the details within a single sentence (e.g., who did what and what was the immediate effect?).
  4. Cross-event graphs: Link different sentences together based on chronological order and coreference (recognizing that "he" in sentence 3 is the same "old man" from sentence 1).
  5. Graph Dimensional Iteration Network (GDIN): This is the core technical innovation. Standard graph neural networks treat all connections the same. GDIN iterates between the two "dimensions" of the graph: it first updates representations within an event (local context) and then updates them across the entire story (global context).
The overall architecture of the proposed method, featuring Scene Graph Building and Dimensional-Iteration Encoding.

Key Insights and Results

The model was evaluated on two major narrative benchmarks: ROCStories (Story Cloze Test) and CosmosQA (commonsense comprehension). The results demonstrated several key findings:

  • Events > Words: Event-based knowledge (how one action leads to another) is far more valuable for stories than concept-based knowledge (knowing that a "face" is part of a "head").
  • The Power of 3D Structure: "Flattening" the graph—treating it as a simple list of sentences—significantly reduced performance. The hierarchical distinction between "what's happening now" (inner-event) and "how does this fit the story" (cross-event) is crucial.
  • Performance: The method achieved 91.9% accuracy on ROCStories and 84.5% accuracy on CosmosQA, outperforming existing baselines like BERT+Diff-Net and ALBERT-FT.

Conclusion

While the model relies on external tools for pre-processing (like dependency parsing and coreference resolution), it marks a significant step toward machines that don't just match text, but actually "understand" the underlying scene. By moving toward "mental" modeling, this research brings us closer to AI that can truly follow the plot.

Summary updated

Abstract

This paper focuses on machine reading comprehension for narrative passages. Narrative passages usually describe a chain of events. When reading this kind of passage, humans tend to restore a scene according to the text with their prior knowledge, which helps them understand the passage comprehensively. Inspired by this behavior of humans, we propose a method to let the machine imagine a scene during reading narrative for better comprehension. Specifically, we build a scene graph by utilizing Atomic as the external knowledge and propose a novel Graph Dimensional-Iteration Network (GDIN) to encode the graph. We conduct experiments on the ROCStories, a dataset of Story Cloze Test (SCT), and CosmosQA, a dataset of multiple choice. Our method achieves state-of-the-art.
BibTeX
@inproceedings{tian2020scene,
  author = {Zhixing Tian and Yuanzhe Zhang and Kang Liu and Jun Zhao and Yantao Jia and Zhicheng Sheng},
  title = {{Scene Restoring for Narrative Machine Reading Comprehension}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=ez8BmJcBME},
}

Loading related content...

Figure