Abstract
This paper proposes a Distilled-Exposition Enhanced Matching Network (DEMN) for the story-cloze test, which remains a challenging task in story comprehension. We divide a complete story into three narrative segments: an exposition, a climax, and an ending. The model consists of three modules: an input module, a matching module, and a distillation module. The input module provides semantic representations for the three segments and then feeds them into the other two modules. The matching module collects interaction features between the ending and the climax. The distillation module distills the crucial semantic information in the exposition and infuses it into the matching module in two different ways. We evaluate our single and ensemble model on the ROCStories Corpus, achieving an accuracy of 80.1% and 81.2% on the test set respectively. The experimental results demonstrate that our DEMN model achieves state-of-the-art performance.
Introduction
The Story Cloze Test is a benchmark for commonsense story comprehension. In this task, a system is given a four-sentence plot and must choose the most plausible concluding sentence from two options. Coherent storytelling requires more than linguistic fluency; it demands causal reasoning and contextual awareness.
Interestingly, prior research indicated that models often performed better when ignoring the first three sentences (the exposition), suggesting that background information could act as "noise" that distracts neural networks. The Distilled-Exposition Enhanced Matching Network (DEMN) challenges this by demonstrating that the exposition is vital when properly filtered. By distilling only the most relevant background information and injecting it into the climax-ending matching process, DEMN achieves state-of-the-art results.
Methods
The DEMN architecture is organized into three narrative-aware modules designed to capture the story's structure:
- Input Module: Employs a bidirectional LSTM (BiLSTM) to generate semantic representations for the Exposition (sentences 1-3), the Climax (sentence 4), and the Ending Options.
- Matching Module: Focuses on the interaction between the climax and potential endings. It utilizes word-level attention and a multi-turn aggregation mechanism with a memory matrix to refine these interactions over several steps.
- Distillation Module: This core innovation "distills" the exposition. Using attention weights calculated from the climax and options, it extracts only the background details relevant to the conclusion.
The distilled information is integrated via two methods: DEEM (Enhanced Memory), which initializes the matching module's memory, or DEEAV (Enhanced Aggregation Vector), which concatenates a summary vector to the final representation before prediction.
Results and Key Insights
DEMN was evaluated on the ROCStories Corpus, a standard collection of five-sentence commonsense stories. The model set a new state-of-the-art (SOTA) performance at the time of publication, with a single model accuracy of 80.1% and an ensemble accuracy of 81.2%. This outperformed existing high-performing models like Memory Chains (78.5%) and HCM (77.6%).
Key insights from the study include:
- Narrative Structure Matters: Treating a story as a sequence of functional parts (Exposition vs. Climax) allows the model to weigh information importance more effectively.
- Noise Reduction: Distillation acts as a filter, preventing the model from being confused by irrelevant details in the early plot.
- Multi-turn Refinement: Visualization of the memory turns reveals that the model progressively "highlights" critical keywords (e.g., "study" or "test") over multiple iterations, effectively "thinking through" the connection between the background and the ending.
Limitations and Future Work
Error analysis identified three primary areas where the model still struggles:
- Negation: The model occasionally fails to grasp the logic of negative constraints.
- Complex Idioms: Phrases like "received a write-up" require deeper semantic understanding than word-level matching provides.
- Redundant Cues: When both ending options share many words with the exposition, the model can struggle to distinguish the correct causal path.
Future research may involve incorporating external knowledge graphs to better handle these complex semantic and idiomatic relationships.
Summary updated