Contrastive Learning with Narrative Twins for Modeling Story Salience

Igor Sterner, Alex Lascarides, Frank Keller

TL;DR

This paper introduces a contrastive learning framework to model narrative salience by training story embeddings on "narrative twins"—stories with identical plots but different surface wording. By evaluating four classic narratological theories, the authors demonstrate that a sentence's ability to summarize the overall narrative is the most reliable computational indicator of its importance to the plot.

What Problem Does This Paper Address?

When we read a story, some events are critical to the plot's progression (like a key revelation in a mystery novel), while others provide background details or minor character color. In Natural Language Processing (NLP), identifying these highly important or "salient" events is crucial for tasks like automatic story summarization and narrative understanding.

However, evaluating narrative salience computationally remains a challenge. Current approaches lack a systematic understanding of how to build story embeddings that capture plot structure rather than surface-level text features. Furthermore, it is unclear which theoretical definition of salience—such as how much a sentence alters a story if deleted, or how resistant it is to being moved around—is most effective when translated into an algorithm.

Methods

The authors propose a contrastive learning framework to learn robust story embeddings. The model is trained using triplets:

  1. Anchor narrative ($x$): The original story.
  2. Narrative twin ($x^+$): A positive sample that shares the exact same plot but uses different words.
  3. Distractor ($x^-$): A hard negative sample that features the same characters and setting but a completely different plot.

By optimizing an InfoNCE loss, a shared bidirectional encoder ($f_\theta$) learns to map twins close together in the embedding space while pushing distractors away.

If explicit narrative twins are unavailable, the authors introduce dropout twins—approximating a twin by passing the same story through the encoder twice with different random dropout masks. For long narratives, they use in-story negatives (different sections/windows of the same text) as distractors.

At inference time, they operationalize four distinct narratological theories to score sentence salience:

  • Deletion: Measures how much the story embedding changes when a specific sentence is removed.
  • Shifting: Measures the embedding change when a sentence is moved to every other possible position in the text.
  • Disruption: Evaluates the level of "surprise" or narrative shift introduced by a sentence given its prior context.
  • Summarization: Computes the direct cosine similarity between an individual sentence embedding and the global embedding of the entire story.
Figure 1: Left: Overview of contrastive training for learning narrative embeddings. Inputs are an anchor narrative x, its narrative twin x+, and a distractor x−. A shared encoder 𝑓𝜃generates embeddings. The embeddings are optimized with a contrastive loss to be similar for twins and dissimilar to distractors (and in-batch negatives). Right: At inference, we score the salience of each sentence via four theoretically motivated operations.

Datasets

  • ROCStories: A collection of short, human-written 5-sentence stories. The authors used an LLM (GPT-4.1 mini) to generate verbose versions, twins, and distractors for 20,000 stories to create a training corpus. They also collected human annotations on 500 stories to establish a ground-truth salience ranking for evaluation.
  • Wikipedia Plot Summaries: A dataset of 99 longer movie plot summaries (averaging ~34 sentences) annotated with human-identified "turning points" (Papalampidi et al., 2019). Aligned narrative twins were sourced from the cross-lingual Tell me again! dataset.

Key Insights and Ideas

  • Summarization Wins the Day: Across both short and long stories, the summarization operation consistently outperformed deletion, shifting, and disruption. This reveals that the most narratively salient sentence is simply the one that aligns closest with the global meaning of the entire narrative.
  • Dropout is a Powerful Substitute: When parallel plot data isn't available, using simple random network dropout works nearly as well as actual textual narrative twins. This makes the approach highly practical for fully self-supervised narrative modeling.
  • Surface Features are Deceptive Shortcuts: Language models easily exploit shallow lexical overlaps (like character names and settings) to match stories. Including hard distractors—negatives with identical characters but altered plots—is absolutely vital to force the model to learn true plot structure.

Key Results

  • ROCStories: Contrastively trained narrative twins combined with the summarization operation achieved a Spearman's correlation ($\rho$) of 0.44 and an Area Under the Curve (AUC) of 0.76, vastly outperforming the standard Masked Language Model baseline ($\rho = 0.21$, AUC = 0.62).
Table 1: Results on ROCStories test set. We report mean ± one standard deviation (computed across three different random seeds). In bold are the winning results according to permutation-based significance tests (see main text).
  • Wikipedia Narratives: The summarization operation achieved the highest average AUC (0.65–0.66), significantly beating out deletion (0.56–0.60) and disruption (0.53–0.54). Dropout twins trained purely with in-story negatives achieved joint-best results, proving the viability of a fully self-supervised setup.

Limitations and Future Work

  • Weak LLM Supervision: For short stories, generating hard negative distractors relies on explicitly prompting LLMs to alter the climax of a story, which injects a subtle form of weak LLM supervision into the pipeline.
  • Coarse Resolution: The current evaluation operates strictly at the sentence level. However, narrative salience is inherently hierarchical—an important event could be a single clause within a sentence, or span an entire chapter. The authors note that future work should explore training models contrastively across multiple structural levels simultaneously.

Summary updated

Abstract

Understanding narratives requires identifying which events are most salient for a story's progression. We present a contrastive learning framework for modeling narrative salience that learns story embeddings from narrative twins: stories that share the same plot but differ in surface form. Our model is trained to distinguish a story from both its narrative twin and a distractor with similar surface features but different plot. Using the resulting embeddings, we evaluate four narratologically motivated operations for inferring salience (deletion, shifting, disruption, and summarization). Experiments on short narratives from the ROCStories corpus and longer Wikipedia plot summaries show that contrastively learned story embeddings outperform a masked-language-model baseline, and that summarization is the most reliable operation for identifying salient sentences. If narrative twins are not available, random dropout can be used to generate the twins from a single story. Effective distractors can be obtained either by prompting LLMs or, in long-form narratives, by using different parts of the same story.
BibTeX
@inproceedings{sterner2026contrastive,
  author = {Igor Sterner and Alex Lascarides and Frank Keller},
  title = {{Contrastive Learning with Narrative Twins for Modeling Story Salience}},
  booktitle = {EACL (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.71},
}
Figure