Generating Contrastive Narratives Using the Brownian Bridge Process for Narrative Coherence Learning

Feiteng Mu, Wenjie LiACL 2024 · OpenReview

Abstract

A major challenge for narrative reasoning is to learn narrative coherence. Existing works mainly follow the contrastive learning paradigm. However, the negative samples in their methods can be easily distinguished, which makes their methods unsatisfactory. In this work, we devise two strategies for mining hard negatives, including (1) crisscrossing a narrative and its contrastive variants; and (2) event-level replacement. To obtain contrastive variants, we utilize the Brownian Bridge process to guarantee the quality of generated contrastive narratives. We evaluate our model on several tasks. The result proves the effectiveness of our method, and shows that our method is applicable to many applications.

Introduction

Understanding narrative coherence—the logical and causal flow of a story—is a cornerstone of natural language reasoning. While contrastive learning has become the standard approach for training coherence evaluators, its effectiveness is often limited by the quality of negative samples. Traditional methods, such as sentence shuffling or random replacement, produce "easy negatives" that models can identify through simple keyword matching or topical shifts. To develop models capable of deeper reasoning, we require "hard negatives": narratives that maintain topical and character consistency but contain subtle logical flaws.

Methodology

We propose a novel framework for generating hard negatives by modeling narratives as continuous trajectories in a latent vector space.

Brownian Bridge Generation

To ensure that generated variants remain grounded in the original story's context, we utilize the Brownian Bridge process. A narrative is represented as a sequence of latent vectors. By fixing the start and end points of a known story, we sample alternative trajectories that connect these endpoints. These sampled paths are then decoded into text using a BART-based architecture, creating contrastive variants that share the same premise and conclusion but differ in their intermediate events.

The CohEval framework utilizing the Brownian Bridge process for narrative generation.

Negative Synthesis Strategies

We employ two primary strategies to synthesize challenging training data:

  1. Crisscrossing: We create hybrid narratives by swapping prefixes and suffixes between a real story and its Brownian Bridge-generated variant. This forces the model to evaluate the transition between components that are individually coherent but logically mismatched.
  2. Event-level Replacement: We replace specific events with semantically similar but logically inconsistent alternatives. This fine-grained approach targets the model's ability to detect precise causal breaks.

The resulting evaluator, CohEval, is a RoBERTa-large model trained with a contrastive loss to distinguish original narratives from these sophisticated negatives.

Experimental Evaluation

We trained CohEval on the ROCStories dataset and evaluated its performance in a zero-shot setting across several narrative reasoning benchmarks, including COPA, e-Care, $\alpha$NLI, and Swag.

Results and Discussion

CohEval demonstrates significant improvements over existing contrastive training methods and even outperforms several large language models (LLMs).

Performance comparison of CohEval against LLMs and other contrastive methods.

As shown in the results table, CohEval consistently outperforms specialized models like RankGen and EventBERT. Notably, on the $\alpha$NLI and Swag benchmarks, our RoBERTa-based model achieved higher accuracy than 7B-parameter models such as Alpaca-LoRA and ChatGLM2-6B. For instance, on $\alpha$NLI, CohEval reached 67.6% accuracy, compared to RankGen-large's 54.8%.

Ablation studies further confirm that both the Crisscrossing ($M_{CC}$) and Event-level Replacement ($M_{ER}$) strategies contribute to the model's robustness, indicating that coherence is a multi-scale property requiring both global and local consistency.

Conclusion

By leveraging the Brownian Bridge process to generate hard negatives, we have developed a more effective approach to learning narrative coherence. CohEval's strong performance on zero-shot benchmarks highlights the importance of high-quality contrastive data in training reasoning models. Future work will explore scaling the backbone architecture and relaxing the fixed-endpoint constraint to accommodate more diverse narrative structures.

Summary updated

Abstract

A major challenge for narrative reasoning is to learn narrative coherence. Existing works mainly follow the contrastive learning paradigm. However, the negative samples in their methods can be easily distinguished, which makes their methods unsatisfactory. In this work, we devise two strategies for mining hard negatives, including (1) crisscrossing a narrative and its contrastive variants; and (2) event-level replacement. To obtain contrastive variants, we utilize the Brownian Bridge process to guarantee the quality of generated contrastive narratives. We evaluate our model on several tasks. The result proves the effectiveness of our method, and shows that our method is applicable to many applications.
BibTeX
@inproceedings{mu2024generating,
  author = {Feiteng Mu and Wenjie Li},
  title = {{Generating Contrastive Narratives Using the Brownian Bridge Process for Narrative Coherence Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=jAUdDS5B4B},
}
Figure