What am I missing here?: Evaluating Large Language Models for Masked Sentence Prediction

Charlie Wyatt, Aditya Joshi, Flora SalimarXiv 2025 · arXiv · OpenReview

TL;DR

This paper investigates how well large language models (LLMs) understand document structure by evaluating their ability to fill in missing sentences (Masked Sentence Prediction). Surprisingly, even top-tier LLMs struggle with this task, especially in less structured domains, revealing a significant gap in their ability to maintain global coherence beyond local fluency.

What Problem Does This Paper Address?

Modern LLMs are primarily trained using Next Token Prediction (NTP), which focuses on predicting the very next word in a sequence. While this approach excels at generating locally fluent text, it doesn't inherently encourage models to grasp the broader context or ensure coherence across entire sentences and paragraphs. This raises a crucial question: how well do these models truly understand the structure and meaning of a document when they need to reconstruct or infer missing pieces of information, rather than just continuing a sequence? The paper aims to bridge this gap by evaluating LLMs on a task that requires sentence-level understanding and global context awareness.

Methods

The core of the evaluation lies in the Masked Sentence Prediction (MSP) task. For a given document, a single sentence is removed and replaced with a special mask token. The LLM is then prompted to generate a replacement sentence. The researchers evaluated three commercial LLMs: GPT-4o, Claude 3.5 Sonnet, and Gemini 2.0 Flash.

Their evaluation focused on two key dimensions:

  • Fidelity: How similar is the generated sentence to the original one? This was measured using standard metrics like BLEURT, SBERT cosine similarity, ROUGE-1, and BLEU.
  • Cohesion: How well does the generated sentence fit into the surrounding context? This was assessed through a blind human preference test where annotators compared the original and generated sentences within their context.

They also explored variations in the task by varying the text domain (narrative, procedural, expository) and the position of the masked sentence (first, middle, last) to understand how these factors influence performance.

Datasets

The study utilized three publicly available datasets:

  • ROCStories: A collection of 5-sentence common-sense narratives.
  • Recipe1M: The cooking-instruction portion of a larger dataset, representing procedural text.
  • Wikipedia: Expository text from encyclopedic articles.

For each dataset, 400 test documents were randomly sampled to balance statistical power with API constraints. No new benchmarks were introduced; the focus was on evaluating existing LLMs on these established domains.

Key Insights and Ideas

  • NTP's Limitation: The paper strongly suggests that NTP, while effective for fluency, is insufficient for tasks requiring deep document understanding and global coherence. LLMs trained solely on NTP may lack the ability to reason about sentence-level relationships.
  • Fidelity vs. Cohesion Trade-off: LLMs can often generate sentences that sound plausible and fit the general flow (cohesion), even if they are factually incorrect or semantically distant from the original (low fidelity). This is particularly evident in less structured domains.
  • Domain Structure Matters: Performance on MSP is significantly influenced by the structure of the text. Procedural texts (like recipes) with clear sequential logic lead to higher fidelity, while narrative and expository texts, which rely more on subtle discourse and global context, pose a greater challenge.
  • Mask Position Impact: Models perform best when the masked sentence is in the middle of the text, benefiting from both preceding and succeeding context. Masking the last sentence yields the lowest fidelity.

Key Results

  • Moderate Fidelity: Across all models and domains, fidelity scores (measured by BLEURT) were generally moderate, rarely exceeding 0.55. This was surprising, as the training data likely contained these corpora.
  • Claude 3.5 Sonnet Leads: Claude 3.5 Sonnet generally achieved higher average BLEURT scores and showed less variance compared to Gemini, indicating more consistent performance.
  • Structured Domains Excel: Fidelity was notably higher in the structured Recipe1M domain compared to ROCStories and Wikipedia.
  • Human Preference: While fidelity was often low, human annotators frequently rated generated sentences as "Equal Preference" (meaning they couldn't definitively choose between original and generated) in narrative and expository domains. However, in Recipe1M, annotators more often preferred the original sentence, highlighting the stricter requirements of procedural text.
  • Failure Modes: Common errors included logical inconsistencies, sequencing issues in recipes, and generating vague or tonally inappropriate sentences in narratives.

Limitations and Future Work

  • Dataset Size: The study used 400 samples per dataset, which, while providing a diagnostic signal, could be expanded for greater statistical significance.
  • Domain Breadth: The evaluation was limited to three domains; a broader range of genres (legal, conversational, etc.) would be needed for more generalizable conclusions.
  • Closed-Source Models: The use of commercial LLMs raises concerns about potential data contamination from their training corpora. Future work could use fully open models with known training data to mitigate this.
  • Single Annotator: The human evaluation was conducted by a single author, which could limit generalizability. Larger-scale evaluations with multiple annotators are recommended.

Future work suggested includes exploring architectures that explicitly capture both local and global context, fine-tuning models on MSP objectives, and incorporating richer prompting strategies. The authors also emphasize the need for caution when deploying LLMs in high-stakes applications where factual precision is paramount, given their tendency to prioritize plausible generation over strict fidelity.

Summary updated

Abstract

Transformer-based models primarily rely on Next Token Prediction (NTP), which predicts the next token in a sequence based on the preceding context. However, NTP's focus on single-token prediction often limits a model's ability to plan ahead or maintain long-range coherence, raising questions about how well LLMs can predict longer contexts, such as full sentences within structured documents. While NTP encourages local fluency, it provides no explicit incentive to ensure global coherence across sentence boundaries-an essential skill for reconstructive or discursive tasks. To investigate this, we evaluate three commercial LLMs (GPT-4o, Claude 3.5 Sonnet, and Gemini 2.0 Flash) on Masked Sentence Prediction (MSP) - the task of infilling a randomly removed sentence - from three domains: ROCStories (narrative), Recipe1M (procedural), and Wikipedia (expository). We assess both fidelity (similarity to the original sentence) and cohesiveness (fit within the surrounding context). Our key finding reveals that commercial LLMs, despite their superlative performance in other tasks, are poor at predicting masked sentences in low-structured domains, highlighting a gap in current model capabilities.
BibTeX
@misc{wyatt2025what,
  author = {Charlie Wyatt and Aditya Joshi and Flora Salim},
  title = {{What am I missing here?: Evaluating Large Language Models for Masked Sentence Prediction}},
  year = {2025},
  eprint = {2508.07702},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.07702},
}
Figure