The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers

Saad Obaid ul Islam, Anne Lauscher, Goran GlavasarXiv 2025 · DOI · arXiv

Abstract

Large Language Models (LLMs) frequently exhibit a "reliability gap," providing inconsistent answers to the same factual question depending on query complexity. This paper introduces the SLAQ (Short-Long Form Alignment for Factual Question Answering) framework to measure this phenomenon. Our evaluation of 16 state-of-the-art models reveals that while models show high raw alignment, they are often "consistently wrong." Furthermore, we identify a "momentum effect" where early errors in long-form generation trigger cascades of hallucinations, and demonstrate that internal neural pathways can predict factual consistency with high accuracy.

Introduction

While LLMs often excel at isolated trivia, their reliability wavers during complex tasks like multi-paragraph summarization. A model might correctly identify a historical date in a short Q&A but hallucinate a different date when asked to write a detailed biography. This research addresses factual consistency over query complexity: will an LLM yield the same answer to the same factual question for queries of varying complexity? If a model cannot maintain internal consistency, its utility as a trustworthy information assistant is fundamentally compromised.

The SLAQ Framework

To systematically evaluate this gap, we developed the Short-Long Form Alignment for Factual Question Answering (SLAQ) framework. The process involves generating a set of five independent "short" questions ($q_1..q_5$) and one "long" query ($LQ$) that requires addressing all five facts simultaneously. Responses are collected from various model families and evaluated using an LLM-as-a-judge against gold-standard answers.

The SLAQ evaluation pipeline: from dataset generation to LLM-as-a-judge verification.

Metrics

We employ four primary metrics to quantify performance:

  1. Factual Correctness ($F_S, F_L$): Accuracy on short-form vs. long-form queries.
  2. Raw Alignment: The frequency with which a model provides the same type of answer (both correct or both incorrect) across formats.
  3. Signed Alignment: A metric distinguishing between being consistently right versus consistently wrong.
  4. Direction of Alignment: Identifying whether errors are more prevalent in short or long formats.

The SLAQ Dataset

The dataset was constructed from Wikipedia, spanning 600 unique topics across 15 categories, including History, Software Engineering, and Theology. To account for the impact of training data frequency on hallucinations, the dataset is balanced between 300 "popular" topics and 300 "obscure" topics. In total, the dataset comprises 3,000 short questions and 600 corresponding long queries, all manually verified for factual grounding.

Key Results and Analysis

The Accuracy Gap and Consistency Trap

Our evaluation across 16 models (including Llama-3, Qwen, and Gemma) shows that short-form accuracy ($F_S$) consistently outperforms long-form accuracy ($F_L$). While models exhibit high raw alignment (73-78%), this is often a "consistency trap." As shown in the results below, the Signed Alignment is almost universally negative, indicating that models are more consistent at being wrong across both formats than they are at being right.

Comparative analysis of factual correctness (Fs, FL), raw alignment, and signed alignment across model families.

The Momentum Effect

We observed a significant "momentum effect" in long-form generation. If a model correctly generates the first two facts in a sequence, the probability of the third being correct increases. Conversely, an early mistake often leads to an "error cascade," where the initial hallucination snowballs into further inaccuracies. This suggests that the context of previous tokens heavily influences the retrieval of subsequent facts.

Mechanistic Fingerprints

Using zero-ablation to identify internal circuits, we found that when a model is consistent across formats, it utilizes similar neural pathways. By analyzing the similarity of internal attention head rankings (using Spearman correlation), we could predict whether a model would be consistent with an ROC-AUC of 0.85. This suggests that misalignment is not just a surface-level linguistic issue but is rooted in the model's internal computational state.

Conclusion

Factual misalignment is an intrinsic property of current transformer architectures that is not easily solved by scaling alone. The tendency for models to be "consistently wrong" and the presence of error cascades highlight the need for targeted interventions. Future work should focus on "nudging" internal circuits to enforce consistency during long-form generation, potentially bridging the reliability gap in complex information retrieval tasks.

Summary updated

Abstract

Large language models (LLMs) can correctly answer "When was Einstein born?" yet fail to provide the same date when writing about Einstein's life revealing a fundamental inconsistency in how models access factual knowledge across task complexities. While models display impressive accuracy on factual question-answering benchmarks, the reliability gap between simple and complex queries remains poorly understood, eroding their trustworthiness. In this work, we introduce Short-Long Form Alignment for Factual Question Answering (SLAQ), a controlled evaluation framework that compares LLMs' answers to the same factual questions asked (a) in isolation (short) vs. (b) integrated into complex queries (long). Looking at 16 LLMs across 600 queries, we find a systematic misalignment of answers to the corresponding short and long queries. We further uncover position-dependent accuracy loss and momentum effects where consecutive correct or incorrect answers create self-reinforcing patterns. Through mechanistic analysis, we find that aligned facts activate overlapping model internals, and that metrics based on mechanistic similarity can predict short-long answer alignment with up to 78% accuracy. Our work establishes factual consistency over query complexity as an important aspect of LLMs' trustworthiness and challenges current evaluation practices, which implicitly assume that good performance for simple factual queries implies reliability in more complex knowledge-seeking tasks too.
BibTeX
@misc{islam2025curious,
  author = {Saad Obaid ul Islam and Anne Lauscher and Goran Glavas},
  title = {{The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers}},
  year = {2025},
  eprint = {2510.11218},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.11218},
}
Figure