The Cross-Domain Generalization Gap in Logical Reasoning

202 concepts161 papers1 proposal

Large language models frequently achieve remarkable scores on specialized mathematical and logical benchmarks. However, this proficiency is often an illusion of narrow specialization. When a model trained to perfection on elementary arithmetic word problems is evaluated on high-school competition math, or when a model optimized for pure algebra is tested on calculus or physics, performance collapses.

This phenomenon is known as the cross-domain generalization gap. As Chengpeng Li et al. demonstrate in MuggleMath, proficiency in one tier of mathematical reasoning does not automatically transfer to adjacent domains. The gap reveals a fundamental tension in AI reasoning: models are rarely learning robust, transferable logical principles. Instead, they are over-fitting to the statistical and linguistic quirks of specific problem distributions.

Why Reasoning Fails to Transfer

To understand why the gap exists, we must look at how models process language versus abstract logic. Ideally, a model should map natural language to an underlying invariant logical skeleton, solve the problem symbolically, and translate the answer back. In practice, models take "learning shortcuts."

As JB. Kim et al. observe in ATHENA, models frequently memorize linguistic patterns rather than developing a grounded understanding of mathematical principles. When the surface-level wording changes or the problem domain shifts slightly, these brittle representations crumble.

This issue extends far beyond word problems into structural reasoning tasks like graph theory. For instance, Yizhuo Zhang et al. show in Can LLM Graph Reasoning Generalize beyond Pattern Memorization? that models struggle to transfer reasoning skills even between closely related graph problems, such as moving from connectivity checks to shortest-path calculations. The models rely on pattern recognition tied to specific problem formats rather than acquiring a robust understanding of graph theory principles.

[Surface-Level Text] ──(Shortcut Learning)──► [Domain-Locked Pattern] ──► Fails on Unseen Domains
         │
  (Desired Mapping)
         ▼
[Abstract Logical Skeleton] ────────────────► [Universal Principle] ──► Generalizes Across Domains

When a model is trained on narrow data distributions, it develops domain-specific biases. Xiangyu Wen et al. argue in Dependency Matters: Enhancing LLM Reasoning with Explicit Knowledge Grounding that supervised fine-tuning for structured reasoning can bake in these biases, preventing the transfer of logical patterns between disparate tasks. Similarly, a single-author study by K. M. Annervaz notes that while transfer learning leverages pre-trained features, the resulting knowledge remains narrow and lacks the structured, general-purpose world knowledge required to navigate diverse logical scenarios.

The Walled Garden of Verifiers and Fine-Tuning

A popular paradigm for scaling reasoning capabilities is Reinforcement Learning with Verifiable Rewards (RLVR) or training on specialized process-supervision verifiers. While this approach yields high performance within a target domain, it often exacerbates the generalization gap.

The authors of Learning to Reason without External Rewards explain that training models on domain-specific verifiers forces them to optimize for the specific constraints of the verifier rather than broad logical principles. This makes it incredibly difficult to transfer reasoning from a script-verifiable domain (like mathematics) to an unscripted domain (like computer programming or natural language argument).

Furthermore, task-specific fine-tuning can actively damage a model's broader cognitive architecture. In On the Impact of Fine-Tuning on Chain-of-Thought Reasoning, the authors reveal that specialized fine-tuning can overwrite fragile, multi-domain reasoning structures, leading to "post-hoc" justifications rather than true step-by-step logic.

A study by Luca M. Schulze Buschoff on vision-language models confirms this brittleness: fine-tuning on intuitive physics boosts task-specific scores but fails to transfer to related cognitive domains like causal reasoning. This proves that the model is learning task-specific shortcuts rather than building an integrated, intuitive theory of the world.

De-coupling Logic from Surface Features

To bridge the cross-domain gap, researchers are shifting from teaching models what to say to teaching them how to plan.

One prominent approach is to decouple high-level strategy from low-level execution. Junzhe Chen et al. introduce a framework called Critical Plan Step Learning (CPL) that shifts the focus of reinforcement learning from task-specific solutions to high-level abstract plans. By focusing on task-agnostic problem-solving strategies, they achieve substantial gains in out-of-domain generalization.

Another avenue is expanding the modalities of thought. Tong Zheng et al. propose Mixture-of-Thought (Mixture-of-Thought for Logical Reasoning), which trains models to reason across natural language, Python code, and symbolic truth tables simultaneously. This multi-format grounding prevents the model from locking into the linguistic or structural quirks of a single domain.

Finally, synthetic data generation is evolving to explicitly target the boundaries of the gap. Rather than generating more of the same data, frameworks like EvolProver, introduced by Yuchen Tian et al., mutate existing problems across different mathematical domains and difficulty levels while keeping the underlying logical skeleton identical. This forces the model to separate abstract logical structures from domain-specific contexts, paving the way for true generalist reasoners.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_domain_reasoning_generalization_cr_fdb4ca55,
  title = {cross-domain reasoning generalization (cross-domain generalization gap)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-domain-reasoning-generalization-cross-domain-generalization-gap/rdr_fdb4ca55fd974d418aea7bc7b9ea60ed},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_domain_reasoning_generalization_cr_fdb4ca55_v1,
  title = {cross-domain reasoning generalization (cross-domain generalization gap)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-domain-reasoning-generalization-cross-domain-generalization-gap/rdr_fdb4ca55fd974d418aea7bc7b9ea60ed/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake