When the Canonical Completion Is Wrong: Formalizing and Measuring the Jump in Large Language Models

Dai Shi, Xiaoyu Li, José Miguel Hernández-LobatoarXiv 2026 · arXiv

TL;DR

This paper provides a rigorous category-theoretic framework to define and test the "abductive jump"—the intellectual leap from known data to an entirely new system of axioms. By evaluating frontier large language models (LLMs) on problems where their natural inductive defaults are explicitly forbidden, the authors demonstrate that modern LLMs are fully capable of overriding these defaults to discover unique, structurally novel solutions.

What Problem Does This Paper Address?

When Einstein formulated general relativity, he didn't just interpolate from existing data; he made an "abductive leap" to a brand-new conceptual framework (curved spacetime) because new observations broke the old Newtonian default. In AI philosophy, a major debate centers on whether LLMs are structurally capable of making these types of creative leaps ("jumps"), or if they are permanently limited to standard induction (pattern matching) and deduction.

The core issue keeping this debate open is the lack of a formal mathematical definition of a "jump" and a clean way to test it. This paper introduces a formal account of the jump using category theory and designs a benchmark to see if LLMs can abandon their default reasoning patterns when constraints demand it.

The historical analogy of a conceptual jump vs. the paper's laboratory test workflow.

Methods

The authors break down a "jump" into four distinct formal steps:

  1. The Default: Determining the natural completion of partial data.
  2. The Override: Forcing the model to abandon that default via constraints.
  3. Correctness: Ensuring the new completion is correct against a hidden ground truth.
  4. Compounding: Tracking how successive jumps build on one another.

This paper specifically focuses on measuring Step 2 (The Override). To do this mathematically, they define the baseline inductive default using Kan extensions—the most parsimonious or permissive completions forced by the data alone.

A jump instance is defined as a data-completion problem where these canonical Kan extensions are explicitly ruled out by constraints. This leaves exactly one unique, valid completion. To solve it, the model cannot simply read patterns off the data; it must "invent" entirely new structural elements that were never present in the initial input.

Datasets

The authors introduce a new synthetic benchmark based on the pointed-chain family ($S(m, \vec{p})$) of finite categories.

  • The benchmark consists of 9 certified instances of varying mathematical difficulty, where the total search space scales up to $10^8$ candidate extensions.
  • To prevent the models from relying on pre-training data or rote memorization, each mathematical instance is translated into a neutral "cover story" using nonce vocabulary (e.g., designing a fictional three-stage signal pipeline with custom rules like "dax", "rell", and "fen").

Key Insights and Ideas

  • Kan Extensions as the Inductive Baseline: The paper establishes that standard error-minimization training naturally biases LLMs toward producing Kan extensions. This provides a mathematically elegant definition of an LLM's "default nature."
  • LLMs Can Pivot Against Their Nature: The most striking insight is that when an LLM's default path is blocked, it doesn't just hallucinate or fail randomly. It can successfully navigate constraints to find a highly specific, non-obvious structural solution.
  • Search, Not Logic, is the Bottleneck: As the problems grow more complex, model failures are driven by exhausted token/reasoning budgets rather than a structural inability to conceptualize the non-canonical answer.

Key Results

The authors evaluated four frontier models: GPT-5.6 Luna Pro, Claude Sonnet 5, Gemini 3.1 Pro, and DeepSeek V4 Pro.

  • The Baseline Check: When tested without constraints, the models returned the canonical Kan extension 98% of the time, proving that Kan extensions are indeed the default behavior of LLMs.
  • The Jump Test: When constraints were applied to exclude the default, the Kan-default rate dropped to exactly 0% across all 248 trials. Instead of falling back on the default, the models successfully generated the unique, non-canonical correct completion.
  • Model Variations: Gemini 3.1 Pro achieved a perfect 1.00 accuracy across all test scales. DeepSeek V4 Pro struggled on the most complex ($m=3$) instances, but its failures were entirely due to reasoning budget truncations rather than a reversion to the default behavior.

Limitations and Future Work

  • Tier 1 vs. Tier 2 Jumps: This benchmark operates at "Tier 1," meaning the constraints and the format of the final answer are explicitly handed to the model in the prompt. A true "Tier 2" jump requires a model to autonomously discover its own constraints from an anomaly, which remains an open challenge.
  • Unmeasured Steps: While Steps 3 (predictive correctness) and 4 (chained jumps) are mathematically formalized in this paper, empirical testing frameworks for them are left for future work.
  • Computational Scaling: Certifying these instances via brute-force enumeration is computationally hard (#P-hard), meaning future benchmark scaling will require more advanced closed-form family theorems.

Summary updated

Abstract

Whether large language models (LLMs) can perform the abductive leap from evidence to a new system of axioms, commonly referred to as a jump, has recently attracted considerable debate. A prominent position holds that LLMs are structurally incapable of such jumps, while recent studies challenge both its mechanism and empirical evidence. One of the main reasons why the debate remains open is the lack of a formal definition of the jump and of algorithms that test whether a jump appears. In this paper, we attempt to develop a formal account of the jump in four steps and measure the second. These steps ask what the default completion of partial data is, when abandoning it is forced, whether the abandonment is correct, and how successive jumps compound. We define a \emph{jump instance} as a finite extension problem constrained by excluding the canonical completion given by the Kan extensions and leave exactly one unique correct completion. Under such an environment, a model offers a jump when it produces the correct completion, and fails to jump when it keeps the canonical default. We tested four frontier models on nine instances, where the Kan-default rate is zero in all 248 constrained trials, suggesting that the models can jump at least in this step. We further formalize the third and fourth steps, where a test of the jump is still wanted. We hope our work paves the path for formalizing and measuring the full jump in the future. The code of the paper is available at https://github.com/EEthanShi/kan-jump-test.
BibTeX
@misc{shi2026when,
  author = {Dai Shi and Xiaoyu Li and José Miguel Hernández-Lobato},
  title = {{When the Canonical Completion Is Wrong: Formalizing and Measuring the Jump in Large Language Models}},
  year = {2026},
  eprint = {2608.26187},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.26187},
}
Figure