The Mental Execution Gap in Code Language Models

112 concepts72 papers9 proposals

Imagine asking a large language model (LLM) to write a Python function that reverses a linked list. It will likely output flawless, syntactically perfect code in milliseconds. Now, take that exact same code, provide a specific input list (such as 1 -> 2 -> 3), and ask the model to trace the exact value of the pointer variables at line 7 during the second iteration of the loop. Suddenly, the model falters.

This paradox defines a major frontier in AI code intelligence: the mental execution gap. While modern models excel at synthesizing code, they struggle immensely with simulating its execution. They can write the instructions for a machine to follow, but they cannot reliably run those same instructions in their own "heads."

The Generative Paradox: Synthesis vs. Comprehension

Intuitively, one might assume that generating a working program requires a deep understanding of how it executes. However, empirical evidence shows that code generation and code comprehension are heavily decoupled.

In a striking demonstration of this asymmetry, Manav Singhal uncovered what can be called a "generative paradox" in NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness: models are frequently capable of generating a correct fix for a buggy piece of code, yet they fail to identify the correct snippet when presented with a simple binary choice between the buggy and corrected versions. This indicates that generative capability does not imply a robust discriminative understanding of code quality.

Similarly, Yuwei Zhao introduced the CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding? benchmark, revealing that models consistently struggle to judge the correctness of existing code or pinpoint execution errors, even when they score highly on generation tasks.

This gap extends to structural understanding. As Weixiang Yan et al. show in CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation, structurally separating generation and understanding tasks reveals that proficiency in writing functional code does not guarantee an equivalent ability to analyze, summarize, or detect defects in existing codebases. The mechanisms for synthesizing syntax appear fundamentally distinct from those required for deep logical comprehension.

Why Does the Gap Exist? Pattern Matching vs. Stateful Simulation

Why can an LLM write code it doesn't understand? The answer lies in how these models are trained. LLMs are next-token predictors optimized on vast corpora of text and source code. They excel at retrieving memorized, high-probability patterns—such as common library API calls or boilerplate algorithmic structures—as noted by Hang Zhang et al. in Comparative Analysis of Large Language Models for Context-Aware Code Completion using SAFIM Framework.

When a model generates code, it often relies on local context and surrounding patterns rather than a deep causal model of the program's logic, a phenomenon explored by Gyeongwon James Kim et al. in From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking. To expose this reliance on surface-level patterns, researchers use counterfactual mutations. For instance, Ashish Hooda et al. demonstrated in Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates that when the "natural" look of code is disrupted—such as by flipping if-else blocks or renaming variables while preserving exact functionality—model performance plummets.

Adrian Swindle et al. observed a similar degradation in Evaluation of Large Language Models on Code Obfuscation (Student Abstract), confirming that models often lack a deep semantic model of program logic and instead rely on probabilistic pattern matching.

True program execution, by contrast, is inherently stateful and discrete. It requires tracking variable mutations, evaluating conditional branches, and maintaining a precise mental stack over time. Standard autoregressive generation lacks an explicit mechanism to reconcile internal logical expectations with the actual step-by-step execution trace of a program, a limitation highlighted by Xueyu Hu et al. in Leveraging Print Debugging to Improve Code Generation in Large Language Models.

Quantifying the Execution Gap

To measure this specific cognitive limitation, recent benchmarks have shifted focus from synthesis to input/output prediction and line-by-line tracing.

The CRUXEval benchmark, developed by Alex Gu et al. in CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution, explicitly evaluates models on their ability to predict the output of simple Python programs given an input, or to deduce the necessary input to achieve a specific output. Their findings confirm that writing code and predicting its behavior are entirely separate capabilities; models fine-tuned to mimic high-quality code style show little to no improvement in execution reasoning.

When forced to trace execution at a granular level, the failure becomes even more pronounced. Claas Beger et al. introduced the CoCoNUT: Structural Code Understanding does not fall out of a tree benchmark to evaluate line-by-line control flow tracing, finding that model performance drops sharply as code complexity and trace length increase. Even on real-world codebases, Monoshi Kumar Roy et al. showed in CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning that frontier models struggle with basic fine-grained reasoning, such as tracking a variable's value across just three lines of code or predicting loop iteration counts.

Expected Logic (Mental Model)  ───?───>  Actual Execution Trace
          │                                         │
          ▼                                         ▼
Probabilistic Syntax Pattern             Discrete State Changes
(e.g., Boilerplate Loops)                (e.g., Variable Mutations)

The Consequences for Complex Software Engineering

The mental execution gap isn't just a theoretical quirk; it imposes a hard ceiling on an LLM's utility in real-world software development.

Bridging the Gap: Teaching Models to Simulate

How can we close the gap between code synthesis and execution reasoning? Researchers are exploring several promising directions:

  1. Externalizing the Trace: If a model cannot simulate execution internally, we can force it to use external tools or explicit scratchpads. Xueyu Hu et al. demonstrated that prompting models to insert print statements and analyze actual execution logs dramatically improves their algorithmic problem-solving.
  2. Execution-Based Pre-training: Instead of training models solely to predict the next token of code syntax, we can train them to predict execution results directly. Qian Liu et al. showed in TAPEX: Table Pre-training via Learning a Neural SQL Executor that pre-training models to act as neural executors (predicting the output of SQL queries over tables) fundamentally improves their underlying structural reasoning.
  3. Structured Logical Planning: Providing models with high-quality logical structures or forcing them to generate explicit thought steps before writing code helps bridge the planning bottleneck, as shown by Yi Cai et al. in Beyond Code: Evaluate Thought Steps for Complex Code Generation.

Ultimately, moving past "rote coding" requires benchmarks and training paradigms that treat code not merely as text to be generated, but as a dynamic process to be executed, tracked, and understood.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_internal_code_simulation_f89b94b3,
  title = {internal code simulation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/internal-code-simulation/rdr_f89b94b338b94fe3804bf0479d419f4e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_internal_code_simulation_f89b94b3_v1,
  title = {internal code simulation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/internal-code-simulation/rdr_f89b94b338b94fe3804bf0479d419f4e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake