Does Learned Program-State Memory Transfer Across Programming Languages?

When a code model fails on an unfamiliar programming language, we usually report a single lower score and blame "less training data." That hides two different failures. The model may misread the new syntax, so it never correctly registers that x was set to 5. Or it may read the syntax fine but lose track of x after forty more executed statements. This proposal borrows a measurement design from cerebellar neuroscience to separate the two. Mouse experiments split working-memory errors into noise at the moment of encoding and decay of the stored trace over time. We apply the same split to code models that predict program state, using the same algorithms written in Python, Java, C++ and Ruby.

The central test is a transfer experiment. We train a model to predict step-by-step execution traces (which line runs next and what each variable holds) in Python only. We then ask which of its gains carry over to the other three languages. Our hypothesis is that the ability to hold state over many steps is largely language-independent and transfers, while the ability to read state off unfamiliar syntax does not. If this holds, trace supervision only needs to exist for one language, and cheap syntax adaptation handles the rest. If it fails, execution training is not the language-invariant scaffold it is often assumed to be.

Why the cerebellum is relevant, and how far the analogy goes

The neuroscience plays one specific role here: it supplies the measurement method and the question, not the architecture. Deverett et al. used light to silence the output cells of the cerebellar cortex while mice counted air puffs across a delay. They then fit a drift-diffusion model, a model of evidence adding up over time, which splits errors into sensory encoding noise, accumulator noise, memory leak and lapses. Silencing mostly increased leak: the memory time constant fell from about 6.7 s to 0.72 s. Crucially, "guided" trials that needed no accumulation were unaffected. That control is what let the authors say the deficit was in maintaining memory rather than in perceiving the puffs.

The research file flags the open neuroscience question directly: whether the same uniform cerebellar circuit supports working memory across other kinds of input, such as auditory, linguistic or symbolic. Across programming languages, the computation is the same and the surface form changes. That is the structure of the cross-language code problem, and it makes the same kind of question answerable in models. We make no claim that code models resemble the cerebellum.

Flierman et al. is a caution we adopt. Their two monkeys solved the same task with visibly different strategies and response timing. So we fit the decomposition separately for each model and each training seed rather than averaging them.

Core Claim

For a query about a variable's value at some point in execution, let $d$ be the number of executed statements since that variable was last written, and $k$ the number of times it has been updated. We model the probability of an exact-match correct answer as

$$ p(d,k) = a\,(1-\epsilon)^{k}\,e^{-d/\tau}, $$

where the three parameters mean the following. The parameter $a$ is encoding accuracy, measured at zero delay and zero updates. The parameter $\epsilon$ is the per-update error, the analogue of accumulator noise. The parameter $\tau$ is the retention constant in executed steps, the analogue of the leak time constant. The exponential form mirrors the leaky memory in Deverett's model. We will also test a power-law decay and keep whichever fits held-out data better.

We fit $(a, \epsilon, \tau)$ by maximum likelihood per language and per model. Deverett's guided-trial control becomes "direct" queries, where the variable was assigned a literal on the immediately preceding line. These queries measure pure syntax reading.

The claim is falsifiable in both directions. After Python-only trace training, the relative gain in $\tau$ in Java, C++ and Ruby should be close to the gain in Python. By contrast, $a$ in those languages should improve much less than in Python.

Research Program

Data. We use problems from CodeNet, the collection of 14 million competitive-programming solutions used by Szafraniec et al.. CodeNet has accepted solutions to the same problem in many languages together with test inputs. From it we build a matched evaluation set: problems that have accepted solutions in all four languages.

We generate ground-truth traces by running each solution under a language-native tracer: Python's settrace, a JVM debugger interface, gdb for C++, and Ruby's TracePoint. We sample queries stratified over $(d, k)$ so that the curve is identified, including long delays well past the lengths where CodeExecutor is reported to hallucinate in nested loops. Trace formats follow CodeExecutor's variable: value convention.

Models and training. Our base is an open code model of roughly 1B parameters. We also run a check on StarCoder-15.5B, the model RePair fine-tuned, to test whether the findings hold at scale. Python trace training reuses the CodeExecutor recipe on CodeNetMut (2.8M mutated Python programs with traces). We extend the context well beyond that paper's 1024-token limit so that long-delay queries fit in the input.

Diagnostic tools. We add a linear probe for variable values on the hidden states. The probe tells two cases apart: information that has actually decayed from the representation, and information that is still present but not read out.

Downstream payoff. We then test whether the transferred retention helps a real cross-language task. That task is CodeNet-based translation scored by computational accuracy, meaning the fraction of translations that pass the tests.

Baselines

The naive reference point is the base model with no trace training, fit with the same three-parameter curve on the same query set. Every gain is measured against its per-language $(a, \epsilon, \tau)$.

The strongest competitor is multilingual trace training: CodeExecutor-style training with traces in all four languages. For fairness it uses the same total number of trace tokens, the same compute and the same hyperparameter search budget as the Python-only run. The strongest reported number for this style of method is CodeExecutor's 48.06% output accuracy on CodeNetMut, against 17.45% for Codex (Liu et al.). We expect multilingual training to win on raw accuracy. Beating it therefore does not mean higher accuracy. It means Python-only training recovers at least 80% of multilingual training's $\tau$ gain in each target language, and the remaining accuracy gap is explained mainly by $a$.

Helper-language selection is a cheaper transfer strategy. Chen et al. found that curated helper languages beat full multilingual fine-tuning by 0.6% to 41.7% BLEU on Ruby summarization, and that Python was the best single helper for Ruby. We run it here as continued pretraining on Ruby plus Python code without traces, under matched compute.

Two ablations isolate the key ingredient, which is supervision on intermediate states.

  • Output-only. The same Python programs are trained to predict only the final output, not the intermediate states.
  • Plain continued pretraining. Python continued pretraining on the same tokens with no traces at all.

If output-only matches full trace training on $\tau$, then forward-modelling intermediate states is not the ingredient that matters.

Downstream translation reference. The file's strongest translation result is the IR-grounded model of Szafraniec et al., at 47.8% average computational accuracy with beam search across its language pairs. For CodeRosetta, which targets C++-to-CUDA and Fortran rather than our pairs, no directly comparable result was found on Lacuna. For Ruby as a translation target, no reported result was found on Lacuna.

Why not just…?

Why not just train trace prediction on every language? This is a real threat for anyone who only wants accuracy, and we include it as the main baseline rather than dismissing it. It does not, however, answer which capacity transfers. That answer matters because building tracers is slow per language and impossible for some runtime environments. The decomposition shows whether multilingual training buys retention or merely syntax reading. If Python-only training already supplies retention, then target languages need only cheap untraced adaptation.

Why not just pivot through compiler intermediate representation? LLVM intermediate representation removes surface syntax, but it excludes Python and Ruby entirely. It also requires code that compiles, and only 3.3% of Rust functions compiled in Szafraniec et al.. Our transfer question concerns exactly the dynamic languages that approach cannot reach.

Why not just execute the code, as execution-guided generation does? EG-CFG reaches 96.6% on MBPP with DeepSeek-V3 by running partial programs at each line. It needs an interpreter, unit tests and heavy per-line latency, and it has only been evaluated in Python. We treat real execution as an oracle upper bound in the translation experiment. Our target is the setting where no sandbox or tests exist.

Why not just use a larger model with step-by-step prompting? Scale may raise $a$ and $\tau$ together. The StarCoder-15.5B check measures this. If scale alone equalizes $\tau$ across languages while trace training does not add to it, we will report that our hypothesis is unnecessary at scale.

Decisive Checks

The first check is whether the three-parameter curve fits at all. On held-out queries it must beat a per-language constant-accuracy model by likelihood. The direct-query control must also behave as the guided trials did: zero-delay queries on real programs should show accuracy matching $a$.

The main result is the transfer ratio of $\tau$ versus $a$. Support for the hypothesis looks like this: the gain in $\tau$ transfers at least 0.8 of its Python-language size to each target language, while $a$ transfers at no more than 0.5. The reverse pattern refutes it. That reverse pattern is a publishable negative in its own right: it would mean execution training teaches language-specific reading, not general state memory.

The linear probe must agree with the behavioral $\tau$. Otherwise the apparent retention gain is just a readout effect.

Finally, in translation, the per-language gain in computational accuracy from Python-only trace training should track the transferred $\tau$ gain across the language pairs. If $\tau$ transfers but translation does not improve, then retention is not the bottleneck for translation. That result would redirect effort toward syntax encoding.