Same Fix, Different Weights: Replicating Weight-Space Programming with Claude

Subham KunduarXiv 2026 · arXiv

This paper, "Same Fix, Different Weights," presents a rigorous replication of a provocative 2026 study on "weight-space programming"—the ability of large language models (LLMs) to read, repair, and write neural network weights directly as raw text. Using Claude Opus 5.5, author Subham Kundu investigates whether frontier models truly "understand" the internal logic of weights or if they are simply leveraging architectural priors and memorized heuristics.

TL;DR

Claude Opus 5.5 successfully replicates "weight-space programming" by identifying transformer circuits and "fixing" RL policies, but it often does so by grafting hardcoded textbook controllers into the weights rather than repairing learned features. Furthermore, a significant portion of the reported performance gains in language model editing was found to be a side effect of weight-based temperature scaling rather than surgical logic edits.

What Problem Does This Paper Address?

The field of mechanistic interpretability usually understands neural networks by "running" them—probing activations and intervening on neurons. "Weight-space programming" is a more radical approach: giving an LLM the raw floating-point numbers of a network's weights and asking it to explain or modify them.

The original study by Takehi et al. (2026) claimed that GPT-6 Astra could perform these tasks with high proficiency. Kundu’s replication aims to determine if these capabilities are robust across different models and environments, and to what extent they rely on "priors" (guessing based on common dataset shapes) versus actual "reading" of the weight values.

Methods

Kundu conducted ten pre-registered experiments using Claude Opus 5.5. The methodology emphasized rigor through:

  • Pre-registered Grading: Rules were fixed before any API calls were made to prevent post-hoc data fitting.
  • Controls: The study used "untrained" networks (randomly initialized) and "hidden-unit permutations" (shuffling neurons while keeping the function identical) to see if Claude was just recognizing shapes or actually reading values.
  • Serialized Weights: Weights were provided as text, often using a specialized integer format to save on token costs.
  • Sandboxed Execution: For complex tasks, Claude was given a Python sandbox to analyze weight arrays using NumPy, though it was forbidden from actually running the subject networks on inputs.
Figure 1: How each experiment runs. Subject networks are trained by me; Claude sees only their weights and the interface description. Grading rules for every experiment were written before its first API call.

Datasets

The study utilized a diverse array of subject networks:

  • Reinforcement Learning: PPO policies trained on Atari (ALE) and Gymnasium (CartPole, MountainCar, LunarLander, Acrobot).
  • Generative Models: Small MLP-based diffusion denoisers (DDPMs).
  • Language Models: Custom character-level transformers, Gemma 3 270M, and Qwen3.5-0.8B.
  • Games: A 9x9 Go engine.

Key Insights and Ideas

  • Structural Recognition vs. Data Inference: Claude is remarkably good at identifying structure (e.g., finding induction heads in a transformer or identifying which RAM bytes a policy is sensitive to). However, it fails at inferring training data, often defaulting to common priors like "English" for any language model or "MNIST" for any small image model.
  • Grafting vs. Repairing: When asked to "fix" a failing policy (like a car that can't climb a hill), Claude didn't debug the existing learned logic. Instead, it zeroed out the original weights and "grafted" a hardcoded textbook controller (e.g., a simple velocity-based heuristic) into a few neurons.
  • The Temperature Confound: In the most impressive result—raising a model's math score—Kundu found that a large chunk of the gain came from simply scaling the final layer's norm. This is functionally equivalent to changing the sampling temperature in weight-space, which helped the model break out of repetitive "thinking loops."
  • Memorized Interfaces: Claude’s success in Atari tasks relied heavily on its ability to recall published RAM annotations (e.g., knowing that byte 99 in Breakout is the ball's x-coordinate), rather than discovering the ball's position from the weights alone.

Key Results

  • Reading: Claude correctly named 3 out of 4 Atari games from weights alone and identified the exact induction heads in 15/15 transformer runs.
  • Fixing: One-shot weight edits took a failing MountainCar policy from 0% to 100% success and a hovering LunarLander to 85.6% success.
  • Writing: Without any training data, Claude wrote a 1.48M-parameter Go network that won ~90% of games against PPO-trained opponents.
  • Editing: Claude raised Qwen3.5-0.8B’s MATH-500 score from 6.7% to 58.1%. While the original study attributed this to removing a "looping direction," Kundu showed that scaling the output norm (temperature control) accounted for about a third of the math gain and nearly all of the IFEval (instruction following) gain.
Table 4: Token cost of one 128→64→64→18 MLP (13,586 weights) in four text formats, measured with count_tokens on claude-opus-5-5.

Limitations and Future Work

The author notes several limitations, including small sample sizes (3-5 attempts per network) and the use of relatively weak PPO baselines, which might make "beating the baseline" an overly easy target. The study also highlights that the "permutation control" was too easy for a model that reads weights row-by-row.

Future work should focus on "shape-neutral" tests where the network architecture doesn't immediately give away the task, and more sophisticated controls to distinguish between "grafting" known algorithms and truly "repairing" learned representations.

Summary updated

Abstract

Takehi et al. (2026) report that GPT-6 Astra, given the weights of small neural networks as text, can say what they were trained to do, repair them, and write new ones without training. I re-ran the core of that study with Claude Opus 5.5 in ten pre-registered experiments, with untrained-network and permutation controls and hand audits of every sandboxed code cell. Reading was mixed: Claude named 3 of 4 Atari games from policy weights and found the measured copying heads in 15/15 character-LM runs, but its answers about training data often followed priors. Unlike the original, it identified a sycophantic Gemma 3 270M finetune from weights alone in 6/6 runs. One-shot weight edits took a failing MountainCar policy from 0% to 100% success and a hovering LunarLander policy from 0% to 85.6%; zero-shot networks beat PPO at 100k steps on 4 of 5 environments, each implementing a textbook controller. With no training or forward passes, Claude wrote a 1.48M-parameter 9x9 Go network that won 85-94% of games against three PPO opponents. Given Qwen3.5-0.8B's weights and 10, 25 or 50 forward passes of 64 practice problems, Claude raised MATH-500 from 6.7% to 51.4%, 55.2% and 58.1% (+51.4 points at 50 passes, 95% CI [47.9, 55.0]; original 48.1%), GSM8K from 11.6% to 68.2% and IFEval from 22.2% to 54.2%. Unlike the original, which removed an internal loop direction, every edit acted at the output: it scaled the final norm, raising the effective sampling temperature, and lowered the logit of "Wait" while raising that of </think>. The norm scaling alone reaches only 20.9-22.1% on MATH-500. On Gemma 3 270M, which rarely loops, no edit helped (-0.8 points), as in the original.
BibTeX
@misc{kundu2026same,
  author = {Subham Kundu},
  title = {{Same Fix, Different Weights: Replicating Weight-Space Programming with Claude}},
  year = {2026},
  eprint = {2611.20953},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20953},
}
Figure