This paper, "Same Fix, Different Weights," presents a rigorous replication of a provocative 2026 study on "weight-space programming"—the ability of large language models (LLMs) to read, repair, and write neural network weights directly as raw text. Using Claude Opus 5.5, author Subham Kundu investigates whether frontier models truly "understand" the internal logic of weights or if they are simply leveraging architectural priors and memorized heuristics.
TL;DR
Claude Opus 5.5 successfully replicates "weight-space programming" by identifying transformer circuits and "fixing" RL policies, but it often does so by grafting hardcoded textbook controllers into the weights rather than repairing learned features. Furthermore, a significant portion of the reported performance gains in language model editing was found to be a side effect of weight-based temperature scaling rather than surgical logic edits.
What Problem Does This Paper Address?
The field of mechanistic interpretability usually understands neural networks by "running" them—probing activations and intervening on neurons. "Weight-space programming" is a more radical approach: giving an LLM the raw floating-point numbers of a network's weights and asking it to explain or modify them.
The original study by Takehi et al. (2026) claimed that GPT-6 Astra could perform these tasks with high proficiency. Kundu’s replication aims to determine if these capabilities are robust across different models and environments, and to what extent they rely on "priors" (guessing based on common dataset shapes) versus actual "reading" of the weight values.
Methods
Kundu conducted ten pre-registered experiments using Claude Opus 5.5. The methodology emphasized rigor through:
- Pre-registered Grading: Rules were fixed before any API calls were made to prevent post-hoc data fitting.
- Controls: The study used "untrained" networks (randomly initialized) and "hidden-unit permutations" (shuffling neurons while keeping the function identical) to see if Claude was just recognizing shapes or actually reading values.
- Serialized Weights: Weights were provided as text, often using a specialized integer format to save on token costs.
- Sandboxed Execution: For complex tasks, Claude was given a Python sandbox to analyze weight arrays using NumPy, though it was forbidden from actually running the subject networks on inputs.
Datasets
The study utilized a diverse array of subject networks:
- Reinforcement Learning: PPO policies trained on Atari (ALE) and Gymnasium (CartPole, MountainCar, LunarLander, Acrobot).
- Generative Models: Small MLP-based diffusion denoisers (DDPMs).
- Language Models: Custom character-level transformers, Gemma 3 270M, and Qwen3.5-0.8B.
- Games: A 9x9 Go engine.
Key Insights and Ideas
- Structural Recognition vs. Data Inference: Claude is remarkably good at identifying structure (e.g., finding induction heads in a transformer or identifying which RAM bytes a policy is sensitive to). However, it fails at inferring training data, often defaulting to common priors like "English" for any language model or "MNIST" for any small image model.
- Grafting vs. Repairing: When asked to "fix" a failing policy (like a car that can't climb a hill), Claude didn't debug the existing learned logic. Instead, it zeroed out the original weights and "grafted" a hardcoded textbook controller (e.g., a simple velocity-based heuristic) into a few neurons.
- The Temperature Confound: In the most impressive result—raising a model's math score—Kundu found that a large chunk of the gain came from simply scaling the final layer's norm. This is functionally equivalent to changing the sampling temperature in weight-space, which helped the model break out of repetitive "thinking loops."
- Memorized Interfaces: Claude’s success in Atari tasks relied heavily on its ability to recall published RAM annotations (e.g., knowing that byte 99 in Breakout is the ball's x-coordinate), rather than discovering the ball's position from the weights alone.
Key Results
- Reading: Claude correctly named 3 out of 4 Atari games from weights alone and identified the exact induction heads in 15/15 transformer runs.
- Fixing: One-shot weight edits took a failing MountainCar policy from 0% to 100% success and a hovering LunarLander to 85.6% success.
- Writing: Without any training data, Claude wrote a 1.48M-parameter Go network that won ~90% of games against PPO-trained opponents.
- Editing: Claude raised Qwen3.5-0.8B’s MATH-500 score from 6.7% to 58.1%. While the original study attributed this to removing a "looping direction," Kundu showed that scaling the output norm (temperature control) accounted for about a third of the math gain and nearly all of the IFEval (instruction following) gain.
Limitations and Future Work
The author notes several limitations, including small sample sizes (3-5 attempts per network) and the use of relatively weak PPO baselines, which might make "beating the baseline" an overly easy target. The study also highlights that the "permutation control" was too easy for a model that reads weights row-by-row.
Future work should focus on "shape-neutral" tests where the network architecture doesn't immediately give away the task, and more sophisticated controls to distinguish between "grafting" known algorithms and truly "repairing" learned representations.
Summary updated