TL;DR
Can a language model "compile" its understanding of a task directly into the weights of a neural network? This paper explores Weight-Space Programming (WSP), finding that while frontier models can indeed write weights that solve unseen games, they essentially produce "programs in network form"—highly sparse, hand-tuned tensors that are far more fragile than trained networks and fail to benefit from standard reinforcement learning fine-tuning.
What Problem Does This Paper Address?
Training neural networks typically requires thousands or millions of examples. A more efficient alternative might be to have a reasoning agent (like a Large Language Model) analyze the rules of a task and directly "write" the parameters of a small, compact neural agent. This would create an agent that is fast to run, requires no training data, and—being a neural network—is theoretically differentiable and refinable via Reinforcement Learning (RL).
The authors investigate whether current frontier models can perform this "compilation" for complex, interactive tasks like games. They specifically look at whether these written weights generalize to unseen levels and whether the neural format offers any real advantage over simply writing a standard Python program.
Methods
The researchers established a rigorous protocol to test "WSP" across several conditions:
- The Writers: Nine frontier models (including GPT-6 and Claude 5.5 variants) acted as "writers." They were given the "mechanics-only" source code of a game—the simulator logic—but no information about specific levels or strategies.
- The Tasks: Writers had to produce an agent to play a game. They were tested in three output formats:
- Fixed Actor: Setting the weights for a predefined 57k-parameter recurrent convolutional network.
- Free Architecture (FreeArch): Designing a custom network topology and its weights using a restricted tensor language.
- Python Player: Writing a standard Python script (serving as a performance ceiling).
- The Comparison: Written agents were compared against "Learners"—the same fixed-actor architecture trained from scratch using standard methods like DAgger or PPO on a small set of "support levels."
Datasets
The study primarily used 20 privately authored game families designed in the style of ARC-AGI-3. Because these games were never published, the models could not rely on memorized solutions. Each family included multiple levels and various "renderer templates" that changed the visual appearance (colors, shapes) without altering the underlying mechanics. They also reported results on 25 public ARC-AGI-3 games as a secondary, potentially familiar tier.
Key Insights and Ideas
- Programs in Network Form: On close inspection, the "neural networks" written by models weren't like the dense, noisy weight matrices produced by gradient descent. Instead, they were incredibly sparse: 93% of parameters were exactly zero, and the rest were mostly small integers or powers of two. The models were effectively writing code using tensors as their programming language.
- Appearance Invariance: Unlike trained networks, which often overfit to specific pixel values, written weights generalized almost perfectly to new visual renderings. Because the models "understood" the rules, they built agents that looked for shapes and patterns rather than specific color IDs.
- The "Neural Benefit" is a Myth (for now): One of the main hopes for WSP is that written weights provide a great starting point for RL. However, the authors found that standard PPO fine-tuning actually destroyed the performance of written agents. The written weights were so "brittle" that even the first small update from an RL algorithm removed most of their success.
- Architecture Matters: Models performed significantly better when allowed to design their own network (FreeArch) rather than using a fixed one. This suggests that the difficulty in WSP isn't just calculating weights, but mapping logic onto a specific, rigid computational graph.
Key Results
- Performance Gap: In the fixed-actor setting, the best writer (GPT-6 Astra) solved 56% of held-out episodes. In contrast, the same model's Python programs solved 100%.
- Free Architecture Gain: When allowed to design the network, success jumped to 68–90% for the strongest models.
- Writers vs. Learners: Learning the weights from examples transferred poorly, with a maximum success of only 11% on held-out levels. The writers' access to the source code provided a massive informational advantage.
- Difficulty Scaling: As games got harder, the performance of written weights plummeted, while Python programs remained near-perfect.
Limitations and Future Work
The authors acknowledge several limitations:
- Binary Success: The games used have a "win/loss" outcome per level, which provides a sparse reward signal that is notoriously difficult for RL fine-tuning.
- Source Code Reliance: The writers were given the game's source code. Future work could explore if models can write weights after discovering rules through interaction, rather than reading them.
- Program-like Tasks: The tasks were logical and grid-based. WSP might behave differently on "statistical" tasks (like image classification) where a symbolic, program-like solution is less obvious.
Ultimately, the paper concludes that while LMs can "compile" rules into networks, we are currently better off just letting them write Python. The dream of a "differentiable reasoning agent" that combines the best of symbolic logic and neural learning remains elusive.
Summary updated