Weight-space programming for tractor guidance controller: large language models natively write neural network weights that steer a real tractor

Timo OksanenarXiv 2026 · arXiv

TL;DR

This paper introduces "Weight-Space Engineering," a paradigm where Large Language Models (LLMs) like GPT-6 and Claude Fable directly read and write the numerical weights of a neural network to design a functional controller. By reasoning about control theory rather than relying on gradient descent, these models successfully programmed a steering system for a real-world tractor, achieving performance comparable to classical engineering methods with a fraction of the data.

What Problem Does This Paper Address?

Traditionally, neural networks are "black boxes" trained via gradient descent—a process that requires millions of simulated trials and results in a dense web of numbers that no human can easily explain. If you move a trained model from one machine to another with slightly different physical properties (like a slower steering actuator), the model usually fails and requires expensive retraining.

The authors ask: Can an LLM bridge the gap between human-readable control engineering and the raw numerical weights of a neural network? Instead of "training" a model, can we ask an LLM to "engineer" its weights by providing it with the plant's physics equations and a few simulation results?

Methods

The researchers developed a process called Weight-Space Programming (WSP). They presented LLMs (GPT-6 Astra and Claude Fable 5.1) with a 253-parameter neural network architecture designed for tractor guidance.

Figure 2: Every weight of a neural network shown as a knob, where (a) training involves a numerical optimiser turning every knob a little from gradients and repeating this millions of times, while (b) weight-space programming involves an LLM reading the knob values as text and writing new values directly without gradients.

Key aspects of their approach include:

  • Weight-Space Engineering (WSE): A systematic framework that provides the LLM with a formal specification, field knowledge (control theory principles), and a small "evaluation budget" (just 25 simulated runs) to verify and refine its designs.
  • Fixed Architecture: The network structure was kept constant (Figure 3), focusing the experiment purely on the LLM's ability to determine the correct values for the 253 weights.
  • Reasoning-First Design: The models were prompted to analyze the system using classical methods like linearization and phase margin analysis before proposing weight changes.
Figure 3: The steering network with 253 parameters and the fixed SE(2) layer, the shared token encoder with 60 parameters and the head with 193 parameters, where the architecture is fixed and only the 253 numbers change. The pose and the path chunks enter only the SE(2) layer, which passes the 4 tokens of 9 features in the vehicle frame to the token encoder.

Datasets

The study did not use traditional large-scale training datasets. Instead, it relied on:

  • Simulated Environments: A kinematic bicycle model of the AMX G-trac tractor.
  • Path Sets: Three fixed sets of paths (lines and arcs) used for feedback (64 paths), selection (256 paths), and final held-out evaluation (512 paths).
  • Real-World Validation: A 4.2-hectare test field in Germany where the final controllers were tested on a physical AMX G-trac tractor.

Key Insights and Ideas

  • LLMs as Control Engineers: The most striking finding was that the models didn't just "guess" numbers. They diagnosed failures in control-theoretic terms (e.g., identifying "wind-up" or "limit cycles") and proposed specific weight edits to fix them.
  • Sparse and Interpretable Weights: Unlike gradient descent, which typically adjusts every weight in a network, the LLMs produced sparse controllers. They often set many weights to zero, creating a "cascaded" structure that mirrored how a human engineer might design a nested control loop.
  • Zero-Shot Repair: In some instances, the models could repair a failing controller purely through reasoning about the weights and the plant equations, without seeing any simulation feedback.
  • Efficiency: LLMs found working solutions within 25 evaluations. In contrast, standard search algorithms (like evolution strategies) failed to find any working controller within the same small budget.

Key Results

The researchers categorized the evolution of tractor guidance into four "Eras," placing LLM-driven design in the newest category.

Table 1 Eras of tractor guidance controllers by who writes the controller and in what form.
  • Simulation Success: Both GPT-6 and Claude Fable successfully repaired unstable networks and synthesized new ones from scratch (all-zero weights).
  • Real-World Steering: All four LLM-designed controllers successfully steered the real AMX G-trac tractor (Figure 1) through a 2.7 km path without human intervention.
  • Quantitative Accuracy: The LLM controllers achieved a mean RMS lateral error between 19.8 cm and 27.8 cm. This is on par with the industry-standard "pure pursuit" algorithm (18.5 cm).
  • Comparison to PPO: While traditional Reinforcement Learning (PPO) eventually reached higher accuracy (12.6 cm), it required 20 million simulation steps. The LLMs achieved functional performance with only 25 steps.

Limitations and Future Work

  • Network Scale: This study used a "tiny" network of 253 weights. It remains to be seen if LLMs can handle the "curse of dimensionality" when dealing with thousands or millions of parameters.
  • Steady-State Errors: The designed controllers lacked "integral action," meaning they drifted slightly on side slopes. Future prompts could explicitly ask for more robust control structures.
  • Hybrid Approaches: The authors suggest that WSE is best viewed as a "complement" to gradient descent. An LLM could design a sparse, stable "warm start" configuration, which gradient descent then fine-tunes for maximum precision.
  • Safety and Certification: Sparse, readable networks designed by LLMs might be easier to audit and certify for safety-critical applications than dense, black-box models.

Summary updated

Abstract

Weight-space programming is a process where a large language model reads and writes the weights of a neural network directly, guided by a natural language specification, without an optimiser or gradient descent. This study applies it to automatic tractor guidance, where a 253-parameter neural network steers along paths of lines and arcs and must be adapted when the machine changes, such as a slower steering actuator. GPT-6 Astra and Claude Fable 5.1 (also 5.2) edited the network for our research tractor using a small budget of simulated evaluations to repair a network trained for another tractor and write a controller into a network with zero weights. We compared them with an evolution strategy, gradient descent and pure pursuit in simulation and on the real tractor. Both models read the network and analysed the closed loop with control theory correctly, and Fable's repair without simulation succeeded only occasionally. With simulated feedback, both models repaired the trained network in single sessions and wrote a working controller from zero weights, whereas from zero the evolution strategy found none. The controllers written from zero were sparse and in a classical cascaded structure; one model edited the trained network while the other replaced it. Gradient descent reached higher accuracy with far more simulation, and all language model controllers steered the real tractor without intervention, with errors of the same order as pure pursuit. Language models can program the weights of a neural controller directly, much as control engineers design one. We call this Weight-Space Engineering.
BibTeX
@misc{oksanen2026weight,
  author = {Timo Oksanen},
  title = {{Weight-space programming for tractor guidance controller: large language models natively write neural network weights that steer a real tractor}},
  year = {2026},
  eprint = {2611.21958},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21958},
}
Figure