TL;DR
This paper introduces software thermodynamics, a novel framework that treats LLM tokens as physical fuel to quantify the efficiency of coding agents. By applying principles from stochastic thermodynamics, the authors demonstrate that roughly 80% of tokens in a successful coding task are "heat" (overhead like reading and testing), while only 14-22% represent the "free energy" or minimum work required for the change.
What Problem Does This Paper Address?
As AI coding agents become ubiquitous, we are increasingly "billed by the token," yet software engineering lacks a formal theory to explain what a token actually buys. We measure code by correctness and coverage, but we have no instrument to measure the efficiency of the agentic process itself.
The authors address this gap by drawing a direct analogy between agentic coding and classical thermodynamics. In their model:
- Tokens are the fuel/energy.
- The Repository is the system being acted upon.
- The Test Suite acts as a Maxwell’s Daemon, a gatekeeper that decides which "molecules" (code changes) are allowed to pass into the final state.
The goal is to provide a predictive theory that helps engineers understand where tokens are "wasted" and how to optimize agent harnesses.
Methods
The researchers conducted an experiment inspired by single-molecule biophysics (pulling an RNA hairpin open and letting it refold). They tasked an agent (deepseek-v4.1-flash) with a "cycle" of coding:
- Coding (Forward): Make a specific change to a repository (e.g., renaming a module).
- Uncoding (Reverse): Undo that exact change to return the repository to its original state.
They measured the Work ($W$)—tokens spent on turns that actually moved the "reaction coordinate" (the fraction of target edits in place)—and the Heat ($h$)—tokens spent on turns that left the state unchanged (reading files, searching, running tests).
Crucially, they distinguished between two types of temperature:
- Hinton Temperature ($T_H$): The standard LLM sampling parameter.
- Software Temperature ($1/\beta$): A measured value derived from the variance of token costs across many runs, representing the "exchange rate" between tokens and information.
Datasets
The study used a private Python codebase consisting of several hundred modules. They evaluated two specific tasks:
- Task A: Renaming a module responsible for backgrounding long-running test checks.
- Task B: Renaming a module that guards software deployments.
- Toy System: A simplified environment with a hidden defect used specifically to measure the error rates and "temperature" of the test suite (the gate).
Key Insights and Ideas
- The 80/20 Rule of Tokens: In a successful run, about 80% of tokens are "heat." This isn't necessarily pure waste; it is the thermodynamic price the agent pays to acquire the information it needs to act.
- Coding is Harder than Uncoding: The "asymmetry" of the cycle showed that making a change costs 1.1 to 1.5 times more than undoing it. This suggests that "building" creates more entropy/dissipation than "destroying" in the context of code.
- Cold Agents Get Stuck: At low sampling temperatures ($T_H \le 0.25$), agents frequently fall into "metastable states"—infinite loops where they repeat the same search command, burning tokens without making any edits.
- The "Hot" Test Suite: A flaky test suite acts like a "hot" Maxwell's Daemon. If the gate is "hotter" than the agent (noisier), it cannot effectively extract disorder from the system, leading to "false landings" where broken code is accepted.
Key Results
- Efficiency: The "free energy" (minimum possible cost) of a change was estimated to be only 14% to 22% of the mean cost of a successful run.
- Token Dissipation: On Task A at a Hinton temperature of 0.5, a landed run cost ~2,300 tokens: ~1,900 were heat and ~400 were work. The estimated free energy was only 351 tokens.
- Looping Costs: In cold conditions, unfinished runs (those that got stuck in loops) accounted for 66% of all tokens generated across the entire experiment.
- Gate Temperature: When the test suite was intentionally made "noisy" (error rate of 0.10), its software temperature was 3 to 16 times higher than the model's temperature, significantly degrading its ability to catch defects.
Limitations and Future Work
The authors acknowledge that this is currently an analogy rather than a proven physical law. LLMs have "memory" and do not start from true thermal equilibrium, which complicates the application of theorems like Jarzynski’s equality.
Future directions include: * Testing if "free energy" is a true state function (independent of the path taken to change the code). * Applying the theory to larger, non-reversible changes (e.g., adding new features). * Studying swarms of agents and agents with persistent memory systems. * Developing "thermodynamic" debugging tools that automatically detect when an agent is "overheated" (wasting tokens) or "stuck" in a metastable loop.
Summary updated