This paper introduces a novel framework for evaluating AI not just by its "intelligence," but by its economic productivity relative to its environmental footprint. By measuring Net Value per Joule (NVJ) and Time Saving per Joule (TSJ), the authors reveal that the most capable models are often not the most energy-efficient choices for the economy.
TL;DR
The "Joules-to-GDP" framework measures the economic return of AI systems per unit of energy consumed across 44 different occupations. The study finds that raw model capability and energy efficiency are often at odds: while large models like GLM 5.2 produce the highest absolute dollar value, smaller models like Gemma 4 31B can be up to 2.8x more efficient in terms of economic return per joule.
What Problem Does This Paper Address?
As AI integration scales, its energy consumption has become a primary concern for both sustainability and operational costs. However, current benchmarks typically evaluate AI in silos: researchers look at model accuracy, engineers look at inference latency, and economists look at labor replacement.
There is a missing link: What is the actual economic "bang for the buck" when the "buck" is a joule of energy? Without this, we cannot determine if a high-performing, energy-hungry model is actually a net positive for the economy compared to a faster, leaner one. This paper bridges that gap by formalizing economic output (in dollars and hours saved) as a function of end-to-end energy consumption.
Methods
The authors define two primary metrics:
- Net Value per Joule (NVJ): The (Dollar value of the output - AI execution cost) divided by the energy used.
- Time Saving per Joule (TSJ): The (Human expert completion time - AI completion time) divided by the energy used.
To calculate these, they used an LLM-as-a-Judge (Gemini 3.1 Flash-Lite) to perform blinded pairwise comparisons between AI outputs and human expert references. If the AI wins or ties, it "earns" the dollar value associated with that task (calculated via the occupation's median wage).
The study also performed end-to-end energy accounting, measuring the power draw of the GPU, CPU, and DRAM during the entire execution window, including tool use and waits.
Datasets
- GDPval: A public benchmark of 220 professional tasks across 44 occupations (e.g., CAD design, software feature building, travel itineraries).
- Joules-to-GDP Extension: The authors introduced 83 new tasks created by Snorkel AI, expanding the scope to 80 distinct occupation codes, including more demanding workflows like bioinformatics and environmental monitoring.
Key Insights and Ideas
- The Efficiency-Capability Trade-off: High absolute value does not equal high efficiency. For example, engineering tasks (like CAD) deliver 10% more dollar value per task than software development, but software development is 27% more efficient per joule.
- The "Gemma" Effect: Smaller, optimized models can dominate efficiency rankings. Gemma 4 31B achieved 56% of the value of the top-performing GLM 5.2 while using only 23% of the energy.
- Hardware is a Force Multiplier: The choice of accelerator is as important as the model. Running the same model on an NVIDIA B200 vs. an H100 can change the economic return per joule by a factor of four.
- Parallelism's Hidden Cost: Moving from one H100 to eight H100s reduces completion time by 22%, but it consumes 3.24 times more energy. In an energy-constrained world, "faster" can be significantly "worse."
Key Results
- Model Performance: GLM 5.2 was the "value leader," producing $72.27 of net value per task. However, Gemma 4 31B was the "efficiency leader," with 2.5x the NVJ of GLM 5.2.
- Energy Equivalency: The energy GLM 5.2 uses for a single task (0.430 MJ) is roughly equivalent to four hours of active laptop use.
- The "Expert-with-AI" Bar: When comparing AI against a human expert who is already using AI tools, the AI's NVJ drops by 70%. This suggests that the "low-hanging fruit" for AI value is in assisting humans, while full autonomous replacement faces a much higher efficiency bar.
Limitations and Future Work
- Proxy Metrics: The study uses median wages as a proxy for GDP contribution, which may not capture the full complexity of economic value.
- System Overhead: The current energy measurements cover the server components (GPU/CPU/RAM) but do not yet include facility-level overhead like cooling or power distribution losses.
- Human Review: The framework assumes the AI output is "final." In reality, many tasks require human-in-the-loop verification, which adds a time and dollar cost not fully captured here.
- Future Directions: The authors suggest using NVJ and TSJ as training signals to optimize models for economic efficiency rather than just next-token prediction accuracy.
Summary updated