Abstract
While the environmental costs of training large language models (LLMs) are increasingly documented, the impact of large-scale inference remains poorly understood. This paper presents a comprehensive, first-party analysis of Google’s Gemini AI assistant. By moving beyond chip-level power measurements to a "full-stack" methodology, the authors demonstrate that a median text prompt consumes 0.24 Wh of energy and 0.26 mL of water. These findings underscore the importance of accounting for host systems, idle capacity, and data center overhead, revealing a 44x reduction in carbon footprint achieved through integrated optimizations over a single year.
Introduction: The "Black Box" of AI Inference
The environmental footprint of serving AI at a global scale has historically been a "black box." Existing third-party estimates often vary by an order of magnitude because they rely on high-level assumptions rather than empirical production data. A primary source of this discrepancy is the "measurement boundary." Most studies focus solely on the power draw of the AI accelerator chip (e.g., TPU or GPU), neglecting the broader infrastructure required for reliable, low-latency production services.
Methodology: A Comprehensive Measurement Approach
To address these gaps, the authors propose a Comprehensive Approach that tracks four distinct energy components:
- Active AI Accelerator Energy: Power used by TPUs/GPUs during prompt processing.
- Active CPU & DRAM Energy: Energy required by the host system to support accelerators.
- Idle Machine Energy: Energy consumed by reserved capacity maintained to handle traffic spikes and ensure reliability.
- Overhead Energy: Data center-level energy for cooling and power conversion, integrated via Power Usage Effectiveness (PUE).
The study utilizes internal production telemetry from Google’s global AI infrastructure serving Gemini between May 2024 and May 2025. Efficiency is benchmarked against model capability using LMArena scores and compared with public benchmarks like ML.ENERGY to highlight how production data differs from lab-based measurements.
Key Results: Energy, Water, and Emissions
The analysis reveals that the median Gemini Apps text prompt is significantly more efficient than previously estimated:
- Energy Consumption: 0.24 Wh per median prompt.
- Water Usage: 0.26 mL per median prompt (equivalent to approximately five drops).
- Carbon Emissions: 0.03 gCO2e per prompt.
A critical finding is the "Full Stack Gap": measuring only the accelerator chip misses more than half of the actual energy footprint. The comprehensive measurement is 2.4x higher than the narrow chip-only view, highlighting the material impact of host systems and idle capacity.
Compounding Efficiency and Progress
The study documents a massive 33x reduction in energy consumption and a 44x reduction in total emissions per median prompt over a twelve-month period. This progress was driven by compounding efficiencies across the stack, including model architecture transitions (e.g., Mixture-of-Experts), software optimizations (distillation and quantization), and the use of custom hardware like TPUs paired with clean energy procurement.
Discussion and Limitations
The results demonstrate that production systems can be far more efficient than lab-based benchmarks suggest. For instance, while benchmarks often use a "batch size of 1" for comparability, production systems utilize large batches to maximize hardware utilization.
However, several limitations remain. The study excludes energy used by user devices and internet networking. Furthermore, the results are specific to Google’s highly optimized vertical stack; providers using off-the-shelf hardware or less efficient data centers may experience different footprints. Future work will aim to include multi-modal prompts and refine Scope 3 (embodied emissions) measurements for hardware.
Summary updated