Measuring the environmental impact of delivering AI at Google Scale

Cooper Elsworth, Keguo Huang, David A. Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James ManyikaarXiv 2025 · DOI · arXiv

Abstract

While the environmental costs of training large language models (LLMs) are increasingly documented, the impact of large-scale inference remains poorly understood. This paper presents a comprehensive, first-party analysis of Google’s Gemini AI assistant. By moving beyond chip-level power measurements to a "full-stack" methodology, the authors demonstrate that a median text prompt consumes 0.24 Wh of energy and 0.26 mL of water. These findings underscore the importance of accounting for host systems, idle capacity, and data center overhead, revealing a 44x reduction in carbon footprint achieved through integrated optimizations over a single year.

Introduction: The "Black Box" of AI Inference

The environmental footprint of serving AI at a global scale has historically been a "black box." Existing third-party estimates often vary by an order of magnitude because they rely on high-level assumptions rather than empirical production data. A primary source of this discrepancy is the "measurement boundary." Most studies focus solely on the power draw of the AI accelerator chip (e.g., TPU or GPU), neglecting the broader infrastructure required for reliable, low-latency production services.

Methodology: A Comprehensive Measurement Approach

To address these gaps, the authors propose a Comprehensive Approach that tracks four distinct energy components:

  1. Active AI Accelerator Energy: Power used by TPUs/GPUs during prompt processing.
  2. Active CPU & DRAM Energy: Energy required by the host system to support accelerators.
  3. Idle Machine Energy: Energy consumed by reserved capacity maintained to handle traffic spikes and ensure reliability.
  4. Overhead Energy: Data center-level energy for cooling and power conversion, integrated via Power Usage Effectiveness (PUE).
Comparison of measurement boundaries between existing research and the proposed comprehensive approach.

The study utilizes internal production telemetry from Google’s global AI infrastructure serving Gemini between May 2024 and May 2025. Efficiency is benchmarked against model capability using LMArena scores and compared with public benchmarks like ML.ENERGY to highlight how production data differs from lab-based measurements.

Key Results: Energy, Water, and Emissions

The analysis reveals that the median Gemini Apps text prompt is significantly more efficient than previously estimated:

  • Energy Consumption: 0.24 Wh per median prompt.
  • Water Usage: 0.26 mL per median prompt (equivalent to approximately five drops).
  • Carbon Emissions: 0.03 gCO2e per prompt.

A critical finding is the "Full Stack Gap": measuring only the accelerator chip misses more than half of the actual energy footprint. The comprehensive measurement is 2.4x higher than the narrow chip-only view, highlighting the material impact of host systems and idle capacity.

Breakdown of total energy consumption per prompt, showing the contribution of accelerators, host systems, idle machines, and overhead.

Compounding Efficiency and Progress

The study documents a massive 33x reduction in energy consumption and a 44x reduction in total emissions per median prompt over a twelve-month period. This progress was driven by compounding efficiencies across the stack, including model architecture transitions (e.g., Mixture-of-Experts), software optimizations (distillation and quantization), and the use of custom hardware like TPUs paired with clean energy procurement.

Discussion and Limitations

The results demonstrate that production systems can be far more efficient than lab-based benchmarks suggest. For instance, while benchmarks often use a "batch size of 1" for comparability, production systems utilize large batches to maximize hardware utilization.

However, several limitations remain. The study excludes energy used by user devices and internet networking. Furthermore, the results are specific to Google’s highly optimized vertical stack; providers using off-the-shelf hardware or less efficient data centers may experience different footprints. Future work will aim to include multi-modal prompts and refine Scope 3 (embodied emissions) measurements for hardware.

Summary updated

Abstract

The transformative power of AI is undeniable - but as user adoption accelerates, so does the need to understand and mitigate the environmental impact of AI serving. However, no studies have measured AI serving environmental metrics in a production environment. This paper addresses this gap by proposing and executing a comprehensive methodology for measuring the energy usage, carbon emissions, and water consumption of AI inference workloads in a large-scale, AI production environment. Our approach accounts for the full stack of AI serving infrastructure - including active AI accelerator power, host system energy, idle machine capacity, and data center energy overhead. Through detailed instrumentation of Google's AI infrastructure for serving the Gemini AI assistant, we find the median Gemini Apps text prompt consumes 0.24 Wh of energy - a figure substantially lower than many public estimates. We also show that Google's software efficiency efforts and clean energy procurement have driven a 33x reduction in energy consumption and a 44x reduction in carbon footprint for the median Gemini Apps text prompt over one year. We identify that the median Gemini Apps text prompt uses less energy than watching nine seconds of television (0.24 Wh) and consumes the equivalent of five drops of water (0.26 mL). While these impacts are low compared to other daily activities, reducing the environmental impact of AI serving continues to warrant important attention. Towards this objective, we propose that a comprehensive measurement of AI serving environmental metrics is critical for accurately comparing models, and to properly incentivize efficiency gains across the full AI serving stack.
BibTeX
@misc{elsworth2025measuring,
  author = {Cooper Elsworth and Keguo Huang and David A. Patterson and Ian Schneider and Robert Sedivy and Savannah Goodman and Ben Townsend and Parthasarathy Ranganathan and Jeff Dean and Amin Vahdat and Ben Gomes and James Manyika},
  title = {{Measuring the environmental impact of delivering AI at Google Scale}},
  year = {2025},
  eprint = {2508.15734},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.15734},
}
Figure