Carbon Monitor-Power: near-real-time monitoring of global power generation on hourly to daily scales

Biqing Zhu, Xuanren Song, Zhu Deng, Wenli Zhao, Da Huo, Taochun Sun, Piyu Ke, Duo Cui, Chenxi Lu, Haiwang Zhong, Chaopeng Hong, Jian Qiu, Steven J. Davis, Pierre Gentine, Philippe Ciais, Zhu LiuarXiv 2022 · DOI · arXiv

Abstract

We constructed a frequently updated, near-real-time global power generation dataset: Carbon Monitor-Power since January, 2016 at national levels with near-global coverage and hourly-to-daily time resolution. The data presented here are collected from 37 countries across all continents for eight source groups, including three types of fossil sources (coal, gas, and oil), nuclear energy and four groups of renewable energy sources (solar energy, wind energy, hydro energy and other renewables including biomass, geothermal, etc.). The global near-real-time power dataset shows the dynamics of the global power system, including its hourly, daily, weekly and seasonal patterns as influenced by daily periodical activities, weekends, seasonal cycles, regular and irregular events (i.e., holidays) and extreme events (i.e., the COVID-19 pandemic). The Carbon Monitor-Power dataset reveals that the COVID-19 pandemic caused strong disruptions in some countries (i.e., China and India), leading to a temporary or long-lasting shift to low carbon intensity, while it had only little impact in some other countries (i.e., Australia). This dataset offers a large range of opportunities for power-related scientific research and policy-making.

Introduction

In the fight against climate change, effective management requires precise measurement. While organizations like the International Energy Agency (IEA) and BP provide consolidated energy data, their reports typically suffer from a time lag of months or even years. Furthermore, most global datasets offer only monthly or annual snapshots. This lack of "high-frequency" data creates a blind spot for policymakers and researchers trying to understand how power systems respond to rapid changes, such as extreme weather events, sudden economic shifts, or social changes like the COVID-19 lockdowns.

The Carbon Monitor-Power project addresses this "latency gap" by introducing the first near-real-time global dataset for power generation. It provides hourly and daily resolution across 37 countries, allowing researchers to track the pulse of the global energy transition and the hour-by-hour intermittency of solar and wind power.

Methods

The researchers constructed a harmonized framework to collect, clean, and standardize data from 12 major regions covering 37 countries. The core workflow involves four primary stages:

  1. Data Acquisition: Raw data is pulled from national grid operators (e.g., ENTSO-E for Europe, EIA for the US, and POSOCO for India) at the highest available resolution—often 5-minute or hourly intervals.
  2. Quality Control: An Interquartile Range (IQR) threshold method is used to automatically detect outliers. Manual overrides ensure that "real" extreme events, such as the sudden drop in power during pandemic lockdowns, are preserved.
  3. Country-Specific Disaggregation: Since reporting formats vary by country, specific algorithms were developed to "unbundle" energy categories. For instance, if a country only reports "Thermal" power, monthly factors from the IEA or BP are used to statistically estimate the split between coal, gas, and oil.
  4. Harmonization: All data is mapped to eight standardized source groups: coal, gas, oil, nuclear, hydro, solar, wind, and "other renewables" (biomass, geothermal, etc.).
Summary of the Carbon Monitor-Power dataset characteristics and measurement scope.

Key Results

The Carbon Monitor-Power dataset includes over 2.4 million hourly records and 4,000+ daily records. Key findings from the validation and analysis include:

  • Near-Global Coverage: The 37 countries tracked represent approximately 70% of global power production and 68% of global power-related CO2 emissions.
  • High Accuracy: When compared against traditional annual datasets, Carbon Monitor-Power showed a correlation coefficient (R²) of > 0.95 for total generation. The model performs exceptionally well for major sources like coal, gas, and nuclear, though it faces higher uncertainty in the "oil" and "other renewables" categories (R² ~ 0.66–0.68).
  • Low Latency: The dataset is updated with a lag of just 1 day to 1 month, a significant improvement over the 1-year+ lag common in official census-based statistics.

Discussion

The dataset vividly illustrates how the global power system responds to external shocks. During the COVID-19 pandemic, China and India saw massive disruptions and a temporary shift toward lower carbon intensity, while Australia’s power mix remained relatively stable. High-frequency data also reveals the "Thanksgiving Friday" effect in the US—a massive, predictable decline in power generation that lower-resolution data would smooth over.

Furthermore, the data captures "Solar Noon," showing how solar and renewables dominate the energy mix during midday hours. This provides a granular look at the actual carbon intensity of a country's grid at any given moment, which is critical for managing grids increasingly dependent on variable renewable energy.

Limitations and Future Work

Despite its high resolution, the dataset has limitations. It relies primarily on national grid operators, which may underestimate "off-grid" power generation, such as small-scale rooftop solar or private industrial generators. Additionally, there is a systematic overestimation of oil-fired power in China due to the method of subtracting coal and gas from total thermal power, which can aggregate non-fossil thermal sources into the oil category. Future work will focus on refining these disaggregation factors and expanding the number of countries covered to move closer to 100% global coverage.

Summary updated

Abstract

We constructed a frequently updated, near-real-time global power generation dataset: Carbon Monitor-Power since January, 2016 at national levels with near-global coverage and hourly-to-daily time resolution. The data presented here are collected from 37 countries across all continents for eight source groups, including three types of fossil sources (coal, gas, and oil), nuclear energy and four groups of renewable energy sources (solar energy, wind energy, hydro energy and other renewables including biomass, geothermal, etc.). The global near-real-time power dataset shows the dynamics of the global power system, including its hourly, daily, weekly and seasonal patterns as influenced by daily periodical activities, weekends, seasonal cycles, regular and irregular events (i.e., holidays) and extreme events (i.e., the COVID-19 pandemic). The Carbon Monitor-Power dataset reveals that the COVID-19 pandemic caused strong disruptions in some countries (i.e., China and India), leading to a temporary or long-lasting shift to low carbon intensity, while it had only little impact in some other countries (i.e., Australia). This dataset offers a large range of opportunities for power-related scientific research and policy-making.
BibTeX
@misc{zhu2022carbon,
  author = {Biqing Zhu and Xuanren Song and Zhu Deng and Wenli Zhao and Da Huo and Taochun Sun and Piyu Ke and Duo Cui and Chenxi Lu and Haiwang Zhong and Chaopeng Hong and Jian Qiu and Steven J. Davis and Pierre Gentine and Philippe Ciais and Zhu Liu},
  title = {{Carbon Monitor-Power: near-real-time monitoring of global power generation on hourly to daily scales}},
  year = {2022},
  eprint = {2209.06086},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.06086},
}
Figure