TL;DR
This paper introduces CarbonMonitor-Power, a novel near-real-time global dataset of power generation at hourly to daily scales, covering 37 countries and eight energy source groups. This dataset fills a critical gap for timely analysis of power system dynamics, enabling better understanding of energy transitions and impacts of global events like the COVID-19 pandemic.
What Problem Does This Paper Address?
Accurate and timely monitoring of global power generation is crucial for understanding energy transitions, managing electricity grids, and informing climate policy. However, existing datasets often suffer from significant time lags (months to years) and lack the high temporal resolution (hourly to daily) needed to capture rapid changes. This makes it difficult to analyze the immediate impacts of events like the COVID-19 pandemic, geopolitical shifts, or the integration of variable renewable energy sources. The paper addresses this by creating a dataset that provides near-real-time insights into global power generation.
Methods
The researchers constructed the CarbonMonitor-Power dataset by collecting raw data from national grid operators for 37 countries. They employed a multi-step process:
- Data Acquisition: Raw data was gathered at the highest available temporal resolution (from 5-minute to monthly intervals).
- Data Cleaning and Preprocessing: A robust method involving Interquartile Range (IQR) thresholding was used to identify and filter outliers. Missing values were filled using linear interpolation. Crucially, extreme values were retained if they corresponded to known abrupt social changes (e.g., COVID-19 lockdowns) or natural disasters, acknowledging their impact on power systems.
- Aggregation and Disaggregation: Data from various sources and energy categories were aggregated into eight standardized groups: coal, gas, oil, nuclear, hydro, solar, wind, and other renewables. For countries where data was not directly available at the desired temporal resolution or source granularity, sophisticated disaggregation and imputation techniques were developed, often leveraging lower-frequency reference datasets (e.g., IEA, BP, IRENA) and country-specific models.
Datasets
The primary dataset introduced is CarbonMonitor-Power, which provides daily and hourly power generation data for 37 countries from January 2016 to June 2022.
Reference datasets used for comparison and disaggregation include:
- IEA (International Energy Agency) Monthly Electricity Statistics: Used for disaggregating fossil fuel sources and for comparison.
- BP Statistical Review of World Energy: Used for comparison, particularly for countries not covered by IEA.
- IRENA (International Renewable Energy Agency) Renewable Energy Statistics: Used for disaggregating renewable energy sources and for comparison.
- Ember: Used for comparison.
- National Grid Operators' Data: Specific sources include OpenNEM (Australia), China Electricity Council/CEC/CCTD (China), POSOCO (India), OCCTO (Japan), ENTSO-E (EU27), BMRS/Sheffield Solar (UK), United Power System of Russia (Russia), Eskom (South Africa), EIA (United States), Gobierno De Mexico (Mexico), and Coordinador Eléctrico Nacional (Chile).
Key Insights and Ideas
- The Power of Near-Real-Time Data: The dataset demonstrates how high-frequency data can reveal dynamic patterns in power generation, influenced by daily activities, weekends, holidays, and extreme events.
- COVID-19's Disruption: The data clearly shows that the COVID-19 pandemic caused significant disruptions in power generation in some countries (e.g., China, India), leading to temporary shifts towards lower carbon intensity, while having a minimal impact in others (e.g., Australia).
- Geopolitical Impacts: The study highlights how events like the Russian invasion of Ukraine can lead to rapid shifts in energy mix, such as increased coal reliance in the EU27&UK, underscoring the need for timely data to monitor such changes.
- Data Harmonization Challenges: The paper details the complexities of harmonizing power generation data from diverse sources with varying formats, temporal resolutions, and energy source categorizations, showcasing innovative methods for disaggregation and imputation.
Key Results
- The CarbonMonitor-Power dataset covers 37 countries, representing approximately 70% of global power production and 68% of global power-related CO2 emissions.
- The dataset provides hourly to daily resolution data for eight energy source groups: coal, gas, oil, nuclear, hydro, solar, wind, and other renewables.
- Validation against reference datasets (IEA, BP) shows good agreement for total generation (R² > 0.95), indicating the dataset's reliability at aggregated levels.
- The dataset successfully captures diurnal, weekly, and seasonal patterns, as well as the impact of events like the COVID-19 pandemic and the Ukraine war on energy mix.
Limitations and Future Work
- Spatial Resolution: While the dataset covers 37 countries, it is primarily at the national level, with some sub-national data available. The authors acknowledge that some distributed solar power generation might not be fully captured.
- Data Gaps and Proxy Data: For countries not covered or with missing data, the researchers plan to use proxy data, including fossil fuel consumption, climate reanalysis data, and profiles from neighboring countries with similar characteristics.
- Oil and Other Renewables Correlation: The correlation with reference datasets for oil and other renewables is lower (R² = 0.661 and R² = 0.682, respectively), attributed to methodological differences in disaggregating thermal power and accounting for smaller energy sources.
- Future Work: The team is actively working on extending the dataset's spatial resolution and coverage, incorporating more countries and regions, and improving data accuracy through advanced proxy data methods. They also aim to provide continuously updated data through a web platform.
Summary updated