TL;DR
This paper introduces the ENERTALK dataset, a high-resolution (15 Hz) collection of aggregate and per-appliance electricity consumption data from 22 Korean houses. This dataset addresses limitations in existing datasets, such as low sampling rates and limited house counts, enabling more advanced research in energy disaggregation and consumption pattern analysis.
What Problem Does This Paper Address?
The widespread adoption of Advanced Metering Infrastructure (AMI) has opened doors to richer energy consumption data, fueling advancements in areas like energy disaggregation (identifying individual appliance usage from total consumption), pattern analysis, demand response, and user segmentation. However, existing publicly available datasets often suffer from low sampling rates (especially for individual appliances) and a limited number of monitored houses. These limitations hinder the development and validation of sophisticated analytical models. For instance, low sampling rates can obscure the subtle electrical signatures of appliances, making it difficult to distinguish them. Similarly, a small number of houses may not capture the diversity of consumption behaviors across different households. The ENERTALK dataset aims to bridge this gap by providing a high-frequency, multi-house dataset specifically from Korea, a region with unique appliance characteristics (like kimchi refrigerators).
Methods
The ENERTALK dataset was collected using off-the-shelf smart meters: ENERTALK for aggregate whole-house consumption and ENERTALK PLUG for individual appliance consumption. These devices were calibrated to ensure an error rate of 1% or below, adhering to the IEC 62053-21 standard. The data collection system involved metering ICs that sampled power signals at 7.8125 kHz, which were then down-sampled to 15 Hz, processed, and stored locally before being transmitted to cloud data collection servers via SSL/TCP. The collected data was then structured and stored in a Hadoop database, finally saved as Parquet files after pre-processing to remove sensitive information. The paper also discusses methods for handling missing data, employing linear interpolation for short gaps (under 30 seconds) and discarding longer gaps. Timestamp alignment between aggregate and appliance-level data was addressed by discretizing milliseconds into 15 bins.
Datasets
The primary contribution of this paper is the introduction of the ENERTALK dataset. This dataset comprises:
- 22 Korean houses: A significant number for appliance-level data.
- Aggregate and per-appliance measurements: Providing a comprehensive view of energy usage.
- 15 Hz sampling rate: The highest among publicly available datasets with both aggregate and per-appliance measurements at the time of publication.
- Appliance coverage: Includes common Korean appliances such as refrigerators, kimchi refrigerators, rice cookers, washing machines, and TVs.
- Data duration: Ranging from 29 to 122 days per house.
The paper also references and compares ENERTALK with other existing NILM (Non-Intrusive Load Monitoring) datasets like REDD, UK-DALE, Dataport, and REFIT, highlighting ENERTALK's advantages in sampling rate and house count.
Key Insights and Ideas
- High-frequency data is crucial for disaggregation: The 15 Hz sampling rate of ENERTALK allows for the visualization of distinct appliance signatures (e.g., oscillations, step changes) that become ambiguous or disappear at lower sampling rates (1 Hz or 1/6 Hz), as demonstrated in Figure 7. This is critical for developing accurate NILM algorithms.
- Regional appliance diversity matters: The inclusion of "kimchi refrigerators" is a unique regional characteristic that adds value to the dataset for understanding diverse energy consumption patterns globally.
- Appliance usage patterns reveal lifestyle: The data allows for detailed analysis of daily and hourly consumption, correlating with typical Korean work schedules and meal times, enabling lifestyle-based user segmentation (Figure 8 and 9).
- "Unknown" consumption highlights data gaps: The presence of "unknown" power consumption (difference between aggregate and sum of appliance usage) underscores the challenge of capturing all appliance loads and the need for robust imputation techniques.
- Appliance signatures vary even within the same type: Figure 6 shows that even refrigerators from different houses exhibit distinct power consumption patterns, emphasizing the need for datasets with multiple instances of each appliance type for generalizable NILM models.
Key Results
- The ENERTALK dataset provides 15 Hz aggregate and per-appliance electricity consumption data for 22 Korean houses, with measurement periods ranging from 29 to 122 days.
- At 15 Hz, distinct power consumption patterns for appliances like TVs, rice cookers, and washing machines are clearly visible, enabling better disaggregation.
- Analysis of the data reveals typical daily consumption patterns that can be clustered to represent different household lifestyles, with some clusters correlating with typical office hours and others with weekend behavior.
- The dataset includes measurements for unique Korean appliances like kimchi refrigerators, contributing to regional diversity in energy consumption research.
Limitations and Future Work
- Missing appliance data: The dataset intentionally excludes heating, cooling, and lighting appliances due to practical reasons (gas heating, measurement period excluding summer, complexity of measuring numerous lighting devices). This means the "unknown" power consumption component can be significant.
- Data collection duration: While some houses have over 100 days of data, others have shorter durations, which might limit long-term trend analysis.
- Imputation necessity: Despite efforts to minimize missing data, imputation methods are still required, and the paper acknowledges that the choice of method can impact results.
- Future work: The authors suggest that the dataset can be used for a variety of research fields beyond energy disaggregation, including user segmentation, consumption pattern analysis, and prediction. The provided tools on GitHub can aid researchers in handling missing values and aligning data for their specific analyses. The high sampling rate and regional specificity of the dataset are highlighted as key assets for future research.
Summary updated