TL;DR
This paper introduces a novel three-stage method for identifying outliers in wind power data, combining physical rules, regression learning, and mathematical morphology. This approach significantly improves the accuracy and robustness of outlier detection, leading to better wind power prediction and overall data quality enhancement.
What Problem Does This Paper Address?
Wind power is a crucial renewable energy source, but its intermittent and volatile nature poses challenges for power system stability. Accurate prediction of wind power output is essential, and this relies heavily on the quality of operational data collected by SCADA systems. However, these systems often suffer from sensor malfunctions, data transmission errors, and extreme operating conditions, leading to contaminated data with outliers. These outliers, especially densely stacked ones that are close to normal data, can significantly degrade the reliability of data-driven applications like wind power prediction. Existing methods often struggle with these complex outlier distributions or require extensive manual parameter tuning.
Methods
The proposed solution is a three-stage composite outlier identification method:
- Physical Rule-Based Preprocessing: This initial stage filters out obvious outliers that violate fundamental physical principles of wind turbine operation. Rules are applied to identify data points with negative wind speed or power, or zero power output when wind speed is above the cut-in speed, and vice-versa. Missing and duplicate data are also identified here.
- Regression Learning-Enabled Detection: A robust regression model is built using the Random Sample Consensus (RANSAC) algorithm. Wind speed data is first expanded using cubic polynomial features to capture non-linear relationships. RANSAC then estimates model parameters and identifies inliers (normal data) and outliers based on residuals. An adaptive threshold is automatically set using the Interquartile Range (IQR) method, making it less reliant on manual tuning.
- Mathematical Morphology-Based Refinement: This stage treats the wind speed-power data distribution as a 2D image. Mathematical morphology operations (erosion and dilation) are applied to extract features and refine outlier identification, particularly for densely stacked outliers that might have bypassed the first two stages. This provides a more comprehensive view of data distribution for outlier detection.
Datasets
The paper evaluates the proposed method using several datasets:
- WT1, WT2: Unlabeled SCADA datasets from two different wind turbines.
- WT3: A labeled SCADA dataset from a third wind turbine, where outliers were manually introduced.
- WT4, WT5: Real-world SCADA datasets from wind farms in Ireland and China, respectively, with operating logs available for anomaly annotation.
- WT6: A simulated dataset generated using MATLAB/Simulink.
Key Insights and Ideas
- Complementarity of Techniques: The core insight is that combining physical rules, statistical regression, and image-based morphological operations creates a synergistic effect, allowing for the detection of a wider range of outlier types than any single method alone.
- Adaptive Thresholding: The use of the IQR method for setting regression error thresholds is a key innovation, reducing reliance on domain expertise and manual parameter tuning, thus increasing applicability.
- Image Representation for Morphology: Treating the wind speed-power distribution as an image for mathematical morphology operations is an effective way to capture spatial patterns of outliers, especially dense clusters.
- Robustness to Outlier Distributions: The proposed method demonstrates strong performance across datasets with varying outlier distributions, highlighting its generalization capability.
Key Results
- The proposed three-stage method significantly outperforms individual methods (LOF, Quartile-DBSCAN, MDUE, RL-based, MM-based) in outlier identification accuracy and robustness.
- For the labeled WT3 dataset, the method achieved an accuracy of over 96% and an F1 score of more than 0.94, with the lowest error rate.
- When applied to wind power prediction, the proposed method led to a significant reduction in prediction errors (up to 20% improvement compared to unprocessed data), demonstrating its practical value.
- Computational efficiency is maintained, with outlier identification completed in under 6 seconds on average for the tested datasets.
Limitations and Future Work
- Parameter Sensitivity: While the method is generally robust, the paper notes that extreme values of the adjustment factor
k(for IQR) and structuring element diameterd(for morphology) can degrade performance. Recommended ranges for these parameters are provided. - Fixing Outliers: The paper focuses on identifying outliers. A significant future direction is developing systematic methods to fix these identified anomalous values, which is crucial for data quality enhancement.
- Extension to Other Data Types: The authors suggest that the proposed method can be extended to handle outliers in other types of SCADA data beyond wind speed and power.
Summary updated