TL;DR
This paper investigates how machine learning models (CatBoost, MBP, LSTM) perform in simulating rainfall-runoff under changing environmental conditions. They found that simply training on the entire historical dataset leads to significant performance degradation due to data inconsistencies. However, by identifying and accounting for a detected change point in the data, staged simulations dramatically improved accuracy, with CatBoost showing superior performance in this phased approach.
What Problem Does This Paper Address?
Accurate rainfall-runoff (RR) simulation is vital for water resource management, flood prediction, and disaster reduction. However, the assumption of stable hydrological time series is increasingly challenged by climate change and human activities, leading to shifts in the underlying processes that generate runoff. This "changing environment" scenario makes traditional RR models, which often assume data consistency, less reliable. The paper aims to assess the applicability and performance of different machine learning (ML) models for RR simulation when faced with such non-stationary, inconsistent hydrological data.
Methods
The study employed a non-real-time correction modeling scheme, meaning the models predict future runoff based on past rainfall and simulated antecedent flow, rather than relying on measured prior flow. Three ML models were chosen for their suitability in time-series prediction:
- CatBoost: A gradient boosting algorithm known for its efficiency with categorical data, fast training, and ability to handle large datasets.
- Multi-layer BP (MBP) neural network: A foundational neural network architecture, with multiple hidden layers chosen to enhance generalization and prediction accuracy for complex relationships.
- Long Short-Term Memory (LSTM) neural network: A type of recurrent neural network specifically designed to capture long-range dependencies in sequential data, crucial for time-series modeling.
The core methodological innovation lies in how the data was handled. Initially, models were trained and tested on the entire 1961-2000 dataset. Subsequently, the researchers identified a significant change point in the hydrological data (around 1989) using Mann-Kendall trend and change point tests. This led to a second approach: splitting the dataset into a "baseline period" (pre-change) and a "changing period" (post-change) and training separate models for each period. This "staged" simulation approach was compared against the "entire dataset" approach.
Datasets
The study utilized daily rainfall and observed flow data from the Dongwan catchment in the Yiluo River Basin, China, spanning from 1961 to 2000. Rainfall data was collected from eight stations within the catchment. A 90m resolution digital elevation model (DEM) was used for watershed delineation. The dataset was divided into training (80%) and testing (20%) sets for various simulation scenarios. Notably, the paper does not introduce a new benchmark dataset but rather analyzes the impact of data consistency issues within an existing, long-term hydrological record.
Key Insights and Ideas
- Data Consistency is Paramount: The most striking finding is that ML models trained on a dataset with a significant change point (disrupted consistency) perform poorly when applied to the entire period without accounting for this change. This highlights that even advanced ML models struggle with non-stationarity if not explicitly addressed.
- Change Point Detection is Crucial: Identifying the historical shift in hydrological behavior (attributed to human activities like land use changes and ecological park construction) was key. This allowed for a more realistic approach to modeling.
- Staged Simulation Outperforms Unified: Splitting the data at the identified change point and training separate models for the baseline and changing periods dramatically improved simulation accuracy compared to using the entire dataset. This suggests that adapting models to specific environmental regimes is more effective.
- CatBoost Shines in Phased Simulations: When using staged simulations, the CatBoost model demonstrated superior accuracy, smoother simulation processes, and better matching of observed runoff across different flow levels.
- MBP's Generalization Strength: Interestingly, when forced to simulate the entire dataset without staging, the MBP model showed the strongest generalization capability and overall accuracy, suggesting its robustness in handling complex, albeit inconsistent, data.
Key Results
- Entire Dataset Simulation: When trained on the full 1961-2000 dataset, all models showed good performance during training (R² > 0.9), but accuracy significantly dropped during testing (R² around 0.6), with peak flow errors exceeding 20%.
- Staged Simulation (Baseline Period): For the 1961-1989 baseline period, all models performed well, with R² values above 0.80 and errors within 20%. CatBoost slightly outperformed MBP and LSTM.
- Staged Simulation (Changing Period): For the 1990-2000 changing period, R² values remained high (above 0.87), and errors were generally below 20%. CatBoost and MBP showed similar performance, with CatBoost's simulated daily runoff processes closely matching observed hydrographs.
- Cross-Sampling (Entire Dataset): When using cross-sampling (even/odd years) on the entire dataset, the MBP model showed the best performance, with higher evaluation indicators and more stable FDC performance compared to CatBoost and LSTM, suggesting its advantage in dealing with complex nonlinear problems in non-stationary data.
Limitations and Future Work
- Data Consistency Issues: The primary limitation identified is the impact of data inconsistency due to environmental changes. The study highlights that using a single model for the entire period without accounting for these shifts leads to significant biases.
- Future Research Directions: The authors suggest several avenues for future work:
- Exploring a wider variety of ML models for daily RR simulation.
- Developing methods for real-time identification of mutation points in RR simulations.
- Applying these improved methods to a broader range of watersheds to validate their generalizability.
Summary updated