Abstract
Physics-based models of dynamical systems are often used to study engineering and environmental systems. Despite their extensive use, these models have several well-known limitations due to simplified representations of the physical processes being modeled or challenges in selecting appropriate parameters. While-state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improves the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physics-based models, while generating outputs consistent with physical laws. An important aspect of our PGRNN approach lies in its ability to incorporate the knowledge encoded in physics-based models. This allows training the PGRNN model using very few true observed data while also ensuring high prediction accuracy. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where physics-based (also known as mechanistic) models are used, e.g., climate science, materials science, computational chemistry, and biomedicine.
Introduction
Accurately modeling environmental systems, such as lake temperature profiles, is critical for managing aquatic ecosystems and understanding the impacts of climate change. Traditionally, scientific inquiry has relied on mechanistic, physics-based models like the General Lake Model (GLM). While these models are grounded in established physical theory, they often struggle with simplified representations of complex processes—such as wind stirring, surface heating, and evaporative cooling—and require the calibration of numerous parameters.
Conversely, pure machine learning (ML) models, such as Long Short-Term Memory (LSTM) networks, excel at extracting patterns from large datasets but often operate as "black boxes." These models lack inherent knowledge of physical laws, potentially leading to predictions that violate fundamental principles like the conservation of energy. This paper addresses the trade-off between the predictive accuracy of ML and the structural consistency of physics-based models, particularly in data-sparse scenarios.
Methodology: The PGRNN Framework
The authors introduce the Physics-Guided Recurrent Neural Network (PGRNN), a hybrid architecture designed to bridge the gap between mechanistic modeling and deep learning. The framework relies on three core innovations:
- LSTM Architecture: LSTMs are employed to capture the complex, long-term temporal dependencies inherent in lake thermodynamics, such as seasonal cycles and thermal stratification.
- Physics-Guided Loss Function: A custom penalty term is integrated into the training objective to enforce energy conservation. The model is penalized if the predicted change in thermal energy does not align with the net heat flux (the balance of incoming and outgoing energy).
- Synthetic Pre-training: To mitigate the challenge of sparse observational data, the PGRNN is first trained on "synthetic" data generated by an uncalibrated GLM. This "warm start" allows the network to learn general physical rules before being fine-tuned on real-world observations.
The model was evaluated using data from Lake Mendota, Wisconsin, utilizing seven meteorological drivers (e.g., solar radiation, wind speed) and over 13,000 temperature measurements spanning 1980–2014.
Key Results and Insights
The PGRNN demonstrated significant improvements over both traditional physics models and standard ML approaches:
- Superior Accuracy: The PGRNN achieved a Root Mean Square Error (RMSE) of 1.47°C, outperforming the calibrated GLM (1.84°C) and standard RNNs (1.49°C).
- Data Efficiency: In extreme low-data regimes (using only 0.2% of available data), the pre-trained PGRNN maintained an RMSE of 2.05°C, while standard RNNs failed completely with errors exceeding 4.6°C.
- Physical Integrity: By using physical consistency as a regularizer, the PGRNN reduced energy conservation violations by orders of magnitude compared to black-box models.
Furthermore, the study revealed that pre-training on synthetic data from different lake geometries (e.g., lakes in different climates) still improved performance, suggesting the model learns universal physical relationships rather than just site-specific patterns.
Conclusion and Future Directions
The PGRNN framework represents a significant step toward integrating scientific knowledge into deep learning. While the current model successfully handles ice-free periods, future work will aim to incorporate the complex dynamics of snow and ice cover. Additionally, extending the model from 1D depth profiles to 3D spatial simulations remains a key challenge. Beyond limnology, this physics-guided approach holds promise for diverse fields governed by conservation laws, including hydrology, climate science, and computational fluid dynamics.
Summary updated