TL;DR
This paper introduces transformer-based deep learning models for predicting the long-term production behavior of oil wells. By leveraging the transformer's self-attention mechanism, these models outperform traditional recurrent neural networks (RNNs) in forecasting bottomhole pressure, even accounting for complex well interferences in an oilfield. The research also demonstrates the effectiveness of transfer learning and a "global" model approach for improved prediction accuracy.
What Problem Does This Paper Address?
Accurately predicting the transient production of oil wells is crucial for optimizing hydrocarbon recovery and planning field development. Traditional methods often rely on complex, time-consuming hydrodynamic simulators that require extensive domain expertise and detailed reservoir information. While data-driven approaches using RNNs (like LSTMs and GRUs) have shown promise, they struggle with capturing long-term dependencies and can suffer from vanishing/exploding gradient issues. This paper aims to bridge this gap by applying the more powerful transformer architecture to this problem, offering a faster and potentially more accurate alternative for modeling well performance.
Methods
The core of this work is the application and adaptation of the Transformer architecture for time-series forecasting in the oil and gas domain. Unlike RNNs that process data sequentially, transformers utilize a self-attention mechanism to weigh the importance of different parts of the input sequence simultaneously. This allows them to capture long-range dependencies more effectively.
The authors developed two main model types:
- Single-well model: A transformer trained on the historical data of an individual well.
- Global model: A transformer trained on data from multiple wells, incorporating well identifiers as input features to capture well interference effects.
Additionally, they explored two key techniques:
- Transfer Learning: Pre-training a transformer model on data from one well and then fine-tuning it on data from a different target well. This leverages learned patterns to improve prediction accuracy and accelerate training for the target well.
- Reduced Feature Set: Investigating the model's performance when trained on a subset of input features (e.g., only choke diameter and bottomhole temperature, excluding flow rates) to assess robustness and the impact of data availability.
Datasets
The study utilizes publicly available field data from the Volve oilfield in the North Sea, provided by Equinor. The dataset contains daily measurements for two wells (15/9-F-1C and 15/9-F-15D) covering parameters such as oil, gas, and water flow rates, bottomhole temperature, choke size, and bottomhole pressure. This data was collected from April 2014 to April 2016.
Key Insights and Ideas
- Transformer's Superiority in Capturing Dynamics: The self-attention mechanism of transformers is demonstrably better at capturing the complex, non-linear transient dynamics of oil well production compared to traditional RNNs. This is particularly evident in more variable test datasets.
- Well Interference is Key: For wells within an oilfield, ignoring the influence of neighboring wells (as in single-well models) can lead to less accurate predictions. A "global" model that accounts for these interferences significantly improves forecasting.
- Transfer Learning Benefits: Transfer learning proves to be a valuable technique, allowing models to adapt more quickly and achieve better performance on new wells by leveraging knowledge gained from pre-training.
- Robustness to Feature Reduction: Surprisingly, in some cases, transformer models can maintain good predictive accuracy even when trained on a reduced set of features (e.g., excluding flow rates), suggesting they can learn effectively from more readily available data like temperature and choke settings.
- Potential for Well-Testing Alternatives: The ability of these models to predict bottomhole pressure with high accuracy opens possibilities for well-testing interpretations without the need for costly shut-in operations.
Key Results
- Outperformance of RNNs: The transformer model consistently outperformed vanilla RNN, LSTM, and GRU models in predicting bottomhole pressure. For well 15/9-F-1C, RMSE was reduced from 7.76 (LSTM) to 7.66 (Transformer), and for well 15/9-F-15D, RMSE dropped from 11.05 (LSTM) to 10.01 (Transformer).
- Global Model Improvement: The global transformer model showed significant improvements over single-well models. For well 15/9-F-1C, RMSE decreased by 24.80%, and for well 15/9-F-15D, it decreased by 20.38%.
- Transfer Learning Gains: Fine-tuning a pre-trained transformer model on a new well showed modest but positive improvements, with RMSE for well 15/9-F-15D decreasing by 4.80% after fine-tuning.
- Reduced Feature Set Performance: For well 15/9-F-1C, the transformer trained on a reduced feature set showed a slight degradation in performance (RMSE increased by 35.24%), but for LSTM, it actually improved. This highlights the different sensitivities of the models to input features.
Limitations and Future Work
- Dataset Size: The authors acknowledge that the dataset size, while sufficient for demonstrating the concepts, is not as large as typically found in NLP tasks. This can lead to overfitting, which they addressed by reducing model complexity.
- Transfer Learning Scope: The transfer learning experiment used data from only one well for pre-training. Future work could explore the benefits of pre-training on a larger, more diverse set of wells.
- Feature Engineering: While the transformer reduces the need for manual feature engineering compared to traditional ML, further investigation into optimal feature sets for different wells and scenarios could be beneficial.
- Flow Rate Prediction: While the paper mentions building a flow rate prediction model, the detailed results are presented for bottomhole pressure. Further exploration of flow rate prediction accuracy and its comparison with other methods would be valuable.
- Generalization to More Wells: The global model was tested on only two wells. Expanding this to a larger number of wells would provide a more robust validation of its ability to capture field-wide dynamics.
- Well-Testing Applications: The paper suggests using the models for well-testing interpretation. Future work could focus on developing and validating specific methodologies for this application.
Summary updated