Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells

Ildar Abdrakhmanov, Evgenii Kanin, Sergei Boronin, Evgeny Burnaev, Andrei OsiptsovarXiv 2021 · DOI · OpenReview

TL;DR

This paper introduces transformer-based deep learning models for predicting the long-term production behavior of oil wells. By leveraging the transformer's self-attention mechanism, these models outperform traditional recurrent neural networks (RNNs) in forecasting bottomhole pressure, even accounting for complex well interferences in an oilfield. The research also demonstrates the effectiveness of transfer learning and a "global" model approach for improved prediction accuracy.

What Problem Does This Paper Address?

Accurately predicting the transient production of oil wells is crucial for optimizing hydrocarbon recovery and planning field development. Traditional methods often rely on complex, time-consuming hydrodynamic simulators that require extensive domain expertise and detailed reservoir information. While data-driven approaches using RNNs (like LSTMs and GRUs) have shown promise, they struggle with capturing long-term dependencies and can suffer from vanishing/exploding gradient issues. This paper aims to bridge this gap by applying the more powerful transformer architecture to this problem, offering a faster and potentially more accurate alternative for modeling well performance.

Methods

The core of this work is the application and adaptation of the Transformer architecture for time-series forecasting in the oil and gas domain. Unlike RNNs that process data sequentially, transformers utilize a self-attention mechanism to weigh the importance of different parts of the input sequence simultaneously. This allows them to capture long-range dependencies more effectively.

The authors developed two main model types:

  • Single-well model: A transformer trained on the historical data of an individual well.
  • Global model: A transformer trained on data from multiple wells, incorporating well identifiers as input features to capture well interference effects.

Additionally, they explored two key techniques:

  • Transfer Learning: Pre-training a transformer model on data from one well and then fine-tuning it on data from a different target well. This leverages learned patterns to improve prediction accuracy and accelerate training for the target well.
  • Reduced Feature Set: Investigating the model's performance when trained on a subset of input features (e.g., only choke diameter and bottomhole temperature, excluding flow rates) to assess robustness and the impact of data availability.

Datasets

The study utilizes publicly available field data from the Volve oilfield in the North Sea, provided by Equinor. The dataset contains daily measurements for two wells (15/9-F-1C and 15/9-F-15D) covering parameters such as oil, gas, and water flow rates, bottomhole temperature, choke size, and bottomhole pressure. This data was collected from April 2014 to April 2016.

Key Insights and Ideas

  • Transformer's Superiority in Capturing Dynamics: The self-attention mechanism of transformers is demonstrably better at capturing the complex, non-linear transient dynamics of oil well production compared to traditional RNNs. This is particularly evident in more variable test datasets.
  • Well Interference is Key: For wells within an oilfield, ignoring the influence of neighboring wells (as in single-well models) can lead to less accurate predictions. A "global" model that accounts for these interferences significantly improves forecasting.
  • Transfer Learning Benefits: Transfer learning proves to be a valuable technique, allowing models to adapt more quickly and achieve better performance on new wells by leveraging knowledge gained from pre-training.
  • Robustness to Feature Reduction: Surprisingly, in some cases, transformer models can maintain good predictive accuracy even when trained on a reduced set of features (e.g., excluding flow rates), suggesting they can learn effectively from more readily available data like temperature and choke settings.
  • Potential for Well-Testing Alternatives: The ability of these models to predict bottomhole pressure with high accuracy opens possibilities for well-testing interpretations without the need for costly shut-in operations.

Key Results

  • Outperformance of RNNs: The transformer model consistently outperformed vanilla RNN, LSTM, and GRU models in predicting bottomhole pressure. For well 15/9-F-1C, RMSE was reduced from 7.76 (LSTM) to 7.66 (Transformer), and for well 15/9-F-15D, RMSE dropped from 11.05 (LSTM) to 10.01 (Transformer).
  • Global Model Improvement: The global transformer model showed significant improvements over single-well models. For well 15/9-F-1C, RMSE decreased by 24.80%, and for well 15/9-F-15D, it decreased by 20.38%.
  • Transfer Learning Gains: Fine-tuning a pre-trained transformer model on a new well showed modest but positive improvements, with RMSE for well 15/9-F-15D decreasing by 4.80% after fine-tuning.
  • Reduced Feature Set Performance: For well 15/9-F-1C, the transformer trained on a reduced feature set showed a slight degradation in performance (RMSE increased by 35.24%), but for LSTM, it actually improved. This highlights the different sensitivities of the models to input features.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the dataset size, while sufficient for demonstrating the concepts, is not as large as typically found in NLP tasks. This can lead to overfitting, which they addressed by reducing model complexity.
  • Transfer Learning Scope: The transfer learning experiment used data from only one well for pre-training. Future work could explore the benefits of pre-training on a larger, more diverse set of wells.
  • Feature Engineering: While the transformer reduces the need for manual feature engineering compared to traditional ML, further investigation into optimal feature sets for different wells and scenarios could be beneficial.
  • Flow Rate Prediction: While the paper mentions building a flow rate prediction model, the detailed results are presented for bottomhole pressure. Further exploration of flow rate prediction accuracy and its comparison with other methods would be valuable.
  • Generalization to More Wells: The global model was tested on only two wells. Expanding this to a larger number of wells would provide a more robust validation of its ability to capture field-wide dynamics.
  • Well-Testing Applications: The paper suggests using the models for well-testing interpretation. Future work could focus on developing and validating specific methodologies for this application.

Summary updated

Abstract

We propose a novel approach to data-driven modeling of a transient production of oil wells. We apply the transformer-based neural networks trained on the multivariate time series composed of various parameters of oil wells measured during their exploitation. By tuning the machine learning models for a single well (ignoring the effect of neighboring wells) on the open-source field datasets, we demonstrate that transformer outperforms recurrent neural networks with LSTM/GRU cells in the forecasting of the bottomhole pressure dynamics. We apply the transfer learning procedure to the transformer-based surrogate model, which includes the initial training on the dataset from a certain well and additional tuning of the model's weights on the dataset from a target well. Transfer learning approach helps to improve the prediction capability of the model. Next, we generalize the single-well model based on the transformer architecture for multiple wells to simulate complex transient oilfield-level patterns. In other words, we create the global model which deals with the dataset, comprised of the production history from multiple wells, and allows for capturing the well interference resulting in more accurate prediction of the bottomhole pressure or flow rate evolutions for each well under consideration. The developed instruments for a single-well and oilfield-scale modelling can be used to optimize the production process by selecting the operating regime and submersible equipment to increase the hydrocarbon recovery. In addition, the models can be helpful to perform well-testing avoiding costly shut-in operations.
BibTeX
@misc{abdrakhmanov2021development,
  author = {Ildar Abdrakhmanov and Evgenii Kanin and Sergei Boronin and Evgeny Burnaev and Andrei Osiptsov},
  title = {{Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells}},
  year = {2021},
  eprint = {2110.06059},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.06059},
}
Figure