LLM-Enhanced Feature Engineering for Multi-factor Electricity Price Predictions

Haochen Xue, Chenghao Liu, Chong Zhang, Yuxuan Chen, Angxiao Zong, Zhaodong Wu, Yulong Li, Jiayi Liu, Kaiyu Liang, Zhixiang Lu, Robert K.y. Li, Jionglong Su, Jianghao WuCommunications 2025 · DOI · OpenAlex

Abstract

Predicting electricity price volatility is a critical challenge in liberalized energy markets due to the complex interplay of weather patterns, supply-demand dynamics, and regulatory shifts. This paper introduces FAEP (Feature Augmented Electricity Price Prediction), a framework that integrates Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to enhance feature engineering. By converting qualitative external data, such as annual weather reports, into quantitative "weather ratings," and employing a hybrid XGBoost-LSTM architecture, the FAEP framework achieves state-of-the-art results. In the New South Wales (NSW) electricity market, FAEP reduced prediction errors by nearly 49% compared to traditional statistical and deep learning models.

Introduction

Electricity price forecasting is notoriously difficult, particularly in high-volatility environments like the Australian National Electricity Market (NEM). Traditional econometric models, such as ARIMA or GARCH, often struggle with the non-linear dynamics inherent in these markets. While deep learning models like Long Short-Term Memory (LSTM) networks offer improvements, they frequently fail to incorporate external context effectively or suffer from the "dimensionality curse" when presented with noisy datasets.

The FAEP framework addresses these limitations by using LLMs as "interpreters" for unstructured domain knowledge. By refining external context into actionable features before the prediction phase, FAEP bridges the gap between qualitative meteorological trends and quantitative price forecasting.

The FAEP Framework

The FAEP methodology is divided into three distinct stages: feature augmentation, advanced metric calculation, and hybrid modeling.

1. LLM-Based Feature Augmentation (RAG)

The core innovation of FAEP is the use of Retrieval-Augmented Generation to ingest unstructured data. The authors processed annual NSW weather reports (PDFs) from 2009 to 2019 using GPT-3.5-turbo via the Llama-Index. The LLM analyzed these reports to generate a numerical "weather rating" or confidence score. This process effectively transforms decade-long qualitative trends into a structured feature set that a machine learning model can process.

2. Advanced Feature Engineering

Beyond weather data, the framework calculates specific volatility metrics to capture market behavior:

  • Realized Volatility (RV): The sum of squared price variations.
  • Price Jumps (PJ): Capturing sudden, sharp fluctuations in spot prices, which the study found account for nearly 32% of daily realized volatility.
  • Continuity of Price Transitions (CV): Measuring the smoothness of price changes within a single day.

3. Hybrid Modeling and Dimensionality Reduction

To manage the high-dimensional data, the framework employs Sequential Feature Selection (SFS) to identify the 30 most impactful features, followed by Kernel Principal Component Analysis (KPCA) for further reduction.

The prediction engine is a two-stage ensemble:

  • XGBoost: Captures complex non-linear relationships and handles high-dimensional interactions.
  • LSTM: Refines predictions by accounting for long-term temporal dependencies and the "memory" of the time series.
  • Aggregation: The final output is a weighted aggregation based on the residuals of both models.

Experimental Setup and Data

The framework was tested using data from the New South Wales electricity market, managed by the Australian Energy Market Operator (AEMO). The dataset spanned ten years (July 2009 to July 2019), comprising 175,344 observations at 30-minute intervals. External data consisted of academic and government weather reports for the same period.

Results and Performance

FAEP was benchmarked against established models, including GARCH and the HAR-family of models. The results demonstrated significant performance gains:

  • Error Metrics: FAEP achieved the lowest error rates across all categories: MAE (0.59), MSE (0.63), and MAPE (0.54).
  • Comparative Improvement: Compared to the next best-performing model (HARQ-L-CJ), FAEP reduced MAE and MSE by approximately 32.18% and 22.21%, respectively.
  • Ablation Study: The importance of LLM-augmented features was stark; removing the "weather rating" caused the MAE to jump from 0.59 to 1.63, confirming that the LLM-derived signals are critical for accuracy.

Discussion and Conclusion

The FAEP framework demonstrates that LLMs can serve as a powerful bridge between unstructured domain knowledge and structured machine learning inputs. The combination of XGBoost’s non-linear handling and LSTM’s temporal memory provides a robust architecture for volatile markets.

However, several areas for future research remain. The current dependency on GPT-3.5-turbo introduces questions regarding computational latency in real-time environments. Additionally, while the results in the NSW market are compelling, further validation in other global energy markets with different regulatory frameworks is necessary to confirm the generalizability of the RAG-LLM approach. Future iterations may also focus on automating prompt engineering to further optimize the feature extraction process.

Summary updated

BibTeX
@inproceedings{xue2025llm,
  author = {Haochen Xue and Chenghao Liu and Chong Zhang and Yuxuan Chen and Angxiao Zong and Zhaodong Wu and Yulong Li and Jiayi Liu and Kaiyu Liang and Zhixiang Lu and Robert K.Y. Li and Jionglong Su and Jianghao Wu},
  title = {{LLM-Enhanced Feature Engineering for Multi-factor Electricity Price Predictions}},
  booktitle = {Communications},
  year = {2025},
  doi = {10.1007/978-981-96-9986-5\_8},
}

Loading related content...

Figure