Satellite-Text-Prompted Large Language Model for Photovoltaic Power Forecasting

Pengfei Jia, Jianghong Ma, Bin Zhang, Kenghong Lin, Xi Zhang, Chuyao Luo, Xutao Li, Yunming Ye

Abstract

Photovoltaic (PV) power forecasting is critical for the operation of solar power plants and the coordination of energy within power grids. This work aims to predict future PV power time series by leveraging multimodal data. While recent studies have incorporated numerical modalities such as satellite image sequences and numerical weather prediction (NWP) time series, they often overlook textual modalities—such as the spatio-temporal context of PV plants—and the potential of pretrained large language models (LLMs). In this paper, we build upon existing numerical inputs and further explore the use of spatio-temporal text prompts, generated based on plant coordinates and forecast start time, to enhance the forecasting process. We propose PV-LLM, a satellite-text-prompted framework that integrates a pretrained LLM to improve PV power forecasting. The framework consists of three key components: Text Prompt Construction, Modality-Specific Encoding, and Adaptive Prompt Tuning. First, the Text Prompt Construction module generates spatio-temporal prompts that offer high-level semantic guidance. Next, the Modality-Specific Encoding module encodes each modality according to its unique characteristics, capturing modality-specific patterns while managing varying context lengths. Finally, the Adaptive Prompt Tuning module fine-tunes the LLM to integrate multimodal embeddings, while an adaptive gating mechanism retains its pretrained knowledge. We validate the effectiveness of the proposed framework on a real-world dataset containing multiple PV plants. Experimental results demonstrate that our approach outperforms existing state-of-the-art methods.

Introduction

Photovoltaic (PV) power forecasting is notoriously difficult because solar output is highly intermittent, fluctuating wildly based on cloud cover and local weather. While current state-of-the-art models use Numerical Weather Prediction (NWP) and satellite imagery, they often struggle with the "modality gap"—the difficulty of fusing raw numbers with 2D grids in a way that captures the "big picture." Furthermore, standard neural networks often treat high-level context, such as plant location or time of day, as just more numbers, missing the semantic "meaning" (e.g., "it's noon in a coastal region") that human forecasters use to refine their predictions.

Methodology

The proposed PV-LLM framework treats solar power prediction as a multimodal "reasoning" task, using a pretrained LLM (GPT-2) as a core engine to fuse three distinct types of data.

1. Text Prompt Construction (TPC)

This module converts metadata into natural language. Instead of raw coordinates, it generates a structured prompt: "Target coordinates are [Lat, Lon]... Starting at [Time]..." This activates the LLM’s internal knowledge about geographic and temporal patterns, leveraging its "common sense" about the world.

2. Modality-Specific Encoding (MSE)

To bridge the gap between continuous physical data and discrete language, the MSE module employs specialized encoders:

  • Temporal-Semantic Encoder: Maps PV and weather time-series into the LLM's "word embedding space" using a Semantic Adapter.
  • Satellite Image Sequence Encoder: Uses Rotary Positional Encoding (RoPE) and a Context Compressor to help the model understand spatial relationships in satellite imagery without overwhelming the LLM's context window.

3. Adaptive Prompt Tuning (APT)

Rather than simple concatenation, satellite information is injected into intermediate LLM layers. An adaptive gating mechanism, initialized to zero, allows the model to gradually integrate satellite data without corrupting the LLM's pretrained knowledge during early training stages.

Overall architecture of the PV-LLM framework, showing the integration of text prompts, satellite imagery, and numerical weather data.

Experimental Results

The model was evaluated on the MMSP(S) dataset, which includes 1.5 years of ground-truth PV measurements from 10 plants in China, high-resolution cloud dynamics from Himawari-8/9 satellites, and meteorological predictions from ECMWF.

PV-LLM outperformed 15 baseline methods, including specialized time-series models (like PatchTST) and other multimodal fusion networks (like FusionSF). As shown in the results table, PV-LLM achieved the lowest Mean Absolute Error (MAE) of 0.03962 across the entire dataset.

Comparison of PV-LLM against 15 baseline methods across 'All', 'Easy', and 'Hard' forecasting scenarios.

Notably, in "hard" scenarios where weather is highly unpredictable (e.g., sudden storms), PV-LLM showed the most significant improvement. This suggests that the multimodal approach provides a robust "safety net" for edge cases.

Conclusion

PV-LLM demonstrates that describing a power plant's context in words and mapping numerical data into "word prototypes" allows LLMs to effectively reason about physical systems. While the computational overhead of LLMs remains a challenge for real-time deployment, the gains in accuracy and robustness—especially in unpredictable weather—mark a significant step forward for renewable energy forecasting.

Summary updated

Abstract

Photovoltaic (PV) power forecasting is critical for the operation of solar power plants and the coordination of energy within power grids. This work aims to predict future PV power time series by leveraging multimodal data. While recent studies have incorporated numerical modalities such as satellite image sequences and numerical weather prediction (NWP) time series, they often overlook textual modalities—such as the spatio-temporal context of PV plants—and the potential of pretrained large language models (LLMs). In this paper, we build upon existing numerical inputs and further explore the use of spatio-temporal text prompts, generated based on plant coordinates and forecast start time, to enhance the forecasting process. We propose PV-LLM, a satellite-text-prompted framework that integrates a pretrained LLM to improve PV power forecasting. The framework consists of three key components: Text Prompt Construction, Modality-Specific Encoding, and Adaptive Prompt Tuning. First, the Text Prompt Construction module generates spatio-temporal prompts that offer high-level semantic guidance. Next, the Modality-Specific Encoding module encodes each modality according to its unique characteristics, capturing modality-specific patterns while managing varying context lengths. Finally, the Adaptive Prompt Tuning module fine-tunes the LLM to integrate multimodal embeddings, while an adaptive gating mechanism retains its pretrained knowledge. We validate the effectiveness of the proposed framework on a real-world dataset containing multiple PV plants. Experimental results demonstrate that our approach outperforms existing state-of-the-art methods.
BibTeX
@inproceedings{jia2026satellite,
  author = {Pengfei Jia and Jianghong Ma and Bin Zhang and Kenghong Lin and Xi Zhang and Chuyao Luo and Xutao Li and Yunming Ye},
  title = {{Satellite-Text-Prompted Large Language Model for Photovoltaic Power Forecasting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i45.41207},
}
Figure