Abstract
Wind power is a cornerstone of the clean energy transition, yet its inherent volatility poses significant challenges for grid stability. We present WindFM, a lightweight (8.1M parameter) foundation model specifically designed for wind power forecasting. By utilizing a "discretize-and-generate" framework trained on a massive dataset of over 126,000 sites, WindFM achieves state-of-the-art zero-shot performance. It consistently outperforms both site-specific models and significantly larger general-purpose time-series models without the need for fine-tuning, demonstrating a robust understanding of domain-specific wind dynamics.
Introduction
Managing wind power is a complex task for grid operators due to its high variability. Traditional forecasting approaches generally follow two paths: site-specific models, which are accurate but expensive to maintain and fail to leverage shared atmospheric patterns, or general-purpose foundation models (like Chronos or TimesFM), which often struggle with the domain-specific physics of the energy sector. WindFM bridges this gap by providing a domain-specific foundation model that understands the non-linear interplay between atmospheric conditions—such as wind speed, pressure, and temperature—and power output right out of the box.
Methodology
WindFM utilizes a discretize-and-generate framework, treating time-series forecasting similarly to how Large Language Models (LLMs) process text.
The core components of the architecture include:
- Time-series Tokenizer: A Transformer-based autoencoder converts multivariate observations (power, wind speed, direction, air density, temperature, and pressure) into discrete tokens.
- Hierarchical Tokenization: To capture both broad trends and fine-grained volatility, each token is factorized into a "coarse" subtoken (representing the principal structure) and a "fine" subtoken (representing residual details).
- Decoder-only Transformer: A compact 8.1M parameter Transformer learns to predict the next token in the sequence autoregressively.
- Fourier-based Temporal Encoding: To handle strong diurnal and seasonal cycles, the model uses a temporal encoding scheme based on sinusoidal functions rather than simple linear embeddings.
Datasets and Training
The model's foundational knowledge is derived from the WIND Toolkit by NREL, comprising approximately 150 billion time steps from 126,691 sites across the continental United States (2007–2012). To ensure the model is scale-agnostic, the data was augmented by downsampling to various resolutions (e.g., 15-min, 1-hour). Evaluation was conducted on unseen US sites from 2013 and a completely out-of-distribution dataset from Inner Mongolia, China, to test cross-geographical generalization.
Key Results and Insights
WindFM demonstrates that domain specificity can be more effective than sheer model scale. Despite having orders of magnitude fewer parameters than general models like Chronos, WindFM consistently delivers higher accuracy.
Key findings include:
- Zero-Shot Superiority: In domestic tests, WindFM achieved lower Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) than specialized models (like PatchTST and DLinear) that were specifically trained on the target data.
- The Power of Hierarchy: The coarse-to-fine tokenization strategy allows the model to capture sharp ramps and high-frequency fluctuations, avoiding the "smoothing" effect common in many deep learning models.
- Global Transferability: A model trained exclusively on US wind data accurately forecasted wind power in China without any retraining, indicating the model learned universal physical principles of wind generation.
- Probabilistic Accuracy: WindFM provides well-calibrated confidence intervals that adjust based on weather volatility, outperforming all baselines in uncertainty quantification tasks.
Conclusion and Future Work
WindFM represents a significant step forward for energy-specific foundation models. Its small size and high efficiency make it ideal for operational environments where computational overhead is a concern. Future research will focus on integrating "slow-thinking" reasoning mechanisms for better interpretability, utilizing Retrieval-Augmented Generation (RAG) to access historical weather events, and extending the framework to include other renewable sources like solar power and electricity price datasets.