FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

William C. Oltjen, Yangxin Fan, Jiqi Liu, Liangyi Huang, Xuanji Yu, Mengjie Li, Hubert Seigneur, Xusheng Xiao, Kristopher O. Davis, Laura S. Bruckman, Yinghui Wu, Roger H. French2022 · DOI

Abstract

The growth of the photovoltaic (PV) market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Many of these power plants have spatiotemporal coherence that can be utilized for improving model accuracy. We have demonstrated in this paper the FAIRification of spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. Going further, we have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. To solve the issue of data missingness, we propose the use of St-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

Introduction: The Challenge of Solar Data

As the photovoltaic (PV) market scales, researchers need to perform fleet-wide forecasting and performance analysis. However, they face two major hurdles: data silos and inconsistency, and the problem of "missingness." Metadata—such as geographic coordinates, inverter specifications, and panel orientation—is frequently stored in disparate, non-standard formats, hindering cross-institutional collaboration and AI-driven querying. Furthermore, real-world sensor data often contains significant gaps due to equipment failure or maintenance, which can compromise traditional modeling techniques. This research aims to transition from ad-hoc data preparation to an automated, FAIR-compliant pipeline that treats solar data as a machine-readable, high-quality resource.

Methodology: A Standardized Pipeline for Solar Data

The proposed workflow consists of four integrated stages:

  1. FAIRification & Ontology: A formal "solar power plant ontology" was developed to define terms and relationships (e.g., Power Plant $\rightarrow$ hasLatitude $\rightarrow$ 28.39). This allows data to be stored as semantic "triples," facilitating searchability via the Semantic Web and compatibility with distributed storage environments like Hadoop/Hbase.
  2. Automated Software Tools: To facilitate adoption, the researchers released the FAIRmaterials package for both R and Python, automating the metadata standardization process.
  3. Quality Assessment: The PVplr R package assigns letter grades to datasets based on three critical metrics: the percentage of missing data, the duration of the longest missing gap, and the prevalence of outliers.
  4. Spatiotemporal Imputation: To mitigate data gaps, the framework employs Spatiotemporal Graph Neural Network (st-GNN) Autoencoders. This approach models a fleet of solar plants as a graph where physical proximity defines "virtual neighbors." By leveraging spatiotemporal coherence—the fact that nearby plants share similar weather patterns—the GNN can accurately impute missing values from one plant using data from its neighbors.

Datasets: The SunSmart Schools Fleet

The framework was validated using data from multiple sources, organized into eight distinct "fleets." The primary focus was Fleet 8, derived from the SunSmart schools system. This dataset comprises 24 systems with 15-minute interval data spanning nearly a decade (2008–2017), including variables such as power output, ambient temperature, and irradiance.

A typical photovoltaic (PV) array installation, similar to those in the SunSmart schools fleet.

Key Insights: Missingness Patterns and Spatial Context

A core contribution of this work is the classification of data "missingness." Not all data gaps are identical; the paper identifies four distinct patterns: Disjoint (gaps at different times), Overlap (simultaneous gaps), MCAR (random flickers), and Blackout (total system failure). Understanding these patterns is essential for selecting appropriate imputation strategies.

Visualization of spatiotemporal missingness patterns in solar power data, showing gaps across multiple systems over time.

Furthermore, the researchers argue that standardization is a prerequisite for modern AI. FAIRification ensures that models can autonomously discover and utilize data. By treating physical proximity as a feature through GNNs, the framework acknowledges that weather is a regional phenomenon, making a neighboring plant the most reliable "virtual sensor" for a failing system.

Results and Future Directions

The project successfully delivered the FAIRmaterials and PVplr packages to the open-source community via CRAN and PyPI. The framework demonstrated its ability to automatically grade PV datasets, allowing researchers to filter for high-quality data efficiently.

While the st-GNN autoencoder shows great promise for imputation, future work will focus on establishing rigorous accuracy benchmarks (e.g., RMSE) and testing the framework's scalability on larger fleets containing over 1,000 systems. Additionally, the authors emphasize the need for broader institutional adoption of FAIR standards in the US to match ongoing efforts in Europe.

Summary updated

Abstract

The growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Many of these power plants have spatiotemporal coherence that can be utilized for improving model accuracy. We have demonstrated in this paper the FAIRification of spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. Going further, we have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. To solve the issue of data missingness, we propose the use of St-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.
BibTeX
@inproceedings{oltjen2022fairification,
  author = {William C. Oltjen and Yangxin Fan and Jiqi Liu and Liangyi Huang and Xuanji Yu and Mengjie Li and Hubert Seigneur and Xusheng Xiao and Kristopher O. Davis and Laura S. Bruckman and Yinghui Wu and Roger H. French},
  title = {{FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data}},
  booktitle = {2022},
  year = {2022},
  doi = {10.2172/1959064},
}
Figure