Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing

Jan Lewen, Max Pargmann, Mehdi Cherti, Jenia Jitsev, Robert Pitz‐Paal, Daniel Maldonado Quinto

Abstract

Concentrating Solar Power (CSP) plants rely on thousands of heliostats to focus sunlight onto a central receiver. Precise characterization of mirror surface deformations and alignment errors is critical for performance but traditionally labor-intensive. This paper presents inverse Deep Learning Raytracing (iDLR), a novel pipeline that reconstructs 3D surface profiles directly from standard calibration images. By bridging the "Sim-to-Real" gap through domain adaptation and randomization, the method achieves a median surface error of 0.17 mm in real-world tests at the Solar Tower Jülich, outperforming standard "ideal mirror" assumptions by 26%.

Introduction

In CSP plants, efficiency is dictated by the precision of solar flux distribution on the receiver. Real-world heliostats deviate from ideal geometries due to "canting" (misalignment of facets) and surface "waviness." While high-precision deflectometry can measure these imperfections, its cost prohibits plant-wide deployment. Consequently, most control systems rely on idealized flat or parabolic models, leading to energy loss and potential receiver damage from "hot spots." The challenge lies in characterizing these surfaces automatically using routine operational data.

Methods: The iDLR Pipeline

The proposed iDLR pipeline processes "target images"—photographs of light spots cast on a calibration target—to infer the underlying mirror geometry.

  1. Domain Adaptation (UNet): To handle real-world noise, background radiation, and exposure variations, a UNet translates raw photos into clean "flux density maps" consistent with idealized simulations.
  2. The iDLR Model: The core engine employs a Vision Transformer (ViT) encoder to process flux maps and a StyleGAN2 decoder to generate the corresponding 3D surface profile.
  3. Domain Randomization: Training data is injected with physical phenomena such as mirror soiling, sunshape variations, and camera artifacts to ensure robustness when transitioning from simulation to reality.
The full iDLR pipeline: from raw observation to 3D surface output.

Datasets and Validation

The model was trained on a "semi-artificial" dataset derived from 393 real deflectometry measurements at the Solar Tower Jülich (STJ). Data augmentation techniques, including rotation and weighted averaging, were used to generate approximately 400,000 synthetic surfaces. Validation was conducted on 63 real-world heliostats at the STJ facility under actual operational conditions. High-resolution deflectometry served as the ground truth for accuracy assessment.

Spatial distribution of the heliostat field at Solar Tower Jülich used for training and validation.

Key Results and Insights

  • Surface and Flux Accuracy: The model achieved a median Mean Absolute Error (MAE) of 0.17 mm on mirror surfaces. Using these predicted surfaces in raytracing simulations yielded 90% accuracy in predicting actual light distribution.
  • Sim-to-Real Efficacy: Bridging the simulation-reality gap proved essential; without domain adaptation and randomization, error rates tripled.
  • Distance-Dependent Performance: Accuracy improved with distance from the tower, as alignment errors (canting) become more distinct from surface deformations at greater ranges.
  • Robustness: The model demonstrated high accuracy in "double-extrapolation" scenarios, predicting flux for unseen sun positions and receiver geometries, indicating it has learned fundamental optical physics.

Conclusion and Future Work

The iDLR method provides a scalable path toward creating "digital twins" for entire heliostat fields. While the inverse problem remains partially underdetermined—leading to a 16% misprediction rate where different surfaces produce similar spots—the overall 26% improvement in flux prediction accuracy is significant. Future work aims to close the remaining performance gap and integrate these models into autonomous flux control and predictive maintenance systems for next-generation solar plants.

Summary updated

Abstract

Concentrating Solar Power (CSP) plants are a key technology in the transition toward sustainable energy. A critical factor for their safe and efficient operation is the accurate distribution of concentrated solar flux on the receiver. However, flux densities from individual heliostats are highly sensitive to surface imperfections, such as canting and mirror deformations. Measuring these surfaces across hundreds or thousands of heliostats remains impractical in real-world deployments. As a result, control systems often assume idealized heliostat surfaces, leading to suboptimal performance and potential safety risks. To address this, inverse Deep Learning Raytracing (iDLR) has recently been introduced as a novel method for inferring heliostat surfaces from target images of focal spots captured during routine calibration procedures. However, until now, iDLR had only been demonstrated in simulation. In this work, we present the first successful Sim-to-Real transfer of iDLR, enabling accurate surface predictions directly from real-world target images. Remarkably, this was achieved through a zero-shot Sim-to-Real transfer, in which the model is trained exclusively with simulated flux density data and applied directly to real target images of heliostat focal spots without the need for additional training on real target images. We evaluate our method on 63 heliostats under real operational conditions. iDLR surface predictions achieve a median mean absolute error (MAE) of only 0.17 mm and show good agreement with deflectometry ground truth in 84% of cases. When used in raytracing simulations, it enables flux density predictions with a mean accuracy of 90% compared to deflectometry over our dataset, and outperforms the commonly used ideal heliostat surface assumption by 26%. We tested this approach in a challenging double-extrapolation scenario, involving unseen sun positions and receiver projections, and found that iDLR maintains high predictive accuracy, highlighting its generalization capabilities. Our results demonstrate that iDLR is a scalable, automated, and cost-effective solution for integrating realistic heliostat surface models into digital twins. This opens the door to an optimized heliostat control, improved flux density distributions on the receiver, and ultimately, enhanced efficiency and safety in future CSP plants. • Deep learning model that predicts heliostat surfaces from target images. • Evaluation under operational conditions for 63 heliostats. • Median MAE of only 0.17 mm to Deflectometry. • Low-cost in-Situ method without new hardware and full automatization.
BibTeX
@inproceedings{lewen2025scalable,
  author = {Jan Lewen and Max Pargmann and Mehdi Cherti and Jenia Jitsev and Robert Pitz‐Paal and Daniel Maldonado Quinto},
  title = {{Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing}},
  booktitle = {Solar},
  year = {2025},
  doi = {10.1016/j.solener.2025.113726},
}
Figure