Shadows as Rulers: Recovering Cloud Height from Its Own Ground Shadow for Plant-Scale Solar Nowcasting

Solar power drops within seconds when a cloud edge crosses the sun. Camera-based forecasters have become good at predicting that drop at one point: the spot where the sky camera and its irradiance sensor sit. A solar plant, though, spans hundreds of meters to kilometers. What the grid operator needs is where and when the cloud's shadow will land on each row of panels. That is a cast-shadow problem in the computer-vision sense: to know which ground point a cloud shades, you need the cloud's height, and a single sky camera cannot see height.

This note proposes to measure that height with the shadow itself. A plant's own distributed irradiance or string-power sensors record the moment the shadow edge sweeps across each of them. The pattern of those arrival times fixes the cloud's height and ground speed. A differentiable shadow-projection layer, placed between a standard sky-image forecaster and per-sensor outputs, can therefore learn height without a stereo rig or ceilometer. It can then forecast a metric shadow map over the whole plant. In two sentences: current sky-image models work because they are only ever tested next to the camera, where cloud height does not matter. Away from the camera, height controls shadow timing, and the plant's own sensors can supply the missing height much as shadows reveal shape in photometric vision.

Core Claim

Computer vision has long treated shadows as a problem: they break the link between brightness and shape. The file frames this as a chicken-and-egg problem. Logothetis et al. state it directly: deciding whether a point is shadowed requires the geometry you are trying to recover. Solar nowcasting has the same problem in a simpler form. The unknown geometry is mostly one number per cloud layer, the cloud base height $h$.

The claim is threefold. First, the success of 2D image-plane forecasters on standard benchmarks hides a height error that grows with distance from the camera. Second, that error can be removed by supervising a height-conditioned projection with shadow arrival times at spatially spread ground sensors. Third, the resulting forecaster beats both 2D models given sensor coordinates and sensor-only graph models, at lead times longer than the plant's own crossing time.

A short geometric argument shows why height is invisible at the camera and decisive away from it. Consider a ground sensor displaced horizontally by $\Delta x$ from the camera. The sensor's sun-line meets the cloud layer $\Delta x$ away from the camera's sun-line. The camera sees that cloud point at an angular offset of roughly $\Delta x / h$ from the sun. A model that assumes height $h + \delta h$ looks at the wrong cloud patch. For a cloud moving at ground speed $v$, this produces a shadow-arrival timing error of

$$\tau \approx \frac{\Delta x}{v}\,\frac{\delta h}{h},$$

This follows from the small-angle projection above: the position error in the cloud layer is $\Delta x\,\delta h/h$, divided by speed. At the camera ($\Delta x = 0$) height errors cost nothing, which is why the models in the file never needed height.

For concrete numbers, take a sensor 1 km away, a 30% height error, and a 10 m/s cloud. The result is about 30 s of ramp-timing error. That is comparable to the entire lag that ECLIPSE was built to remove. The same argument shows that ground speed is not observable from one camera: the camera measures angular speed $\omega$, and $v = \omega h$. Height and velocity are therefore one unknown, and shadow arrival times across sensors pin both down.

Research Program

Base forecaster. We keep a standard image forecaster as the backbone. A spatio-temporal encoder in the style of ECLIPSE predicts future sky frames and cloud/sun segmentation masks. The only change is that its outputs are expressed on the hemispherical sky dome rather than regressed directly to a scalar.

Projection layer. A differentiable projection layer maps the predicted cloud-opacity field on the dome to a ground shadow map. It does this by casting a ray from each ground point toward the sun and intersecting it with one or two horizontal cloud layers, each at a latent height. This is the flat-layer version of the ground-projection equation in the file. It is deliberately much cheaper than the volumetric ray tracing in SSD-GS, whose authors note that shadow rays through dense volumes are costly.

Height prediction. Layer heights are predicted per frame by a small head from image features. They are trained only through the loss on per-sensor irradiance. A clear-sky normalization, borrowed from Gao et al., who showed that predicting only the residual from a clear-sky curve helps, keeps the network focused on occlusion. A soft-edge term models the penumbra so that gradients flow across shadow boundaries.

Output and downstream use. The output is a forecast shadow map. From it we read per-sensor irradiance, per-string power (through a simple bypass-diode mismatch model), and plant-aggregate ramp rate. As a secondary product, shadow-edge speed and covered fraction are passed as features to a conditional value-at-risk reserve rule following Dall'Anese et al.. This tests whether geometric shadow features shrink the forecast-error tails that the rule prices. We treat this last step as an application check, not the core contribution.

Role of each prior work. ECLIPSE, SIAT and PhyDNet-dual motivate the work. All three regress to a point sensor next to the camera, so they never face the height problem, and they supply our backbone. Volumetric Cloud Field Reconstruction supplies the stronger-geometry reference and synthetic data. It recovers cloud base height with a three-pair stereo rig over 10 km × 10 km, but never casts shadows or forecasts power. We ask whether a single camera plus the plant's own sensors gets most of that benefit. Its rigid horizontal advection assumption constrains us too: we inherit it over short horizons and measure where it breaks. MDCN-PS supplies the methodological lesson that a coarse geometric prior regularizes shadowed, light-starved regions. Our geometric prior is supervised by shadow timing rather than by a pretrained depth network.

Baselines

Naive reference. The do-nothing point is smart persistence: the current clear-sky index held constant and applied at every sensor. On SIRTA, SIAT reports a smart-persistence 1-step root-mean-square error (RMSE) of 93.33 W/m² against SIAT's 76.94 W/m². We rerun smart persistence on every split so that forecast skill is computed against the same reference throughout.

Strongest image baseline. This is ECLIPSE retrained with each sensor's offset $(\Delta x, \Delta y)$ as an input and one output head per sensor. It is the fairest version of "2D models are enough," since the network is free to learn height implicitly. Published colocated numbers are ECLIPSE's 10-minute forecast skill of 24.0% on SIRTA and SIAT's 15-minute forecast skill of 21.45% on Chilbolton, where ECLIPSE scores 17.69% (all from SIAT and ECLIPSE). For displaced sensors, no reported result was found on Lacuna.

Sensor-only baseline. The spatiotemporal graph network of Karimi passes early warnings from upwind to downwind sites using sensors alone. No reported result was found on Lacuna for it at plant scale.

Oracle geometry. On synthetic data we use the flat-layer projection with ground-truth height. On real data, where available, we use ceilometer height, and on synthetic data we also use the stereo reconstruction.

Fairness and the bar for beating. All learned models share the backbone, input frames, training data and hyperparameter search budget. Beating the strongest baseline means two things on held-out days: higher per-sensor forecast skill at sensors farther than a few hundred meters, and lower ramp-arrival timing error. At colocated sensors we must stay within noise of the baseline.

Ablations. Three ablations isolate the key ingredient: freezing height at a climatological constant, predicting height from images with no ground-timing supervision, and using a single cloud layer versus two.

Why not just…?

Why not just give the 2D network the sensor coordinates and let it learn height implicitly? This is the main threat, and it is a baseline above. The argument against it is that height changes by the hour, and the image cues for it are weak. A network conditioned on offset would have to infer $h$ from texture on every frame, with no supervision tying it to geometry. The equation predicts that its timing error grows linearly with $\Delta x$. Ours should stay flat if the learned height is accurate. If the curves match, the explicit layer adds nothing, and we will say so.

Why not just use the sensor network, as in the spatiotemporal graph model? Sensors only warn about a shadow once it has reached the upwind edge of the plant, so the lead time is capped at crossing time. For a 1 km plant at 10 m/s that is about 100 s. The camera sees clouds kilometers away. We expect the graph model to win at very short horizons and lose beyond crossing time. Reporting skill as a function of horizon settles where the crossover lies. A hybrid that feeds graph features into our model is included.

Why not just buy a ceilometer or build a stereo rig? Both work. The ceilometer is our real-data oracle and the stereo reconstruction is our synthetic oracle, with 64.1% Jaccard similarity reported against 57.3% for VIP-CT in Volumetric Cloud Field Reconstruction. The question is how much of the oracle's gain comes from sensors a plant already has. A ceilometer also measures height at one point, while broken multi-layer skies vary. We report the gap to the oracle openly.

Why not just time-shift the colocated forecast by distance over wind speed? Wind speed at cloud level is exactly the unobserved $v = \omega h$, so this reduces to our frozen-height ablation, plus a forecast wind that we also test.

Proposed Experiments

Synthetic experiments. We first use the Terragen/Blender cloud scenes of Volumetric Cloud Field Reconstruction: 1,000 conditions with bases between 400 m and 4,000 m. We render a fisheye camera plus a grid of ground sensors up to 3 km away. This lets us measure height error, timing error against $\Delta x$, and the gap to oracle height, with ground truth known.

Real colocated check. The colocated benchmarks (SIRTA, Chilbolton, La Réunion) test only that the projection layer does not hurt the case where 2D models already work. We compare with the published figures above.

Real displaced-sensor test. The decisive real test needs a sky camera beside a plant with string- or inverter-level data and, ideally, a ceilometer. The file lists no such public dataset, so this requires acquiring or instrumenting a site. We treat it as a requirement, not an assumption.

Metrics. We report forecast skill against smart persistence at 2–20 minute horizons per sensor, the temporal distortion index used by ECLIPSE for ramp lag, ramp-arrival error in seconds, height error against ceilometer, and plant-aggregate ramp-rate error. Downstream, we report reserve violations under the conditional value-at-risk rule.

Decisive checks. The idea succeeds if three things hold together:

  1. The offset-conditioned 2D baseline's timing error rises with $\Delta x$ while ours stays near the oracle's.
  2. Removing ground-timing supervision brings back that rise.
  3. Our skill exceeds the graph model beyond the plant crossing time.

If the 2D baseline shows no distance-dependent error even on synthetic data, where height truly varies, the core claim is wrong.