Do Sky-Camera Solar Forecasts Fail on New Cameras Mostly at Long Horizons?

Solar plants increasingly forecast their own power output from ground-based sky cameras: fisheye or dome-mirror cameras that photograph the whole sky every few minutes, so that a model can see clouds approaching the sun. A new plant has no years of historical power logs, so operators hope to reuse a model trained on someone else's camera. Different cameras, however, render the same sky differently. They differ in lens projection, saturation around the sun, color response, and in fixtures such as the mechanical shadow band some cameras use to block the solar disk. This note proposes that the damage from swapping cameras is not spread evenly across forecast lead times. It should concentrate at the longer horizons, roughly one to four hours ahead. At those horizons the clouds visible in the current image carry little real information about the future, so whatever the model still takes from the image is more likely to be camera-specific appearance than cloud evidence.

The proposal has two parts. The first is a measurement. Gradient-based saliency, a technique that backpropagates the forecast to the input pixels to show which pixels moved it, is computed against a clear-sky photo taken by the same camera. The resulting maps are then projected into a shared sky coordinate system. Comparing them across two cameras that photograph the same sky at the same moment should predict, horizon by horizon and without labels from the new camera, where a transferred model will fail. The second part is an intervention. During training, the model is penalized when its saliency maps disagree across the two co-located cameras. The test is whether this agreement penalty extends cross-camera skill to longer horizons better than the obvious alternatives: masking known artifacts, adversarial invariance, or simply requiring the two cameras' predictions to agree.

Core Claim

The claim has three parts, and each can fail on its own. First, cross-camera error should grow with forecast horizon faster than same-camera error. Second, the per-horizon disagreement between saliency maps computed on two co-located cameras should rank-correlate with the true per-horizon transfer error on held-out cameras, which would make it a label-free reliability signal for commissioning a new site. Third, training the forecaster so that its evidence agrees across cameras should reduce long-horizon transfer error by more than requiring agreement of its outputs or internal features, because two cameras can agree on a forecast while each uses a different, camera-specific reason.

The motivation comes from Integrating Weather Foundation Model and Satellite to Enable Fine-Grained Solar Irradiance Forecasting. That paper fuses Himawari satellite imagery with a weather foundation model and uses Integrated Gradients after training to show a hand-off: satellite input drives forecasts up to about 6 hours ahead, and the weather model dominates beyond that. That paper treats attribution as a diagnostic on a single sensor family. This proposal asks whether the analogous hand-off in sky-camera models, from image evidence to clear-sky and solar-geometry covariates, is what breaks under a camera swap, and it turns attribution into a training signal.

What Is New Relative to the Closest Work

The closest work is SPIRIT. SPIRIT forecasts irradiance with a pretrained Vision Transformer plus clear-sky and solar-geometry covariates, and it transfers across the TSI880, ASI16 and SKIPP'D camera systems. It reports gains of "up to 70%" in zero-shot transfer and "up to 45%" across locations and tasks. It does not examine what its models attend to, and it does not break transfer error down by horizon. Short-term Solar Irradiance Prediction from Sky Images with a Clear Sky Model predicts residuals above a clear-sky baseline up to 4 hours ahead on TSI880 and ASI16 and inspects attention weights qualitatively. It does not study camera transfer. Solar Irradiance Anticipative Transformer shows attention concentrating on the solar disk, but only on 15-minute windows.

The obvious combination of these papers is "Right for the Right Reasons" applied to sky cameras, which file Gap 1 describes. Right for the Right Reasons penalizes input gradients on a hand-specified mask of irrelevant pixels, and on Decoy MNIST it keeps about 96% accuracy where an unregularized model drops to 55%. That is not the proposal here. Shadow bands, bezels and fixed infrastructure sit at known or geometrically computable positions, so a gradient penalty on them is nearly equivalent to masking those pixels before training, which is far cheaper. The camera differences that matter are the ones without a fixed location: sun-region blooming, color and dynamic-range response, and projection distortion. These cannot be specified as a mask. Our penalty needs no artifact mask at all. It requires only that two cameras viewing the same sky point to the same sky regions as evidence.

Research Program

The program uses a data property that the cross-device literature treats as rare. Paired acquisitions across devices are listed in the file as a major constraint, and the retinal feature-translation work depends on a dataset built for exactly that purpose. TSI880 and ASI16 are both operated at NREL's Solar Radiation Research Laboratory in Golden, Colorado. They are paired with the same irradiance measurements, so overlapping periods give two cameras viewing one sky. The first task is to verify and quantify the temporal overlap. If it is too thin, the agreement penalty falls back to weakly paired frames matched on time of day and cloud fraction, and that fallback is reported as such.

Each camera's pixels are mapped to sky coordinates, zenith and azimuth, using its known lens or mirror geometry. Saliency is computed with Integrated Gradients, which addresses the problem that plain gradients go flat in saturated units by integrating them along a path from a reference input to the actual one. The file names the choice of reference as an open problem for meteorological imagery. We take as reference a clear-sky image from the same camera at a nearby solar position. The attribution then answers a physically meaningful question: how did today's departure from a clear sky move the forecast? Static, camera-specific content largely cancels against that reference, which is why it is the right instrument for this study.

The forecaster follows the clear-sky residual design that both the clear-sky sky-image model and the modified autoregressive model found essential. It predicts the residual from the clear-sky irradiance curve at horizons $h$ from 10 minutes to 4 hours, using a shared image encoder with per-horizon output heads. Training on paired frames minimizes

$$ \mathcal{L} = \sum_{d \in \{T, A\}} \sum_h \ell\big(\hat r^{d}_h, r_h\big) + \lambda \sum_h \big\lVert \Pi_T(S^{T}_h) - \Pi_A(S^{A}_h) \big\rVert_1 . $$

Here $d$ indexes the TSI880 ($T$) and ASI16 ($A$) cameras, $\hat r^{d}_h$ is the predicted clear-sky residual at horizon $h$ from camera $d$, and $r_h$ is the measured residual. $\ell$ is the forecast loss. $S^{d}_h$ is the attribution map for horizon $h$, computed against the same-camera clear-sky reference, and $\Pi_d$ projects it onto a common sky grid and normalizes it to unit mass. $\lambda$ sets the strength of the agreement penalty. Full Integrated Gradients inside the training loop would be expensive. We therefore use a cheap approximation during training, gradient times the difference between the input and its clear-sky reference, and use full Integrated Gradients only for evaluation. The Right for the Right Reasons timing result of 0.00052 seconds per gradient explanation suggests that a single-step gradient penalty is affordable, although that figure came from a much smaller model.

The diagnostic part of the claim is tested separately. For any model, we compute the per-horizon attribution disagreement on the co-located pair and correlate it with the per-horizon error on cameras the model never saw.

Baselines

The do-nothing reference is smart persistence, which assumes the current ratio of measured to clear-sky irradiance holds into the future. Every skill score is reported relative to it, following the convention of Solar Irradiance Anticipative Transformer. A second naive reference is an image-free clear-sky residual MLP or temporal convolutional network that uses irradiance history and solar geometry only. What Matters in Deep Learning for Time Series Forecasting? shows such simple models are hard to beat: on the Solar benchmark, iTransformer improves from 0.208 to 0.194 MSE once its spatial attention is removed. This baseline also sets the floor below which images are not worth using at a given horizon.

The strongest existing method is SPIRIT, a frozen Vision Transformer encoder with clear-sky and solar-geometry covariates. We rerun it on our splits rather than quoting it, because its reported numbers are relative improvements ("up to 70%" zero-shot, "up to 45%" cross-location) and the file gives no absolute errors on these splits. The other published reference points are the clear-sky sky-image model's nowcasting error of 6.9% normalized mean absolute percentage on ASI16, against 13.1% for earlier methods. SIAT reports 21.45% forecast skill at 15 minutes on Chilbolton and 76.94 W/m² RMSE against 93.33 for persistence on SIRTA. These anchor the short-horizon end only. No reported result was found on Lacuna for cross-camera error at horizons beyond one hour.

The remaining comparisons each remove or replace one ingredient. One baseline masks or inpaints the shadow band, bezel and fixed infrastructure before training. Another adds a domain-adversarial discriminator on camera identity, following the adversarial component of Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound. A third requires agreement of predictions and of pooled encoder features across the paired cameras, which is the critical ablation because it uses the same pairing without the attribution term. A fourth adds a learned horizon gate that down-weights the image branch at long horizons. Finally, the attribution penalty itself is run with a zero-image reference instead of the clear-sky reference, to test whether the reference choice matters.

All methods share the encoder initialization, training frames, number of epochs and a fixed budget of hyperparameter trials, tuned on a validation split from the source cameras only. Our method wins if, across three seeds, it reduces held-out-camera error at horizons of one hour and beyond relative to both SPIRIT and the paired prediction-and-feature consistency baseline, by more than the seed-to-seed spread, while staying within seed noise of the best baseline at 10 to 30 minutes.

Why not just…?

Why not just mask the shadow band and lens edges? For the artifacts masking can describe, this is the correct answer, and it is included as a baseline rather than argued away. It does nothing about sun-region saturation, color response or projection distortion, because these are not confined to fixed pixels. The proposal's claim is specifically that what remains after masking is what drives long-horizon transfer error. If masking alone closes the cross-camera gap at all horizons, that claim is false and we will report it.

Why not just normalize and augment the images? Histogram equalization, color jitter and undistortion are cheap and should be part of every pipeline, so all methods, including ours, receive them. The medical-imaging evidence in the file indicates that augmentation leaves hardware signatures in place. In the ultrasound study, an augmented standard network still showed 3.56 mm mean surface distance on an unseen vendor, which fell to 2.01 mm only with explicit priors. That is a different modality, so the augmented SPIRIT baseline is the real test here.

Why not just use adversarial domain adaptation? A discriminator on camera identity removes whatever separates the cameras. Some of that information may be physical, since the shadow-band and unmasked cameras capture circumsolar brightness differently. The file documents adversarial stripping discarding critical signal in ultrasound and washing out weak signals in FlowPET. Our penalty does not ask the representations to be indistinguishable, only that the cited sky regions coincide. The domain-adversarial baseline measures whether this distinction matters in practice.

Why not just make the paired cameras' predictions agree? This is the most serious threat, because it uses the same pairing more simply. Two cameras can produce the same forecast for different, camera-specific reasons, and only the reason fails when a third camera arrives. That argument is a hypothesis, not evidence. The paired consistency ablation settles it directly: if it matches our method on held-out cameras, the attribution term is unnecessary and the contribution shrinks to the diagnostic.

Why not just drop the image beyond some horizon? If images carry no transferable signal at three hours, a gate that switches to the image-free residual model is the honest solution. The image-free baseline and the learned gate measure exactly this. The proposal is useful only at horizons where the image still beats the image-free model on the source camera. A gate trained on one camera keeps any image cue that helps in-distribution, so it may not discard the non-transferable ones.

Proposed Experiments

The clean, device-only test trains on one NREL camera and evaluates on the other, in both directions, across horizons from 10 minutes to 4 hours. We plot same-camera and cross-camera skill against horizon. The first part of the claim fails if the gap between the two curves is flat across horizons.

The broader transfer test trains on the TSI880 and ASI16 pair and evaluates on Chilbolton and SIRTA irradiance, on La Reunion's fisheye camera, and on SKIPP'D photovoltaic power. In these datasets the camera, the climate, and in SKIPP'D's case the target all change at once. We report them as combined site-and-camera transfer and do not claim device effects from them alone. La Reunion's 10-second imagery and SKIPP'D's 1-minute imagery are resampled to the 10-minute cadence.

The decisive diagnostic check is the Spearman rank correlation, across all trained models and horizons, between the label-free attribution disagreement on the NREL pair and the true error on held-out cameras. We compare it against two label-free alternatives that can be computed the same way: prediction disagreement on the pair, and the norm of the feature shift between cameras. The diagnostic matters only if it ranks reliability better than both.

The decisive intervention check compares our method with the paired prediction-and-feature consistency baseline at horizons of one hour and beyond. We also inspect saliency maps under the clear-sky reference to see whether the long-horizon evidence moves from the bright region around the sun toward upwind cloud fields.

A smaller follow-on applies the same diagnostic to the satellite–weather-model hand-off in the Himawari study. It asks whether attribution placed on satellite input beyond 6 hours flags sensitivity to a satellite sensor change. That step depends on data access and is not required for the main conclusions.