Filling in the missing hours of a weather or climate field is not like filling in the missing frames of a home video. In a video, a moving car looks the same whether it sits in the left or right half of the image. In a geophysical field, absolute position and absolute time shape what happens. Surface solar radiation between two snapshots depends on where the sun is relative to each grid cell. Wind and temperature evolve differently over the Arctic coast than over open ocean. Ordinary convolutional networks apply the same filter everywhere, so they are built to ignore exactly this information. Current deep temporal interpolators for such data are translation-invariant 3D convolutional networks, and they can only produce frames at fixed sub-steps.
We propose a convolutional interpolator that is told where and when it is at every layer. Each feature map is stacked with extra input planes: seam-free spherical position, the signed time offset from each context frame to the requested moment, and absolute clock coordinates such as local solar time and day of year. The network is a 3D U-Net that takes several irregularly spaced context frames and can be asked for the field at any continuous time inside the gap. The central claim is falsifiable. Coordinates that carry physical meaning (spherical position, local solar time) should let the interpolator generalize to geographic regions it never saw during training. Flat image-index coordinates of the kind used by the original CoordConv should help only on regions seen in training, or should even hurt on new ones.
Core Claim and Position Relative to Prior Work
The coordinate-channel trick comes from An intriguing failing of convolutional neural networks and the CoordConv solution. That paper showed that stacking normalized $x,y$ grids onto a convolution's input lets the network learn position-dependent mappings. On a held-out-quadrant split, the coordinate network reached 100% accuracy with 7,500 parameters, while a standard network with 200,000 parameters reached 86%. This result motivates our work, but its coordinates are flat, static, and tied to image indices. It also contains a constraint: on ImageNet, where position does not matter, coordinate channels gave +0.04%. Coordinates help only when the target actually depends on position, so our benchmark choice must ensure that it does.
On the interpolation side, STint is the closest working system and our main competitor. It trains a 3D U-Net with cycle-consistency losses and does not rely on optical flow, which is the per-pixel motion estimate used by video methods. On IPSL wind fields it beats the flow-based SuperSloMo, 31.17 dB to 29.71 dB in PSNR. However, its convolutions do not know absolute position, and it only produces frames at fixed sub-steps, typically doubling the frame rate. GIMM supplies the evidence that spatial coordinates matter for continuous-time interpolation. Its ablations show that querying a small network with $(x,y,t)$ beats querying with $t$ alone. GIMM routes this through a 0.25M-parameter coordinate network fed by a pretrained flow estimator and uses only two frames, and STint's results suggest flow estimators break on fluid motion.
Deep Random Features constrains our design. It shows that flat latitude/longitude inputs create seams at the date line and singularities at the poles, and it fixes this with spherical-harmonic features in a pointwise kernel model that does not process gridded tensors. CoBSI supplies a design cue. It gives time and space separate frequency budgets in its positional encodings, because a seismic wavefield varies at different rates along each axis. It is, however, retrained from scratch on every survey.
Our proposal differs from all of these in one specific way. It puts physically grounded spherical and clock coordinates, together with a continuous query time, inside every scale of a convolutional hierarchy. It trains one amortized model across many regions and years rather than per instance, and it tests the coordinates on held-out geography rather than held-out frames.
Research Program
The model is an STint-style 3D U-Net over a stack of $K$ context frames, with $K$ between 2 and 6, observed at irregular times $t_1,\dots,t_K$, and a query time $\tau$. At every resolution level, the feature tensor is concatenated with three groups of planes.
The first group is spherical position, written as the unit vector $(\cos\phi\cos\lambda, \cos\phi\sin\lambda, \sin\phi)$, where $\phi$ is latitude and $\lambda$ longitude. This representation is continuous across the date line and well behaved at the poles, which carries the lesson of the Deep Random Features paper over to gridded convolution.
The second group is relative time: for each context frame, a constant plane holding $(t_k-\tau)$ scaled by the typical gap length. The network therefore knows how far each observation lies from the requested moment, and $\tau$ can be any real value.
The third group is absolute clock time: $\sin$ and $\cos$ of local solar time $(\mathrm{UTC}(\tau)+\lambda/15)$ and of day of year. These form spatially varying planes because local solar time depends on longitude.
Following CoBSI, the clock planes use a few low-frequency harmonics, while the relative-time planes stay linear. Training is supervised by withholding real frames. ERA5 and CARRA are hourly, so we subsample them to irregular 3–12 hour gaps, draw $\tau$ uniformly inside each gap, and regress the withheld hour. We keep STint's cycle loss only as an optional regularizer, because its authors report it destabilizes training.
The analysis has two parts. First, we zero out each coordinate group at test time and record which errors rise, broken down by latitude, local hour, and gap length. Second, we fit a linear probe from intermediate features to solar zenith angle, to check whether the network rebuilds that angle from the generic coordinates.
Baselines
The naive reference is per-pixel linear interpolation in time between the two nearest observations. It is what operational pipelines often use, as in the crop-mapping preprocessing of BreizhCrops. Any learned model must clearly beat it at every gap length. We found no reported result on Lacuna for linear interpolation on ERA5, CARRA or IPSL. The only reported linear-blending figure is 30.09 dB PSNR on video (Vimeo-Septuplet-Flow 6X, from GIMM), which does not transfer to geophysical data, so we will measure linear interpolation ourselves.
The flow-based reference is SuperSloMo. STint reports 29.71 dB PSNR for it on IPSL wind. It is included because arbitrary-time video interpolators share its reliance on optical flow, and we will rerun it on our splits.
The strongest existing method is STint, which reports 31.17 dB PSNR on IPSL wind. STint reports no ERA5 or CARRA numbers on Lacuna beyond qualitative superiority, so no reported figure exists for those datasets. We will reimplement STint with the same backbone width, the same multi-frame input, the same data, and the same tuning budget of 20 trials per method over learning rate and loss weights. As a sanity check, we will confirm that our reimplementation gets close to 31.17 dB on IPSL. Because STint cannot query arbitrary times, we evaluate it only at its native midpoints. We also run a variant that receives the relative-time planes alone, so that the continuous-time ability is credited separately from the coordinates.
The ablation that isolates our key ingredient is STint plus flat CoordConv planes, meaning image-index $x,y\in[-1,1]$ and optionally radius, as in Broadcasting Convolutional Network. This is identical to our model except for the coordinate content.
The oracle baseline feeds a computed solar-zenith-angle plane, the hand-engineered physics feature.
Beating these baselines means three things. Our model must exceed STint by a margin larger than the spread across three seeds at its native midpoints. It must exceed the flat-CoordConv ablation on held-out longitude bands. On solar radiation, it must come within that seed spread of the zenith oracle without being given the zenith angle.
Why not just…?
Why not just add standard CoordConv channels to STint? This is the most serious threat. Within a fixed regional domain such as CARRA's Arctic crop, flat image indices stand in one-to-one for geography, so on random splits they may match our coordinates. That is why the decisive test holds out geography. Flat indices let the network memorize behaviour cell by cell, which the CoordConv paper's own quadrant experiment suggests is what convolutions do with positional cues. Spherical and local-time coordinates instead describe a held-out region in the same terms as the training regions. If flat CoordConv matches us on held-out longitude bands too, the claim is false, and we would report coordinate injection as a generic, minor gain.
Why not just feed the solar zenith angle as an input? For solar radiation this probably works well, which is why it appears as the oracle baseline. It answers a narrower question, though. No comparable closed-form feature exists for wind or surface temperature, whose position dependence comes from terrain, coastlines, and the diurnal cycle together. The test is whether generic spherical and clock coordinates recover the oracle's gain on solar radiation and also produce gains on IPSL wind and CARRA temperature, where no oracle exists. If the gains appear only on solar radiation, the honest conclusion is to use hand-built physics features.
Why not just use a continuous coordinate network such as GIMM or a per-field implicit representation? GIMM depends on optical flow, and STint's SuperSloMo comparison shows flow-based warping tears on non-rigid fields. Per-instance implicit representations like CoBSI must be retrained for every field, which is impractical across decades of hourly global data. We will still run GIMM with its flow backbone on our arbitrary-time splits. If it wins, it would show that flow failure is not the bottleneck we assume.
Proposed Experiments
Data come from ERA5 downward solar radiation, CARRA Arctic surface temperature, and IPSL 3-hourly wind, the three datasets on which STint was evaluated. Splits run by year (train on earlier years, test on later ones) and by geography. For ERA5 we hold out a 60° longitude band that includes the date line, and for CARRA a contiguous sub-region. This is a geophysical version of CoordConv's quadrant split.
Metrics are PSNR and SSIM for comparison with STint, and RMSE in physical units (W/m², K, m/s). Each is reported as a function of gap length and of query position within the gap. Two decisive diagnostic curves accompany them: error near the date line and above 80° latitude, which tests whether the spherical encoding removes seams, and solar-radiation error as a function of local solar hour, where a coordinate-blind network should fail most at sunrise and sunset.
The proposal succeeds if three things hold. Physical coordinates must beat flat coordinates on held-out longitude while tying them on random splits. The continuous-time model must stay ahead of linear interpolation as gaps grow from 3 to 12 hours. Generic coordinates must close most of the gap to the zenith oracle on solar radiation and still help on wind and temperature.
References
- An intriguing failing of convolutional neural networks and the CoordConv solution
- STint: Self-supervised Temporal Interpolation for Geospatial Data
- Generalizable Implicit Motion Modeling for Video Frame Interpolation
- Deep Random Features for Scalable Interpolation of Spatiotemporal Data
- Coordinate-Based Seismic Interpolation in Irregular Land Survey: A Deep Internal Learning Approach
- BREIZHCROPS: A TIME SERIES DATASET FOR CROP TYPE MAPPING
- Broadcasting Convolutional Network for Visual Relational Reasoning