Lags in Seconds, Not Samples: Rate-Portable Dilated Convolutions for Time Series

A dilated convolution over a time series reads the input at fixed integer offsets, for example 1, 3, 5 and 9 steps back. Each output is therefore a weighted sum of lagged copies of the signal: it is lagged feature engineering with learned weights. Because those offsets are counted in samples, a trained model is tied to the sampling grid it was trained on. Feed it the same physical process recorded every 45 minutes instead of every hour, or with missing and jittered timestamps, and the lags point to the wrong moments in physical time. The usual fix is to resample the data onto the training grid. That fix is itself a discretization step, and it throws information away.

We propose a multi-scale dilated convolution whose tap positions are learnable real-valued delays measured in physical time. It reads the input at arbitrary past times by interpolating between the observations that actually exist. Our main claim is narrow. On a regular grid at the training rate, this layer should not beat a dense convolution, since a fractional lag on uniformly sampled data is just a mix of neighbouring integer lags. Its real advantage should appear when train and test grids differ by non-integer ratios, or when sampling is irregular. In those settings, integer-dilation models must resample their input, and continuous-delay models should not.

Core Claim

A single layer computes, for each output channel, $y(t) = \sum_{k=1}^{K} w_k \, \hat{x}(t - \tau_k)$. Here $w_k$ is the weight of tap $k$, and $\tau_k > 0$ is its learnable delay in physical units such as minutes. $\hat{x}(s)$ is the input evaluated at time $s$ by interpolating the observed samples (linear interpolation, or a narrow Gaussian as in the DCLS follow-up). Delays in different channel groups are initialised log-spaced across scales, as in multi-scale dilated networks, and are then trained by gradient descent.

Because $\tau_k$ is in physical time, the same trained parameters can be applied at any sampling rate or to irregular timestamps without retraining or resampling. When the delays happen to sit on integers of the training grid, the layer reduces exactly to a sparse discrete autoregressive filter.

The claim to test has two parts. First, at the native rate the layer matches the strongest integer-dilation convolution while using far fewer taps. Second, under non-integer rate shift and irregular sampling it degrades much less than integer models fed resampled input. If the first part holds but the second fails, the idea has no reason to exist beyond sparsity.

How This Sits Against Prior Work

Direct parents. Two papers supply the method. Multi-Scale Dilated Convolution Network for Long-Term Time Series Forecasting (MSDCN) contributes the architecture. It runs parallel long-kernel and short-kernel dilated branches with dilations $2^i+1$, plus a linear autoregressive skip, and its authors name fixed dilations as a limitation to address. Dilated convolution with learnable spacings (DCLS) contributes the mechanism. It makes 2D kernel positions continuous through bilinear interpolation, and its extension beyond bilinear interpolation supplies smoother interpolants.

The contrast with DCLS matters. DCLS interpolates the kernel onto a fixed pixel grid, so its benefit on images is optimisation and inductive bias, not access to off-grid inputs. We instead interpolate the signal at physical times. That is what makes the layer usable across sampling rates and irregular timestamps. In the file, DCLS is evaluated only on ImageNet, COCO and ADE20K.

Motivation, and a qualification of it. Generalizing Multi-Scale Time-Series Modeling with a Single Operator (SiGMA) proves that integer-window scaling incurs an irreducible "quantization gap" in forecastability. It closes the gap with a single continuous Gaussian smoothing operator that costs $O(L^2)$ in sequence length $L$. We take its theorem as motivation but narrow its reach. On a uniform grid, a dense filter can already express any fractional lag. The gap therefore bites hardest when the evaluation grid differs from the training grid, and that is where we test it. Unlike SiGMA, we learn many sparse lags rather than one smoothing scale, and the cost is linear in $L$ times the number of taps.

Constraint. Effectively Modeling Time Series with Simple Discrete State Spaces (SPACETIME) shows that continuous-time and diagonal state-space models cannot represent simple discrete AR(p) processes. This could look like an argument against any continuous formulation. Our layer avoids the objection by construction: integer delays recover AR(p) exactly. We also keep MSDCN's linear AR skip, so that discrete recurrence never has to be learned through interpolation.

Lag-based models with hand-chosen lags. Lag-Llama picks lags by calendar heuristics. A Time-Series Foundation Model by Universal Delay Embedding (UDE) builds Hankel matrices, which stack many lagged copies of the series, and admits that embedding dimension and patch size are tuned empirically. Our layer can be read as a learned, sparse delay embedding that replaces that tuning. We do not compete with either model as a foundation model.

Closest continuous-delay model. Modeling Irregular Astronomical Time Series with Neural Stochastic Delay Differential Equations (Neural SDDE) handles irregular sampling with a single fixed delay $\tau$ inside an SDE solver, using adjoint gradients. We use many learned delays in a feed-forward convolution with no solver.

Research Program

We will build a 1D continuous-delay convolution as a drop-in replacement for MSDCN's dilated modules, keeping MSDCN's branch structure, AR skip and Huber loss so that the only change is how taps are placed and read. The forecasting experiments stay channel-independent like MSDCN. Cross-channel learned delays, the kind of delayed coupling targeted in Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates, are a secondary extension. They are reported only if the main result holds.

The implementation follows the depthwise strategy DCLS uses for throughput. Each tap gathers from the two (linear) or few (Gaussian) observations bracketing $t-\tau_k$, so cost scales with the number of taps rather than the receptive field.

Training includes an optional rate augmentation: each batch is randomly subsampled at a ratio drawn from a range, with timestamps passed through. This augmentation is applied identically to the baselines that can accept it, so that we do not confuse "continuous delays help" with "training on varied grids helps."

We will also analyse what the learned $\tau_k$ converge to. Possibilities include known periods such as 24 h and one week, Takens-style delays near the first minimum of autocorrelation, or neither. This analysis tests whether learned delays answer UDE's "how to choose $m$ and $\tau$" question in practice.

Baselines

Naive reference. The do-nothing reference is the seasonal-naive forecast (repeat the value one period back) and last-value carry-forward. Any model that fails to beat these under rate shift is broken, not merely weak.

Strongest integer-dilation convolution. This is MSDCN, run from its published configuration on ETT, Electricity, Weather and Traffic, with input resampled by linear interpolation onto its training grid whenever test data arrive at another rate. MSDCN reports the best result in 80% of tasks across eight benchmarks and a 9.4% MSE reduction against ETSformer. The file contains no per-dataset, per-horizon MSE values for MSDCN, so we reproduce it ourselves and report our own numbers alongside the paper's aggregate claims. Beating it at native rate is not required; matching within roughly one standard deviation over five seeds, with fewer taps, counts as success. Under non-integer rate shift, beating it means a consistent MSE gap larger than seed variance at every shifted rate.

Dense large-kernel convolution. A dense depthwise convolution with the same physical receptive field and every integer tap is the strongest expressivity control at native rate. It is also the main skeptic's model. Under rate shift it is run with resampling, like MSDCN.

Continuous-scale and patching references. SiGMA represents the continuous-scale alternative. PatchTST (A Time Series is Worth 64 Words) is the standard patch Transformer, which splits the series into fixed-length chunks.

Irregular-sampling reference. On PLAsTiCC the competitor is Neural SDDE. It reports 0.656 accuracy in the 50%-missing "Fully Irregular" setting, against 0.509 for a GRU, and a novelty-detection AUROC of 0.707.

Fairness rules. All models get the same input lengths, the same splits, the same per-model hyperparameter search budget of 20 trials on validation, the same rate augmentation when applicable, and parameter counts within a factor of two.

Ablations isolating the key ingredient. The first ablation freezes delays at their integer initialisation, which yields a sparse fixed-dilation convolution. The second rounds learned delays to the nearest integer after training. The third compares linear with Gaussian signal interpolation. If rounding costs nothing under rate shift, continuous delays are not the cause of any gain.

Why not just…?

Why not just use a dense kernel that covers every integer lag? On a regular grid at the training rate, this is the real threat. A fractional delay on uniform samples is exactly a weighted pair of adjacent integer taps, so a dense kernel loses nothing in expressivity. We concede this and do not claim native-rate accuracy gains; there we expect at best a tie with fewer parameters. Its weakness shows when the test grid differs, because its taps are defined in samples and its input must be resampled. The rate-shift experiment measures directly whether that resampling costs anything.

Why not just rescale integer dilations by the rate ratio? For integer ratios such as hourly ETTh versus 15-minute ETTm (a factor of four), rescaling is exact, and this baseline would tie us. That is why the decisive tests use non-integer ratios such as 45 and 75 minutes against a 60-minute training grid, where rescaled dilations must be rounded. We include the rescaled MSDCN as a baseline. If rounding to the nearest integer barely hurts it, the honest conclusion is that the quantization gap is negligible on these benchmarks.

Why not just resample everything to one grid, or retrain per rate? Retraining per rate is the upper bound, and we report it. Resampling is the practical default and is what our baselines do. The question is purely empirical: does interpolating the input before an integer filter lose more than interpolating inside a learned continuous filter? The fixed-integer and rounding ablations, run with identical rate augmentation, answer it.

Why not just use a continuous-time ODE or SDE model for irregular data? Neural SDDE already handles irregular light curves well. It does so with one fixed delay and with solver cost, the limitations its own authors note. PLAsTiCC tests whether many learned delays in a solver-free convolution can match its reported 0.656 accuracy at a fraction of the training time. Falling short there would limit our claim to forecasting.

Proposed Experiments and Decisive Checks

Native-rate check. We train on ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Weather and Traffic at standard horizons (96 to 720), reporting MSE and MAE over five seeds, along with tap count and Traffic inference time. MSDCN reports 3.68 s for Traffic inference. The goal is parity with MSDCN and the dense kernel.

Decisive check 1: non-integer rate shift. We train on ETTm1 and ETTm2 subsampled to 60 minutes, then test without retraining at 45, 75 and 90 minutes, all derived from the 15-minute original so that ground truth is exact. We repeat this on Weather by building 10-, 25- and 35-minute grids from the 10-minute data. The prediction is that the continuous-delay model's error grows much more slowly with $|\log(\text{rate ratio})|$ than resampled MSDCN, rescaled MSDCN, or the dense kernel. We also predict that this gap disappears when learned delays are rounded to integers.

Decisive check 2: irregular sampling. We randomly drop 30–50% of timestamps in ETT and Weather test inputs and compare against interpolate-then-forecast baselines. On PLAsTiCC Fully Irregular, we report accuracy and novelty AUROC against Neural SDDE's 0.656 and 0.707.

Interpretive check: learned delays. We cluster learned delays across seeds and datasets and compare them with dominant spectral periods and autocorrelation minima. Stable delays that align with these would support reading the layer as a learned delay embedding. Delays that wander across seeds would suggest the gains, if any, come from interpolation smoothing rather than lag discovery.

Kill criterion. If, under the 45- and 75-minute shifts, rescaled-and-rounded MSDCN comes within seed variance of our model, we will report that the quantization gap is practically negligible for these benchmarks. We would treat that as a useful negative result against over-reading SiGMA's theorem in delay space.