Factor Out the Weekly Cycle Before Training Traffic Forecasters Across Regions

Traffic in every city follows a weekly clock: the morning rush, the evening rush, and quiet weekends. When a traffic forecasting model is trained jointly by several regional operators, each training on its own sensors and syncing only occasionally over slow links, that clock does two kinds of damage. Each region has its own rush-hour shape, so the models drift apart. And all regions hit rush hour at the same wall-clock time, so their drift points the same way and averaging does not cancel it. We propose a simple split. Each region keeps a sensor-owned table of time-of-week averages that it fits locally and never sends anywhere. A single shared neural network learns only what is left over (incidents, propagation of congestion along roads, short-term deviations), and it is synchronized rarely and with compressed messages. The claim is that most of the cross-region disagreement lives in the periodic part. Once that part is taken out of the shared weights, the methods that save communication by syncing rarely or sending compressed updates should lose much less accuracy than they do on raw traffic.

The opportunity sits where two literatures miss each other. Traffic forecasting has found that separating periodic structure from residual fluctuation is the single most important modelling decision. A historical-average-plus-linear-regression baseline matches deep models on several benchmarks (Rodrigues), and decoupled architectures that model the periodic part and the residual part separately improve both accuracy and cost (HyperD; DGST). Communication-efficient training has shown that infrequent synchronization works on language models with shuffled, identically distributed data (DiLoCo; MT-DAO). Its behaviour when workers hold structurally different, time-correlated data is listed in the file as largely uncharacterized. Traffic gives that heterogeneity a known form, and that is what makes it removable.

Core Claim

Write the forecast for sensor $s$ at time $t$ as

$$\hat y_{s,t} = P_s[\tau(t)] + f_\theta\big(x_{\mathcal{N}(s),\,t-L:t} - P_{\mathcal{N}(s)}[\tau(\cdot)]\big),$$

where $\tau(t)$ is the time-of-week slot (2016 slots at 5-minute resolution), $P_s$ is a per-sensor table of historical averages held only by the region that owns sensor $s$, $x_{\mathcal{N}(s),\,t-L:t}$ is the last $L$ observations of $s$ and its road-graph neighbours $\mathcal{N}(s)$, and $f_\theta$ is a shared spatiotemporal network with weights $\theta$ trained on de-cycled inputs. We will measure cross-worker drift as

$$D(H) = \frac{1}{K}\sum_{k=1}^{K}\lVert \theta_k^{(H)} - \bar\theta^{(H)} \rVert^2,$$

where $K$ is the number of regions, $\theta_k^{(H)}$ is region $k$'s replica after $H$ local steps from a common start, and $\bar\theta^{(H)}$ is their mean. The hypothesis has two parts. First, factoring out $P$ reduces $D(H)$ substantially at every $H$, and in particular removes the part of the drift that is shared across workers in phase with the clock. Second, this reduction makes infrequent synchronization and compression work: at a fixed accuracy target, the factored model tolerates a larger $H$ and a lower bit budget than an unfactored model or a model with a generic local head.

The construction also addresses a tension the file identifies. The correlation-driven federated method for cellular traffic (Gradient Compression and Correlation Driven FL) argues that heterogeneous traffic needs personalized, correlation-weighted aggregation. Agarwal et al. show that moving away from symmetric All-Reduce scales poorly. In our design, personalization lives entirely in tables that are never communicated. The shared $\theta$ uses plain All-Reduce of compressed outer updates, so no peer-to-peer weighting is needed.

Research Program

The first step is diagnostic and is worth publishing on its own. We train a standard spatiotemporal forecaster in the region-partitioned setting and record per-worker update vectors over local intervals. We then ask how much of the cross-worker update variance is explained by the time-of-week slot of the minibatch, and how much of the worker-mean update oscillates with daily and weekly periods. If the periodic share is small, the core claim fails early and cheaply.

The second step builds the factored trainer. Regions fit $P_s$ in closed form from their training history. Because a pure historical average goes stale under drift, a known weakness of Rodrigues, regions also update the tables locally with a slow exponential moving average during training. This is free in communication. For the shared network we use a residual module in the style of HyperD's frequency-aware residual branch (HyperD), with neighbourhood graph aggregation, so that the same $\theta$ applies to graphs of different sizes. Synchronization uses DiLoCo-style outer steps, consisting of $H$ local steps followed by averaging and an outer Nesterov update (DiLoCo). It is optionally combined with MT-DAO's slow-momentum anchor, which addresses fast decay of local Adam momentum (MT-DAO). Outer updates are compressed with either 4-bit weight-difference quantization after a Hadamard transform (SDP4Bit) or DCT top-$k$ momentum sparsification (DeMo). We reuse these techniques as they are. The contribution is not a new compressor. It is the claim, and the test, that removing the cycle from the trained parameters is what makes them usable on traffic.

The third step analyses one specific interaction: when $H$ steps of local training span a fraction of a day, does the outer average inherit a phase bias? We sweep $H$ against the time-of-day coverage of each local window, comparing contiguous-time windows with windows that mix phases. We test whether factoring makes the result insensitive to that alignment.

Baselines

The naive reference is HA+LR from Rodrigues, fitted independently per region with zero communication. It is also exactly our model with $f_\theta$ replaced by a per-sensor linear map, so it doubles as the lower bound on what the shared network must add. Rodrigues reports that it beats DCRNN and STGCN on METR-LA, but the file quotes no figure: no reported result found on Lacuna for HA+LR MAE on METR-LA or PeMS. We will therefore compute it ourselves on identical splits. Beating it means lower MAE at 15 and 30 minutes, the horizons where spatial propagation should matter, on at least the PeMS flow datasets.

The strongest accuracy reference is a fully synchronized model trained centrally with shuffled data. It is HyperD-style, with every gradient step All-Reduced at full precision. This is the upper bound that communication savings must approach. For context, HyperD reports an average MAE reduction of about 23% over CycleNet across PEMS03/04/07/08, and TSTGNN reports MAE 15.51, RMSE 26.78 and MAPE 15.06% on PeMSD3. Our centralized re-run should land near these figures, or we report the gap.

The strongest communication-efficient competitor on traffic is the correlation-driven, error-feedback sparsified federated method (Gradient Compression and Correlation Driven FL). It reports a best MAE of 0.3621 on Trentino and needs 3–13 MB to reach loss 0.2, against more than 30 MB for FedAvg. We run it on Milan and Trentino with the authors' setup and on our PeMS district split. Beating it means matching its MAE with fewer bytes while using only All-Reduce. We also run unfactored DiLoCo and unfactored MT-DAO with the same compressors, since the file reports no traffic results for these.

The key ablation replaces the periodic tables with a generic per-region local head of matched parameter count, also never communicated. If the generic head closes the gap, the benefit comes from personalization in general and not from the weekly clock. All methods get the same data splits, the same network backbone, an equal hyperparameter search budget of trials per method, and the same simulated inter-region bandwidth.

Why not just…?

Why not just use HA+LR and skip distributed deep training? On some datasets it may well win, and the file says it beats DCRNN and STGCN on METR-LA, Seattle, Citi Bike and SZ-taxi. Its stated failures are residual non-stationarity under drift and short horizons where congestion travels across segments (Cheng et al.). Our model contains HA+LR as a special case, so the experiment directly measures whether the shared residual network earns its communication cost. Where it does not, we will say so.

Why not just pool the data and run ordinary data-parallel training with shuffling? That removes the heterogeneity, and it is our upper-bound baseline. The regime we target is regions that cannot ship raw sensor streams, or that train online on their own time-ordered data, which is the setting of REFOL and the cellular work. If pooling is allowed, local architectural speedups such as Memsizer, which cut METR-LA training from 877.43 s to 310.24 s (LLGformer), are the simpler answer.

Why not just keep a personalized local layer, or use correlation-weighted aggregation? A generic local head is the most serious threat, and the matched-parameter ablation tests it directly. Correlation-weighted aggregation works (Gradient Compression and Correlation Driven FL), but it needs per-pair weighting that does not map onto All-Reduce (Agarwal et al.). It is run as a baseline on its own benchmark.

Why not just sync more often, or use MT-DAO alone? MT-DAO fixes optimizer timescale mismatch on identically distributed data, which is a different problem from the clock-correlated data drift we target. We will report the accuracy-versus-bytes curve across $H$ for MT-DAO with and without factoring. If the curves coincide, our hypothesis is wrong.

Proposed Experiments

Workers are the four PeMS districts: PEMS03 (358 sensors), PEMS04 (307), PEMS07 and PEMS08 (170). These are four genuinely different regions with their own graphs. We add METR-LA and PEMS-BAY as a six-region speed variant, Milan and Trentino cells for comparison with the federated baseline, and Hangzhou metro for a check on a different transport mode, where DGST reports a 60-minute inflow MAE of 24.71 against AGCRN's 26.05 (DGST). Metrics are MAE, RMSE and MAPE at 15, 30 and 60 minutes; total bytes communicated to reach a target validation MAE; simulated wall-clock time at 1 and 10 Gbit/s inter-region links; and the drift $D(H)$ with its periodic share.

The first decisive check is whether factoring reduces $D(H)$ and removes its daily oscillation. The second is whether, at $H \in \{10, 30, 100, 300\}$ with 4-bit or DCT compression, the factored model stays within a small margin of the centralized MAE while unfactored DiLoCo and MT-DAO do not. The third is whether the periodic tables beat the matched generic local head; this is the experiment that ties the result to temporal structure. The fourth is whether the shared residual network beats per-region HA+LR at short horizons. As a stress test, we inject a mid-training regime shift by replaying a held-out period with sensor dropouts and surges, as HyperD does, and check whether slow local table updates keep compressed weight differences smooth or whether quantization error spikes. A negative result on the first or third check would itself be informative. It would show that heterogeneity in regional traffic training is not mainly about the clock, and it would redirect effort toward personalization methods that do not depend on periodic structure.