Learner-Driven Block Sizing for Pipelined Edge Training

A small device often has to ship its data over a wireless link to a nearby server that must train a model before a hard deadline. Waiting for all the data to arrive before training wastes time. A better approach is pipelining: cut the data into blocks, and let the server train on the blocks already received while later ones are still in transit. Each block pays a fixed header cost, though. Under a deadline, the system must decide how big each block should be and when to stop sending data altogether. Every second spent transmitting is a second taken from optimization. The best existing answer fixes one block size in advance from a worst-case bound on the training loss. It assumes the channel rate never changes and that no packets are lost. We propose that the learner should choose the size of the next block, and whether to request one at all, while training runs. It would do so from two quantities it can measure for free: how much a freshly arrived block surprises the current model, and how fast the loss is falling on the data already received.

The idea rests on one observation. In a pipeline, every newly arriving block is unseen data, and it can be scored before the model trains on it. That score gives a running estimate of how much the model is still missing for lack of data. The training curve gives a running estimate of how much it is missing for lack of computation. A controller that compares the two can make the trade-off the static analysis makes on paper, but with measured numbers in place of worst-case constants. It can also absorb channel slowdowns and retransmissions as they happen.

Core Claim

Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning has two properties that matter here. It establishes the bias–variance trade-off for this setting. It also proves that withholding part of the dataset is optimal when per-packet overhead is high. Its block size $\tilde n_c$ is constant over the horizon and is derived offline from known rate, overhead and smoothness constants. The paper explicitly ignores channel errors. Our claim is that a closed-loop rule can do better under realistic conditions: it picks block $k+1$'s size $n_{k+1}$ and a stop-transmission time from online estimates of estimation error and optimization speed. The realistic conditions are time-varying rate, packet loss, and problems whose conditioning departs from the worst-case bound. In a stable channel, the rule should lose nothing relative to the best static size.

The controller works as follows. After $k$ blocks totalling $N_k$ samples, it records $g_k$, the loss of the current model on block $k$ measured on arrival before any training on it, minus the training loss on the $N_{k-1}$ earlier samples. This is a progressive-validation estimate of the generalization gap. It also records $\rho_k$, the recent rate of training-loss decrease per second of compute. For a strongly convex empirical risk, excess risk decays roughly as $1/N$, so sending $n$ more samples is predicted to shrink the gap by about $g_k\, n/(N_k+n)$. The transmission costs $(n+n_o)/R_k$ seconds, where $n_o$ is the header overhead and $R_k$ is the measured effective rate including retransmissions. That time is taken from optimization only to the extent that compute would otherwise sit idle or run on stale data. The controller picks

$$ n_{k+1} = \arg\max_{n \ge 0}\; \Big[\, g_k \frac{n}{N_k+n} \;-\; \hat c_k(n) \,\Big], $$

where $\hat c_k(n)$ is the predicted optimization loss forgone within the remaining deadline. It is computed by extrapolating $\rho_k$ over the time the block occupies the link. Choosing $n=0$ stops transmission, and all remaining time goes to optimization. Both signals are smoothed with an exponential moving average. The smoothing weight is the rule's only meta-parameter, and we report sensitivity to it directly. This addresses the concern from the adaptive-parameter literature that adaptive rules merely move the tuning burden onto higher-order knobs.

Relation to Prior Work

The pipelined-transmission literature supplies the problem and the latency logic. Fog-RAN analyses such as Fog-Aided Wireless Networks for Content Delivery: Fundamental Latency Tradeoffs show that overlapping fetch and transmit can cut delivery latency by up to a factor of two. They work at infinite SNR with fixed file splits, and they deliver content rather than training a model. Hoplite pipelines chunked objects through task systems, but it treats chunk size as a systems constant unrelated to the learning objective. To Stream or Not to Stream motivates the need for runtime adaptation. It documents transfers that should take 0.16 s spiking past 5 s once link utilization exceeds 90%. Its own model is an offline stream-or-stage decision, not a controller.

The adaptive-parameter literature supplies the control style. Adaptive balancing of gradient and update computation times (ACM) adjusts inner-iteration counts with additive, multiplicative-additive and gradient rules to maximize progress per unit time. Our objective is the same kind of quantity, but the cost being balanced is link time rather than in-memory gradient time, and ACM models no communication. AdaBatchGrad grows the batch size when its inner-product and orthogonality tests show the stochastic gradient is too noisy. This points to the tension the file calls Gap 4.4. Statistical rules want more data as training converges, while a deadline wants less transmission. Our rule settles the question per block using measured magnitudes rather than a fixed preference in either direction. Pipeline-parallel schedulers in datacenters, AdaPtis and ZeroPP, adapt partitions and micro-batch schedules from offline profiles. They play no methodological role here, but they confirm that static schedules leave large idle fractions when stage costs vary.

Research Program

We will build a simulator of the single-device, single-server pipeline from the static analysis. To it we add three channel processes: a constant rate; block-fading rates with per-packet erasures and retransmission; and bursty slowdowns modeled on the congestion behavior Castro et al. report. The server runs gradient descent, or SGD for the non-convex case, on the received data. Alongside the simulator we will analyze the rule. In the strongly convex case with constant rate, we will bound the gap between the controller's final loss and that of the best static block size. The bound will be stated in terms of the estimation noise in $g_k$ and $\rho_k$. This tells us when adaptation cannot help, which matters as much as showing when it does. A small physical check on a Wi-Fi link between an embedded board and a laptop will confirm that the simulator's rate and erasure processes are not optimistic.

Baselines

The naive reference point is serial transmission: send all $N$ samples, then train until the deadline. Pipelining must clearly beat it, or the setting is uninteresting.

The second baseline is the analytical static block size $\tilde n_c$ of Skatchkovsky and Simeone. It is computed from the true mean channel rate and true overhead, which gives it more information than it would have in practice.

The strongest static competitor is the oracle static block size, chosen by grid search over $n_c$ separately for each dataset, deadline and channel regime. It is tuned with the same per-run budget our meta-parameter sweep uses. The only reported number on this axis comes from Skatchkovsky and Simeone on California Housing: direct numerical search improves on the analytical formula by only 3.8%. That tells us the static bound is already tight in a constant channel, so we expect little gain there. No reported result found on Lacuna for any dynamic block-sizing method in this setting.

The strongest adaptive competitor is channel-only replanning. After each block, it re-solves the static formula for the remaining deadline and remaining data, plugging in the currently measured rate. This is what a careful systems engineer would build first.

We also run ACM's multiplicative-additive rule applied to block size, driven by training progress per second, and AdaBatchGrad running at the server over a static-$\tilde n_c$ link.

The ablation that isolates our key ingredient is our controller with $g_k$ replaced by the bound's worst-case constant. This is in effect channel-only replanning plus the stop rule. It shows whether the learner-side measurement, rather than adaptivity per se, produces the gains.

All methods see the same data order, channel traces (identical random seeds), deadline and compute rate. Beating a baseline means a lower test loss at the deadline, averaged over at least twenty channel traces, with non-overlapping 95% intervals. It also means losing to the oracle static size by no more than 1% in the constant-channel regime.

Why not just…?

Why not just use the static optimal block size? In a constant channel this is likely nearly optimal, and the 3.8% figure says the bound is tight there. We concede that regime in advance and use it as a no-regression test. The static size cannot react to a rate drop or to erasures, which the original analysis omits. Its constants are also worst-case, so on problems that are better conditioned than the bound assumes, it keeps sending data the model no longer needs. The fading and burst regimes, together with datasets of varied conditioning, measure exactly this.

Why not just re-solve the static formula with the current measured rate? This is the real threat. If the static bound's constants are accurate, channel-only replanning captures all the gain, and our learner-side signal is redundant. The ablation decides it. Our claim fails if our controller does not beat channel-only replanning on datasets where the empirical generalization gap departs from the bound's prediction. If it fails, the honest conclusion is that replanning suffices and the contribution reduces to adding loss handling to the static analysis.

Why not just apply a TCP-like rule or AdaBatchGrad? ACM's multiplicative-additive rule responds to progress rate alone. It has no notion of estimation error, so it cannot tell "more data would help" from "more compute would help." AdaBatchGrad's tests only ever push toward larger batches, which runs against what a deadline requires. Both are run as baselines. We expect them to over-transmit late in training, and we will check this by logging the stop times they imply against the oracle's.

Proposed Experiments

We use California Housing with the static paper's split, $N=18{,}576$ training samples, for linear regression. We add the LIBSVM sets a9a, w8a, gisette and rcv1 for logistic regression and Lasso; these differ widely in dimension and conditioning. FashionMNIST with a small convolutional network serves as a non-convex stress test, where the $1/N$ scaling is only a heuristic. For each, we sweep the deadline, the header overhead and the three channel regimes, and report test loss at the deadline, the number of samples actually sent, and the stop time.

Three checks decide the proposal. First, in the constant channel, the controller must land within 1% of the oracle static size. Second, under fading with erasures, it must beat channel-only replanning, with the margin growing as the bound's constants become looser. To test this directly, we construct redundant variants of each dataset by duplicating samples. The controller should then stop transmitting far earlier than any bound-based rule. Third, final loss should vary by under 2% across a tenfold range of the smoothing weight. If it does not, the meta-parameter problem has simply moved, and we will say so.