A fixed-function neural network accelerator runs a schedule that was compiled before the chip ever saw an input. For every layer it is known in advance which processing elements switch on, when the on-chip memory reloads, and when the pipeline fills and drains. That rigidity is usually counted as a weakness. This note argues it has an unrecognized benefit: the accelerator's electrical current demand is largely predictable ahead of time. If the compiled schedule tells the on-chip voltage regulator, tens of cycles in advance, that a current surge is coming, the regulator can brace for it instead of reacting after the supply voltage has already sagged. Bracing in advance shrinks the safety margin of extra voltage that chips normally carry, and that recovered margin is what lets dynamic voltage and frequency scaling (DVFS: slowing the clock and lowering the supply when a task would otherwise finish early) actually deliver its energy savings at very low voltages.
The opportunity is largest where margins hurt most. A near-threshold accelerator such as ChewBaccaNN, a flexible 22nm binary-network accelerator reaching 223 TOPS/W at 0.4V, is a good example: at that voltage, a 100 mV guardband raises dynamic energy by roughly 56%. The regulators we have today are either reactive or open-loop against a fixed load. The DVFS schemes we have today treat the supply rail as ideal. Nobody has used the accelerator's own schedule as the forecast that connects the two.
Core Claim
For accelerators whose dataflow is fixed at compile time, a feed-forward signal generated from the compiled schedule predicts the large current steps well enough to cut worst-case supply droop substantially below what a reactive compensator achieves. The lower droop allows a smaller voltage guardband. The smaller guardband makes slack reclamation save measurably more energy at the same deadline-miss rate and accuracy. The schedule-derived signal also sets the budget for a second choice the compiler controls. At each large transition it can spend slack on the load side, by ramping processing-element enablement over several cycles to soften the current step, or spend a little regulator energy on the supply side through pre-emptive compensation. The contribution is the forecast, the regulator interface that uses it, and a compiler pass that allocates slack across three uses: stretching execution, softening transitions, and bracing the supply.
Where This Sits in Prior Work
Two regulator papers supply the circuit mechanisms, and both leave the forecast missing. Park and Jeon's capacitor-free LDO (an LDO is a low-dropout linear regulator) adds a Tunable Undershoot Compensator, an auxiliary path that pulls the pass-transistor gate once it detects a fast output drop. It bounds undershoot to 150 mV and settles in 100 ns for a 200 mA step in 3 ns. Because it must first see the droop, some droop is unavoidable, and its self-calibrating clock cannot prepare for an impending step. Seshadri and Scott's pulse-shaping generator and its neurostimulator companion show the opposite style. A pre-programmed, irregular duty-cycle burst at switch-on removes SEPIC overshoot entirely and halves settling from 60 µs to 28–31 µs. That burst is tuned by hand for one constant 920 Ω load. The authors also report that pole placement limits how much can be compensated: the Ćuk converter's tightly packed poles prevented full elimination of overshoot. We reuse both mechanisms, a triggered auxiliary path and pre-shaped control, and supply what each lacks, which is a load-specific, time-stamped trigger. The load-current feed-forward of Li et al. is closer in spirit. It still senses current rather than forecasting it.
On the accelerator side, SCALE-Sim and Systolic-CNN show that systolic-array activity can be computed cycle by cycle from layer shapes and dataflow. SCALE-Sim also shows that mismatched dataflows cost up to 16× in latency. This is the motivating evidence that the current profile is knowable, though neither work connects it to power delivery. E4 is the closest cross-direction work. It turns early-exit slack into layer-level DVFS on Jetson boards, saving 26% energy on average and 37% at peak. It runs through operating-system governors on heavily guardbanded commodity rails and never models droop. Precise Scheduling of DAG Tasks with Dynamic Power Management and Simon et al. supply the dual-budget and run-time slack machinery we reuse at layer granularity.
Mittal's survey constrains the design in two ways. It notes that DVFS transitions have real cost, and that below 22nm leakage can make racing to sleep beat stretching execution. Our evaluation therefore has to include that crossover rather than assume stretching always wins. It also reports that three or four discrete operating points capture most of the savings, so we restrict the design to a few rails.
Research Program
The first component is a schedule-to-current compiler pass. From a network's layer list and the accelerator's fixed dataflow, it emits a per-cycle predicted current trace. It builds this from the active processing-element count, SRAM or standard-cell-memory access counts, and per-event energies taken from a 22nm near-threshold model calibrated against ChewBaccaNN's reported 1.1 mW at 154 MHz. It then marks "transition events", meaning cycles where predicted $di/dt$ exceeds a threshold. Each event gets a lead time, the number of cycles before the step at which a command must reach the regulator.
The second component is a regulator with a forecast port. We model a Park-style capacitor-free LDO whose undershoot compensator can also be fired by a digital command ahead of the step, with a strength code and a timing offset. On the rarer, larger voltage-change transitions that DVFS itself causes, the regulator applies a short pre-shaped control sequence in the manner of Seshadri. We fit that sequence per operating-point pair by numerical optimization rather than by hand. The Seshadri papers say closed-form inversion is intractable, so we do not attempt it.
The third component handles data-dependent events. Early exits, sparsity, and memory stalls break determinism. The design point is that most of these events are resolved some cycles before they change current. An early-exit decision is known before the next layer's processing elements start, so the command can be issued at the decision point. We will measure how much lead time survives for each event type. Where lead time is too short, the system falls back to the reactive path.
The fourth component is the allocation pass. For each transition $j$ with available slack $s_j$ cycles, the compiler chooses a softening ramp length $r_j$, a pre-emption strength $a_j$, and an operating point $V_j$. It minimizes
$$E = \sum_j \big[\, C_j (V_j + g(r_j, a_j))^2 + P_{\text{leak}}(V_j)\, t_j(V_j, r_j) + e_{\text{ff}}(a_j) \,\big]$$
subject to the deadline. Here $C_j$ is the switched capacitance of segment $j$, $g$ is the guardband needed to keep simulated droop above the timing limit, $t_j$ is the segment duration including any ramp, and $e_{\text{ff}}$ is the energy of the pre-emptive action. The squared term comes from dynamic energy scaling with $V^2$. The leakage term is what allows race-to-sleep to win when it should.
Baselines
The naive reference is a fixed-voltage accelerator with a static guardband sized for the worst simulated droop, running at one operating point and idling after each inference. Every energy number is reported relative to it.
The strongest supply-side competitor is Park and Jeon's reactive undershoot compensator, which reports 150 mV undershoot and 100 ns settling for a 200 mA / 3 ns step. We run it in the same power-delivery-network model, with the same pass device and current budget, scaled to our accelerator's current. It gets a guardband equal to its own simulated worst droop, and it is combined with the same slack-reclamation scheduler we use. The comparison therefore isolates only whether the forecast helps.
The strongest scheduling competitor is E4-style layer-level DVFS on idealized rails, which reports 26% average and 37% peak energy savings on Jetson hardware. Those numbers come from a different platform and are context, not a target. We reimplement its policy on our simulated accelerator, give it the same operating points and slack, and then re-evaluate its schedule under realistic droop with the guardband that droop requires.
The key ablations remove one ingredient each: the forecast replaced by the same command issued at a random offset, which tests whether timing or merely extra compensator strength matters; load-side ramping only; and pre-emption only. Beating the baselines means lower energy per inference than reactive compensation plus DVFS, with zero timing violations and no increase in deadline misses. The margin must exceed the spread across process-corner and power-delivery-network parameter sweeps.
Why not just…?
Why not just add more decoupling capacitance? More capacitance works. The regulator literature we build on exists because it costs area and board space that implants and mobile systems-on-chip cannot spare: Park's design is capacitor-free by construction, and Seshadri's feed-forward removed a whole filtering stage. We will report the on-chip capacitance that matches our droop reduction. If that area is a small fraction of the accelerator, the claim weakens, and we will say so.
Why not just rely on the reactive compensator? Its 150 mV undershoot is the number to beat. At 0.4V that is a large fraction of the supply, and reacting after detection cannot remove the part of the droop that occurs before detection. The random-offset ablation tests directly whether advance timing matters or whether a stronger reactive path would do as well.
Why not just soften the current steps on the load side by staggering processing-element start-up? This is the most serious alternative, because it is cheap and needs no regulator changes. Its cost is cycles, which is exactly the slack DVFS wants to spend. We treat it as a competitor and as one option inside the allocator. If load-side ramping alone matches the full system on our workloads, the honest conclusion is that the schedule-derived forecast is valuable mainly for deciding where to ramp. The regulator port would then be unnecessary, and the experiments are set up to reveal this.
Why not just race to sleep? Below the leakage crossover it may be the better choice. The allocation objective includes leakage, and we report the crossover operating point. Brief idle gaps between streamed inferences favor DVFS, but we will measure that rather than assume it.
Proposed Experiments
The workloads are MLPerf Tiny's CIFAR-10 image classification, Visual Wake Words on MS COCO, and keyword spotting (MLPerf Tiny); binarized ResNet-18 layers on ImageNet at ChewBaccaNN's operating regime; and an early-exit ResNet on CIFAR-10 and on Mini-Kinetics frames to supply data-dependent slack as in E4. Fathom's recurrent models (Fathom) serve as a stress case where the schedule is less regular. The simulation stack is SCALE-Sim traces fed to the current model, then to a SPICE-level power-delivery-network and LDO model. Accuracy is held at the MLPerf quality target, meaning within 98–99% of the reference model's accuracy.
Four checks decide the outcome. The first is predictability: whether the correlation between the schedule-predicted and simulated current is high enough, and the lead time long enough, for most of the droop-causing events. If not, the idea fails early and cheaply. The second is margin: whether worst-case droop, and with it the guardband, falls below the reactive baseline across process corners. The third is end to end: whether energy per inference at equal deadline-miss rate and zero timing faults beats reactive compensation plus DVFS, and whether it beats load-side ramping alone. The fourth is robustness: how the gain degrades as the early-exit rate and sparsity make the schedule less deterministic. That curve locates the boundary where accelerator rigidity stops being useful as a forecast. As a limited physical validation, we will measure rail current on an Arria 10 board running Systolic-CNN to test whether schedule-predicted current tracks measured current on real silicon. This checks prediction fidelity only, not voltage scaling.
References
- ChewBaccaNN: A Flexible 223 TOPS/W BNN Accelerator
- A 270-mA Self-Calibrating-Clocked Output-Capacitor-Free LDO With 0.15-1.15V Output Range and 0.183-fs FoM
- Pulse-Shaping Feed-Forward-Compensated Generator
- Energy-Efficient SMPS-Based Pulse Generator for Neurostimulators
- Control Method for Improving Voltage Response and Ripple of Isolated DC-DC Converter
- SCALE-Sim: Systolic CNN Accelerator
- Systolic-CNN: An OpenCL-defined Scalable Run-time-flexible FPGA Accelerator Architecture for Accelerating Convolutional Neural Network Inference in Cloud/Edge Computing
- E4: Energy-Efficient DNN Inference for Edge Video Analytics via Early Exiting and DVFS
- Precise Scheduling of DAG Tasks with Dynamic Power Management
- Energy Minimization in DAG Scheduling on MPSoCs at Run-Time: Theory and Practice
- A survey of techniques for improving energy efficiency in embedded computing systems
- MLPerf Tiny Benchmark
- Fathom: reference workloads for modern deep learning methods