Paralog-Split Placement: Thermal Load Shedding in Neural Inference by Design

When a phone or embedded board runs a neural network and gets hot, the hardware slows every processor down uniformly. The better thermal schedulers instead split each layer's work between the CPU and GPU, sized so that both stay under the temperature limit. Both approaches assume that every channel of the network must be computed. Biology suggests another option. Organisms keep duplicated genes, called paralogs, that can cover for each other. Losing one copy costs almost nothing, and losing both is catastrophic. Trained networks show similar structure: some groups of channels matter only when their partner group is also removed. This proposal combines the two observations. Before deployment, we find which channel groups are redundant partners. We then split each layer across processors so that every partner pair straddles the CPU/GPU boundary. When the hotter processor has to shed work, it drops only groups whose partners are still running on the cooler one, and a cheap correction rescales those surviving partners.

In short, we make placement decide what gets lost. If the split between processors is chosen with redundancy in mind, a thermal emergency on one processor removes computation the network can already do without. That should give a much better accuracy-versus-temperature trade-off than lowering clock speed, and than dropping the "least important" channels under a split chosen only for speed.

Core Claim

Our central claim concerns placement, not pruning. Take any fixed rule for which channels to drop when hot. We claim that assigning channel groups to processors by their pairwise redundancy will beat assigning them by processor speed. It should keep more accuracy at the same sustained throughput and the same peak temperature, and it should show fewer temperature swings. We also claim that pairwise redundancy has to be measured by knocking out pairs of groups together. Single-group knockouts cannot see it, and activation magnitude misleads.

The knockout measurement borrows from genetics. For channel groups $g$ and $h$, define a double-knockout interaction score

$$e_{gh} = P(\{g,h\}) - P(\{g\}) - P(\{h\}) + P(\emptyset),$$

Here $P(L)$ is the network's accuracy on a held-out calibration set when the groups in $L$ are zeroed. A strongly negative $e_{gh}$ means the two groups are each dispensable alone but jointly essential. That is the paralog signature seen for soybean LEAFY1/LEAFY2, where only the double knockout arrests floral development (LEAFY1 and 2 are required for floral organ development in soybean). The same double-knockout logic is used to find redundant parallel pathways in yeast interaction graphs (Exploring weighting schemes for the discovery of informative generalized between pathway models to uncover pathways in genetic interaction networks).

Where This Sits

The mechanism we build on is the channel-wise CPU/GPU split of Thermal-Aware on-Device Inference Using Single-Layer Parallelization with Heterogeneous Processors. That paper has two parts. The first (TADF) picks the highest safe frequencies from a model that combines temperature-dependent leakage power with dynamic power. The second (HSWD) splits output channels in proportion to processor speed. The split is dense and fixed per layer, and the choice of which channels go where carries no functional meaning. We keep its thermal model and change both that choice and the response to transient heat.

Two other systems degrade workloads gracefully, but at much coarser grain. TAPAS handles datacenter heat events by changing batch size, lowering GPU clocks, or routing queries to smaller quantized checkpoints. The model itself is treated as a black box. Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls prunes offline, then schedules operators across processors according to which operators each processor supports. It also reports that sharing work across processors matters more as the chip heats from 30°C to 70°C, which motivates this work. Neither system decides at runtime which internal computation to give up.

On the redundancy side, Who Does What? A Novel Algorithm to Determine Function Localization supplies the diagnostic idea. Its performance prediction algorithm fits per-unit contributions and a monotone curve mapping surviving contributions to performance, using many multi-unit knockouts. A curved (concave or sigmoidal) mapping is the formal sign of redundancy. It also shows two traps. Single knockouts of redundant units show zero loss, and highly active units can be functionally useless bystanders. That work used a 10-neuron network with no hardware. We scale the idea by knocking out channel groups rather than single units, and we tie it to physical placement.

The "active compensation" step also has a biological analog. Knocking out KDM5B upregulates its paralogs KDM5A and KDM5C, which masks the metabolic phenotype (The Lysine Demethylase KDM5B Regulates Islet Function and Glucose Homeostasis). Our correction step plays the same role: it boosts surviving partners when a group is dropped.

Research Program

Step 1: find redundancy. For each layer of AlexNet, VGG-16, ResNet-18 and ResNet-50, we cluster output channels into roughly 16–32 groups by the similarity of their activations on calibration images. Grouping keeps pairwise knockouts tractable, which addresses the combinatorial explosion that limits the function-localization method. We then compute single and double group knockouts to obtain $e_{gh}$. We also fit the contribution-plus-curve predictor on sampled multi-group knockouts, and we use its fit on held-out knockout patterns as a sanity check.

This step is a go/no-go gate. If the fitted curve is close to linear and few pairs show strongly negative $e_{gh}$, the networks lack exploitable paralogs. In that case the proposal collapses to ordinary pruning, and we will report that result.

Step 2: place. For each layer, we solve a small assignment problem. It keeps HSWD's speed-proportional share of work on each processor. It rewards putting partner pairs on opposite processors, and it places groups with no partner, the essential ones, on the processor that is more thermally stable. The expected shed state of the hot processor is scored by the predictor from Step 1.

Step 3: shed and compensate at runtime. When the temperature forecast from the TADF-style model predicts a crossing of $T_{max}$, the hot processor sheds its partnered groups in order of predicted loss, instead of lowering frequency. For each shed state we precompute a per-channel scale and bias for the surviving partners, fitted in closed form on calibration data to restore the next layer's input statistics. Applying it costs one multiply-add per channel. When the chip cools, the shed groups are restored. This gives a reversible, temporary knockout rather than permanent pruning.

Baselines

The naive reference is dense execution on the Jetson Nano under stock hardware thermal management. The GPU and CPU throttle reactively at their limits and the model is unchanged. This anchors accuracy at its maximum and shows what reactive throttling costs in latency and temperature swings.

The strongest existing method is TADF plus HSWD from Thermal-Aware on-Device Inference Using Single-Layer Parallelization with Heterogeneous Processors. On the Jetson Nano at 40°C ambient, that paper reports AlexNet latency cut by 21%–36% and VGG-16 latency cut by 32%–43%, with temperature held below 85°C. We found no reported accuracy or tail-latency result on Lacuna for it under transient ambient steps. Being dense, its accuracy should equal the unmodified model. We will reimplement it on the same board, models, ambient schedule and 85°C cap. To beat it, we must match its sustained throughput with at most 1 point top-1 loss while cutting 99th-percentile latency during ambient surges. Alternatively, at equal latency, we must reduce thermal-cycle damage, counted with the rainflow procedure used by A Two-Level Thermal Cycling-aware Task Mapping Technique for Reliability Management in Manycore Systems.

The main competitor for shedding is HSWD's speed-ratio split combined with the same runtime shedding and compensation, where groups are ranked by a standard importance score (weight magnitude or single-group knockout loss). This isolates placement as our key ingredient, and a win must show on accuracy-at-equal-throughput curves across ambient temperatures.

A coarse competitor in the style of TAPAS switches to a smaller or quantized checkpoint when hot. TAPAS's reported numbers (17% lower maximum server temperature, 23% lower peak row power, on Azure A100/H100 traces) come from a datacenter setting and are not directly comparable. We will implement the policy on the board with ResNet-18 as the fallback for ResNet-50.

Three ablations complete the set. The first removes compensation. The second replaces the double-knockout redundancy scores with activation-similarity clustering alone. The third replaces them with single knockouts, which the function-localization work predicts will misclassify redundant partners as unimportant.

Why not just…?

Why not just lower the clock speed, as TADF already does optimally? Lowering frequency gives up throughput to buy temperature headroom, and it keeps full accuracy. Our method gives up a little accuracy to buy headroom. Which trade is better depends on the application, so we report full Pareto curves rather than a single number. The expected advantage comes during transient surges. Frequency reduction must be chosen for the worst steady state, while shedding is instant and reversible. If TADF alone matches our curves, then partner-aware shedding adds nothing, and the experiment will show that directly.

Why not just prune offline or train a width-adjustable network and drop the least important channels? This is the most serious threat. A network trained to run at several widths might tolerate dropping channels well without any redundancy analysis. We treat it as a baseline, not a strawman. We will train such a variant and apply both speed-ratio placement and partner-split placement to it. If partner-split placement gives no gain on top of it, our claim reduces to "width-adjustable networks work for thermal shedding," and we will say so. Offline pruning alone cannot respond to a 30°C-to-70°C swing like those in Deep Learning Inference on Heterogeneous Mobile Processors. A model sized for the hot case wastes accuracy in the cool case.

Why not just rank channels by activation or single-knockout loss, and skip pairwise knockouts? The 10-neuron results in Who Does What? show both rankings failing on redundant units and bystanders. Whether that holds at channel-group scale in convolutional networks is an open empirical question, and our ablations settle it. If single-knockout ranking matches pairwise placement, the biology-derived measurement is unnecessary, and the contribution narrows to the placement principle.

Proposed Experiments

All runs use the Jetson Nano setup from the heterogeneous-inference paper: four Cortex-A57 CPU cores and a 128-core Maxwell GPU sharing DRAM. The board sits in an ambient chamber held at 25°C, 40°C, and a stepped 25→45→25°C schedule, with an 85°C cap. Models are AlexNet, VGG-16, ResNet-18 and ResNet-50, evaluated on the ImageNet validation set, with a disjoint calibration split used for knockouts and compensation fits. Metrics are top-1 accuracy, median and 99th-percentile latency, sustained frames per second, time above threshold, and rainflow-counted thermal-cycle amplitude.

Check 1: does redundancy exist? We first run the go/no-go test. It asks what fraction of group pairs have strongly negative $e_{gh}$ in each layer, and whether the contribution-plus-curve predictor fits held-out multi-group knockouts well enough to guide placement.

Check 2: does placement matter? At matched sustained throughput, and with the dropping rule held fixed, we compare accuracy under partner-split placement versus speed-ratio placement. The gap should grow with ambient temperature, since more groups are shed. It should be larger for VGG-16 than for ResNet-50 if wide, plain layers carry more paralog-like redundancy.

Check 3: what explains the gain? The compensation and scoring ablations show how much of any gain comes from the correction step and how much from pairwise measurement.

Check 4: does it hold on another platform? We repeat the core comparison on a Snapdragon 855 phone to test portability across GPU architectures. CoDL's reported 48.4% latency reduction on that chip in Deep Learning Inference on Heterogeneous Mobile Processors serves as the dense reference there. The same paper reports large fallback penalties when operators are unsupported, which will limit which channels can be split.