Neural workloads on edge chips often run as task graphs. A perception stack might run two camera encoders side by side, feed both into a fusion network, and then pass the result to a planner. Mixed-precision quantization stores each layer's numbers at its own bit-width, for example 8 bits for sensitive layers and 4 bits for robust ones, to save time and memory. The standard way to choose those bit-widths is to profile or simulate each layer alone and add up the results. That works when layers run one after another. It breaks when independent branches of the graph run at the same time on a multi-core chip, because concurrent branches compete for the same memory bus and caches. Changing one branch's bit-width also changes how much memory bandwidth it consumes. That in turn changes how slow its neighbors run, and the slowest chain of dependent work (the critical path) can move to a different part of the graph.
This proposal treats that interaction as the missing piece. We would learn, from a small number of measured co-runs on the physical chip, how each node's latency depends on its own precision and on the precision of whatever runs beside it. We would combine that model with a standard critical-path computation over the graph, and search for bit-widths that maximize accuracy under a measured end-to-end deadline. Put simply, the bit-width of one branch should be chosen with knowledge of what it does to the branches it shares memory with. The proposal includes a test that could falsify this: if isolated per-node profiling composed over the graph already predicts real end-to-end time well, the contention term is unnecessary and we will say so.
Core Claim
On multi-core edge processors running task graphs with concurrent branches, isolated per-layer profiling composed through the graph will misrank mixed-precision configurations. A contention-aware latency model fitted from on-chip co-execution measurements will fix that ranking and produce better accuracy–latency trade-offs at equal measurement budget. We expect the effect to be largest exactly where precision changes a node's memory traffic, such as activation-heavy layers and 8-bit-versus-4-bit tile reshaping. We expect it to vanish on purely sequential networks.
The prior work that motivates this claim found the gap in pieces. Free Bits profiled layers on physical RISC-V microcontroller clusters and showed that lowering a layer's precision can increase its latency, because of software unpacking of sub-byte operands and because of scratchpad tiling mismatches. Its search, however, models network latency as a plain sum of isolated layer latencies. The authors note that this assumption fails under pipelining or multi-branch execution. HAQ and its companion Hardware-Centric AutoML for Mixed-Precision Quantization put hardware feedback inside a reinforcement-learning bit-width search. That feedback comes from cycle-level simulators (BitFusion, BISMO), and the search walks the layers as a linear chain. FlashDrive assigns different precisions to the stages of a multi-stage driving model: 4-bit weights with 8-bit activations for prefill, and 16-bit floating point for the action expert. Those assignments are hand-chosen for one pipeline.
On the scheduling side, the cloud workflow schedulers GATES and DEFT reason about graph structure and deadlines. They treat node costs as fixed constants. Cello co-designs a tensor-graph schedule with an on-chip buffer, but only at uniform precision and only in simulation.
None of these works lets a node's latency depend on the precision of the nodes running alongside it. That dependence is what we propose to model.
Research Program
The method has three parts.
A contention-aware node latency model. For a graph with nodes $i$ and bit-width assignment $b$, we model each node's measured latency as
$$L_i(b) = L_i^{\mathrm{iso}}(b_i)\Big(1 + \sum_{j \in O_i(b)} \phi_i\, \theta(d_i(b_i), d_j(b_j))\Big).$$
The terms are defined as follows. $L_i^{\mathrm{iso}}(b_i)$ is node $i$'s latency when profiled alone at bit-width $b_i$; this is the Free Bits-style table. $O_i(b)$ is the set of nodes whose execution overlaps node $i$ in time. $d_i(b_i)$ is node $i$'s memory bandwidth demand at that bit-width, measured with hardware performance counters. $\theta$ is a small learned function giving the slowdown caused by a pair of co-running demands. $\phi_i$ is the fraction of node $i$'s isolated runtime spent stalled on memory.
The pairwise form is an assumption. We will test it against measured three-way co-runs. Using performance counters as regressors follows the approach of Powmon/GemStone, where careful counter selection gave a 2.8% mean absolute percentage error for power on unseen workloads. We borrow the methodology, not the power target. Fitting requires on the order of a few hundred pairwise co-run measurements per platform. Their cost is reported alongside every result.
Graph-level makespan with migrating critical paths. Overlap sets depend on timing, and timing depends on overlap. We therefore compute the total runtime of the graph (the makespan) by simulating a list schedule on the target cores and iterating until the latencies converge. This exposes something sum-of-layers models cannot represent: a branch that was off the critical path can become critical once a neighbor's precision change starves it of bandwidth. We will log how often the predicted critical path differs between the isolated and contention-aware models. We will also check against measured runs which of the two prediction is correct.
Precision search under a measured deadline. Accuracy cost per node comes from a standard sensitivity estimate, the Hessian trace as in APTQ. We minimize the summed sensitivity-weighted quantization error subject to predicted makespan staying at or below a deadline $T$. The search uses an evolutionary method over discrete bit-widths of 2, 4, and 8 bits. The top few candidates are verified on the real chip, and those measurements are added back to the fitting set. This verify-and-refit loop applies the hardware-in-the-loop idea of Air Learning to precision search: measured hardware behavior is injected where a simulator would otherwise assume it away. Final configurations receive quantization-aware fine-tuning, matching the Free Bits recipe.
Platforms and workloads. We target three platforms that share memory across cores in different ways. The first is a multi-core PULP-family RISC-V cluster deployed through DORY, the platform family where Free Bits found non-monotonic latency. Where available, we also use the Dustin 16-core cluster, which supports 2- to 32-bit precision. The second is an Arm big.LITTLE board (ODROID-XU3, as used by Powmon). The third is a Jetson-class embedded GPU, the class FlashDrive used.
Workloads are task graphs with real concurrency. The first type is multi-model perception graphs built from ImageNet-trained MobileNetV1/V2 and ResNet backbones: two to four parallel encoders, a fusion head, and a downstream classifier or planner. Multi-DNN co-execution scenarios are adapted from Sparse-Multi-DNN-Scheduling. The second type, on the Jetson only, is a scaled-down version of FlashDrive's multi-camera encode-then-reason pipeline. Accuracy is measured on ImageNet for the backbones and with open-loop displacement for the driving pipeline.
We deliberately exclude the scientific workflow benchmarks (CyberShake, Montage) used by GATES and DEFT. Their nodes are not neural layers whose precision can be changed.
Baselines
The naive reference points are uniform 8-bit and uniform 4-bit quantization of every node, scheduled with the same list scheduler and measured on the same chip. They define what "beating doing nothing" means. Free Bits reports that its method cut end-to-end latency by 28.6% relative to uniform 8-bit for MobileNetV1 (0.75 width) on XpulpNNv2 at 68.6% top-1. For MobileNetV2 it reduced the latency of a 4-bit configuration by 46% while raising accuracy by 0.6 points (Free Bits). Those numbers are for sequential networks.
The strongest existing method is Free Bits extended to graphs in the most natural way. We use its measured isolated latency table and its upgrade-when-free heuristic, and replace the sum over layers with a critical-path computation over the graph using the same isolated latencies. We call this "isolated-graph Free Bits." It is the most dangerous competitor because it already handles non-monotonic per-layer latency on silicon and already exploits slack off the critical path. The only thing it lacks is the interaction between co-running nodes. It receives the same bit-width options, the same sensitivity estimate, the same fine-tuning recipe, and an equal or larger budget of hardware measurements. We let it spend our co-run budget on additional isolated repetitions to reduce noise.
We also run HAQ-style reinforcement-learning search with a simulator-derived latency signal. This quantifies how much a purely simulated latency model loses on concurrent graphs. HAQ reports 1.4× to 1.95× latency reduction over fixed 8-bit in simulation (Hardware-Centric AutoML for Mixed-Precision Quantization). We found no reported result on Lacuna for any mixed-precision method on concurrently executed multi-branch graphs measured on silicon, so all graph-level baseline numbers will be produced here.
The key ablation removes only the contention term, setting $\theta = 0$, while leaving the search, the measurement loop, and fine-tuning unchanged. A second ablation keeps the contention term but forbids critical-path recomputation, which tests whether critical-path migration matters separately from the slowdown itself. We beat the strongest baseline if, at matched measured makespan, accuracy is higher by more than seed-to-seed variance on at least two of three platforms. An equivalent win is lower measured makespan at matched accuracy. We also require held-out makespan prediction error clearly below that of the isolated-graph model.
Why not just…?
Why not just profile each layer in isolation and compose through the graph? This is the real threat, and it is the isolated-graph Free Bits baseline above. If memory contention on these chips is weak, or does not depend on precision, it will match us at lower measurement cost. The prediction-accuracy experiment settles this directly. If the contention-aware model does not reduce held-out makespan error by a meaningful margin, and the critical path rarely migrates, we will report that composition suffices and that the contribution reduces to Free Bits on graphs. Our prior that the effect exists comes from Free Bits' own finding that bit-width changes memory tiling and transfer behavior, which is exactly the resource concurrent branches share. That is a hypothesis, not evidence.
Why not just search with the real chip in the loop and skip the model? Black-box search over measured end-to-end runtime is simple and avoids modeling assumptions. Its cost grows with the number of candidates, and each measurement requires recompiling and redeploying. We will compare against hardware-in-the-loop evolutionary search given exactly the same total number of on-chip runs, and plot Pareto quality against measurements. If black-box search wins at realistic budgets, the model is not worth having. We expect it to lose at small budgets and converge with ours at large ones.
Why not just use a better cycle-level simulator? Simulators can in principle model bus contention. The record in the file, however, is that uncalibrated simulators misestimate exactly these effects: Powmon/GemStone document severe specification errors in gem5 power estimates, and Free Bits shows sub-byte overheads that simulators assuming monotonic latency would miss. Our simulator-driven HAQ baseline measures how large that misranking is on concurrent graphs.
Why not just let the scheduler reorder tasks and keep precision fixed? Reordering can separate memory-heavy branches in time. Doing so lengthens the makespan whenever those branches are both needed early. We include a contention-aware schedule-only variant at uniform 8 bits to measure how much of the gain comes from scheduling alone.
Proposed Experiments
The first experiment is predictive. For every platform–workload pair, we sample a few hundred random bit-width assignments, measure their real makespans, and compare three predictors on held-out configurations: the sum of isolated latencies, isolated latencies composed through the graph, and the contention-aware model. We report mean absolute percentage error and rank correlation. We also report the fraction of configurations whose measured critical path differs from the one the isolated model predicts. We additionally include sequential MobileNets as a negative control, where all three predictors should agree. If they disagree on those networks, our measurement setup is suspect.
The second experiment is the search comparison. We plot accuracy–makespan Pareto fronts against all baselines at matched hardware-measurement budgets. Accuracy is ImageNet top-1 for the perception graphs and minimum average displacement error for the driving pipeline. We use three seeds for fine-tuning.
The third experiment tests where the gain comes from. We stratify gains by how memory-bound the concurrent branches are, varying input resolution and branch count. The claim predicts gains that grow with the amount of memory-bound concurrency and disappear for compute-bound branches.
Thermal and power effects, flagged in the file through Powmon's temperature-dependent leakage, are logged throughout but are not optimized. If sustained runs on the Jetson-class board show throttling that shifts makespans, that becomes a follow-up rather than a hidden confound. The proposal's central result is the predictive comparison: either contention changes which configurations are fast, or it does not.
References
- Free Bits: Platform-Aware Latency Optimization of Mixed-Precision Neural Networks for Edge Deployment
- HAQ: Hardware-Aware Automated Quantization With Mixed Precision
- Hardware-Centric AutoML for Mixed-Precision Quantization
- FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving
- GATES: Cost-aware Dynamic Workflow Scheduling via Graph Attention Networks and Evolution Strategy
- Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts
- Cello: Co-Designing Schedule and Hybrid Implicit/Explicit Buffer for Complex Tensor Reuse
- Accurate and stable empirical CPU power modelling for multi- and many-core systems
- APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
- Air Learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation
- Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster With 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Mode
- Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge