Programmable network switches can run small machine learning models on every packet at terabit speeds, but only if the model fits into a few megabytes of fast on-chip memory and a fixed chain of roughly a dozen processing stages. Today's tools, chiefly Planter, which compiles decision trees, naive Bayes, SVMs and 1-bit neural networks into switch tables, treat that budget as a yes/no fit test. They also inherit habits from GPU-era model compression, where the goal is to cut parameters or arithmetic. On a switch, arithmetic is nearly free and largely irrelevant. What runs out is table entries, per-stage memory accesses and stage depth. We propose to make the switch's actual resources the compression objective. The plan is to build a resource model that predicts stages, memory and recirculations from a model's structure, check it against the vendor compiler, and use it to search and compress classifiers. The test is whether models compressed this way beat models chosen by parameter or operation count at a fixed switch budget.
The concrete vehicle is a "table-native" network. Each neuron reads only a few low-bit inputs and is stored as one lookup table, so any nonlinearity, bias or correction costs nothing extra at run time. The cost of such a neuron grows exponentially in fan-in times bit-width and does not depend on how many multiplications it would represent on a GPU. Each layer consumes one stage. Under this cost structure, a sparse, low-fan-in, moderately multibit network and a dense binarized network of equal "size" have very different footprints. The compression problem therefore becomes a choice of fan-in, bit-width, depth and stored-feature precision. We will test whether making that choice explicitly yields more accurate in-switch classifiers on encrypted-traffic and intrusion tasks than trees or binarized networks get under the same budget.
Core Claim
Operation count is the wrong proxy for in-switch models, just as it is the wrong proxy on GPUs, but the failure takes a different form. Three findings show the GPU version. ShuffleNet V2 showed that memory access cost, not FLOPs, sets GPU latency. All You Need Is a Few Shifts found that zero-FLOP shift operations ran slower than convolutions. MOSAIC showed that FLOP-optimal sparse mixtures of experts lose to denser configurations once real hardware utilization is counted: the denser model reached 0.031 nats lower loss under the same 5-day B200 budget. We claim the switch analogue of that ranking flip exists and is larger. Among classifiers of equal parameter count, the one that fits a Tofino pipeline alongside normal forwarding with the highest accuracy is determined by four quantities. These are the largest table, measured as fan-in times input bits; the depth of the table dependency chain; the per-flow register bits needed to hold input features; and the number of stateful accesses per stage. A compression procedure that optimizes these directly should dominate both FLOP- or parameter-driven compression and the fixed model families Planter offers.
The resource model can be written compactly. For a table-native network with neuron $j$ in layer $\ell$ reading $k_j$ inputs at $b_\ell$ bits and emitting $b_{\ell+1}$ bits, the SRAM cost is
$$ M = \sum_j 2^{k_j b_\ell}\, b_{\ell+1} \;+\; F \cdot \sum_i q_i , $$
where $F$ is the number of concurrently tracked flows and $q_i$ is the stored bit-width of per-flow input feature $i$. Stage cost is the network depth plus the stages needed for feature extraction, subject to per-stage access limits. The exponential term is why we compress fan-in rather than weights. The flow term is why feature precision has to be quantized in the same way QSketch quantizes sketch registers to 8 bits.
Position Relative to Prior Work
Three in-switch systems motivate the gap, and each stops short of it. Planter supplies the compilation primitives we reuse: encode-based feature partitioning, lookup-based tables, and XNOR/popcount for binarized networks. It reports line rate at 6.4 Tbps, under 4.7% latency overhead and accuracy within 1% of scikit-learn. Planter offers no route to multibit networks and does not model how table count and depth trade against co-resident forwarding tables. CyberSentinel distills autoencoders into an isolation forest held as whitelist rules in 9 Tofino stages using under 10% of SRAM/TCAM. Its compression target is fixed by the student family, and it does not explore the trade-off between depth and memory. Kaleidoscope shows that raw-byte neural models reach 92.8% to 99.2% accuracy on ISCX-VPN-2016 and USTC-TFC. It gets there by moving inference to an FPGA/ASIC side processor, so it never compresses anything into the switch itself. Our proposal sits between Planter and Kaleidoscope. We aim to bring multibit neural accuracy inside the switch by compressing for the switch's own cost structure.
MOSAIC supplies the method template we borrow: an operator-level analytical performance model, ScalePlan, with under 15% error, used to re-rank model configurations. Our resource model plays the same role on a pipeline ASIC instead of a GPU cluster. RestoreLCC supplies a second idea. It restores pruned language models by folding learned constant corrections into existing biases, adding zero latency. In a table-native network the analogue is stronger, because a neuron's table can absorb any per-entry correction, not only a constant one. We therefore fine-tune table contents after structural compression and treat this as a measured ablation, not a headline. Two papers act as constraints. The stateful-access limits documented for INDDoS, together with the observation in Lu Tang et al. that switches lack division, loops and floating point, rule out designs that need per-packet arithmetic beyond table lookup and addition. Semi-structured sparsity work such as ARMOR targets 2:4 patterns built for GPU tensor units. We expect that pattern to be the wrong one for switches, where the binding quantity is per-neuron fan-in, and we test this directly.
Research Program
The first component is the resource model. We will enumerate a grid of table-native networks, binarized networks and trees, compile each with the Tofino compiler alongside a reference forwarding program, and record stages, SRAM, TCAM, stateful accesses and whether recirculation was needed. We then fit and validate the analytical model above against these reports. The model is useful only if it ranks configurations correctly, so the success criterion is rank correlation and fit/no-fit accuracy on held-out configurations, not absolute error. Where hardware is available we also measure per-packet latency and throughput. Without hardware, the compiler's stage and recirculation report is the latency proxy, and we will say so.
The second component is the compression procedure. Starting from a full-precision teacher trained on raw early-packet features, the procedure works in three steps. It learns a sparse connectivity in which each neuron reads at most $k$ inputs, using a straight-through, top-$k$ masked training scheme. It then assigns per-layer bit-widths and per-feature storage precision. Finally, it searches $(k, b_\ell, \text{depth}, q_i)$ to maximize validation accuracy subject to the resource model staying within a budget, defined as a fraction of the stages and SRAM left over after forwarding. After choosing a structure, we enumerate each neuron's table from the trained network and optionally fine-tune the table entries directly on training data, which is the RestoreLCC-style correction step.
The third component is the ranking-flip analysis. For every task we take the set of compilable models and compare the order given by parameter count, by operation count and by our resource model with the order given by accuracy at a fixed budget. This test checks whether the FLOPs-latency discrepancy actually carries over to switches and is not just asserted.
Baselines
The naive reference is a FLOP- or parameter-matched model chosen without switch awareness. We take the most accurate binarized network and the most accurate small MLP, at a fixed parameter budget, that compile at all. Each gets the same training data and the same hyperparameter budget as our method. Beating it shows that switch-aware compression matters beyond simply fitting a model.
The strongest existing in-switch competitor is Planter. We run its decision-tree, random-forest and binarized-network back ends on identical features and identical stage and SRAM budgets with forwarding co-resident, and tune their depth and width over the same number of configurations we search. Planter reports accuracy within 1% of server-side scikit-learn and 6.4 Tbps line rate. We found no reported result on Lacuna for Planter on ISCX-VPN-2016, USTC-TFC or UNSW-NB15, so we will produce these numbers ourselves. Beating Planter means higher macro-F1 at equal or lower stage and SRAM use on at least the multi-class encrypted-traffic tasks, with no recirculation.
For anomaly detection on UNSW-NB15 and the IoT botnet traces, we include CyberSentinel at its reported operating point: 9 stages, under 10% SRAM/TCAM and 532 ns average per-packet latency. We compare detection quality at that same footprint.
The off-switch accuracy ceiling is Kaleidoscope's reported 92.8% to 99.2% on ISCX-VPN-2016 and USTC-TFC. We do not expect to match it. We report the gap, since closing a meaningful fraction of it without extra silicon is the point of the work.
Three ablations isolate the key ingredients. The first replaces our resource model with operation count inside the same search. The second fixes fan-in and varies only bit-width, separating the fan-in effect from quantization. The third removes table-entry fine-tuning, separating structural choice from post-hoc correction.
Why not just…?
Why not just use Planter's random forests? This is the most serious threat. Trees fit switches naturally and are strong on tabular flow statistics, so on UNSW-NB15-style features they may well match or beat us. Our hypothesis is narrower: on multi-class tasks over raw early-packet byte and length sequences, where Kaleidoscope's neural models excel, trees need too many feature partitions to stay within budget. The direct comparison at equal budget settles this per task, and if trees win everywhere we will report that the multibit neural route is not worth it.
Why not just add a coprocessor like Kaleidoscope or FENIX? These buy accuracy with extra hardware. Kaleidoscope adds 256 to 352 ns of mouse-flow inference latency plus a 15.5 ns query, and FENIX adds an FPGA. We do not claim to beat their accuracy. We quantify how much of it fits into commodity switches already deployed, which matters wherever adding silicon is not an option.
Why not just compile candidates in a loop instead of building a resource model? This is a fair alternative when the search space is small. We will run compiler-in-the-loop random search with the same wall-clock budget as our model-guided search. If it finds equally good configurations, the resource model's contribution shrinks to an explanatory tool, and we will present it that way.
Why not just binarize harder and widen? Wider binarized layers increase popcount stage depth and per-stage accesses. The fan-in-only versus bit-width-only ablations show directly whether width or precision is the better use of a given budget.
Proposed Experiments
The datasets are ISCX-VPN-2016 and USTC-TFC for multi-class encrypted and malware traffic, UNSW-NB15 plus the Aidra, Bashlite and Mirai traces for anomaly detection, and CAIDA 2018 replayed as background load for concurrency. Metrics are macro-F1 and accuracy; compiled stages, SRAM, TCAM and stateful accesses; recirculation count; and, when hardware is available, per-packet latency and throughput with the reference forwarding program co-resident. Budgets are swept from 25% to 75% of remaining stages and SRAM, and from 10^4 to 10^6 tracked flows, so that the per-flow feature-storage term is actually exercised.
Three checks are decisive. First, on the compiled grid, the resource model must rank fit and accuracy-at-budget substantially better than parameter or operation count, which would be a MOSAIC-style ranking flip on a switch. If the rankings agree, the central claim fails. Second, at matched budget on ISCX-VPN-2016 and USTC-TFC, the compressed table-native networks must beat the best tuned Planter tree and binarized network in macro-F1 without recirculation. Third, as the number of tracked flows grows, accuracy should degrade more gracefully when feature precision $q_i$ is jointly optimized than when features are stored at fixed width. That result would show switch memory acting as a model-design variable and not only as a deployment limit.
References
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
- All You Need Is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
- Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts
- QSketch: An Efficient Sketch for Weighted Cardinality Estimation in Streams
- Automating In-Network Machine Learning
- CyberSentinel: Efficient Anomaly Detection in Programmable Switch using Knowledge Distillation
- Inference-to-complete: A High-performance and Programmable Data-plane Co-processor for Neural-network-driven Traffic Analysis
- Restoring Pruned Large Language Models via Lost Component Compensation
- In-Network Volumetric DDoS Victim Identification Using Programmable Commodity Switches
- A Fast and Compact Invertible Sketch for Network-Wide Heavy Flow Detection
- ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
- FENIX: Enabling In-Network DNN Inference with FPGA-Enhanced Programmable Switches