Paying for Channels Only Where Inputs Go: Traffic-Weighted Feature Budgets in Compressed Hierarchical Classifiers

A hierarchical classifier breaks a hard many-class problem into a tree of small decisions: first "vehicle or animal," then "dog or cat," and so on down to a single label. Each input travels only one root-to-leaf path. That property is already known to improve accuracy, and older work on shallow models showed that it also saves resources when each decision point, or node, looks at only the features it needs. In deep networks, however, every node still reads the same full-width feature vector from a shared backbone. So the tree adds branches but removes no computation. We propose that each node of a deep binary tree read and compute only a small, structured subset of the backbone's late-stage channels (whole feature maps, not scattered weights). Inference then computes only the channels needed along the path an input actually takes.

The central design question is how to split a fixed compute budget across nodes. The literature pulls in two directions here. One view holds that errors near the root are fatal because they cannot be undone further down. The other holds that leaves are fragile because they see little data. We resolve this by weighting each node's cost and each node's contribution to accuracy by its traffic, meaning the fraction of inputs that reach it. The compute budget, and optionally the numerical precision, then goes wherever it buys the most end-to-end accuracy per unit of expected computation. The claim to test is specific. At matched measured latency, a traffic-budgeted tree with node-specific channel subsets should beat both a flat network compressed to the same cost and a tree with uniform per-node budgets.

Core Claim and What Is New

Three strands of prior work each hold part of this idea. Embedding Feature Selection for Large-scale Hierarchical Classification selects features independently at each node of a text taxonomy, using filter criteria such as the Gini index. It reports up to 3× faster training, up to 45% less weight memory, and roughly 2% better Micro- and Macro-F1. Its companion version, Large-scale hierarchical classification with rare categories and inconsistencies, shows that tuning the number of features per node on validation data beats a uniform count. Both work only with linear classifiers on bag-of-words features. HD-CNN puts a CNN into a two-level hierarchy learned from the confusion matrix of a flat network. It runs only the top-$k$ fine-category expert heads at test time, which gives 2.8×–6.3× speedups over running all experts, and it compresses expert weights with product quantization. Every expert in HD-CNN, though, receives the same full representation. Hierarchical Metric Learning learns a separate distance metric at each node of a binary tree over frozen 4096-dimensional VGG features. It reaches 84.6% on NWPU-RESISC45, against 72% for a single global metric, but it does not compress the network.

This proposal differs from all three in two ways. First, the node-specific reduction happens inside the deep network, as channel subsets of the backbone's late layers, so it cuts real computation rather than only head parameters. Second, per-node budgets are set jointly by a traffic-weighted rule rather than by local validation at each node in isolation. We treat Naik et al. as the source of the method (per-node selection with adaptive budgets). We treat HD-CNN as the architectural starting point and the closest competitor. The file's cited evidence of compounding top-down errors in Goel et al. and of leaf overfitting in HiFSRR supplies the tension that the allocation rule must resolve.

Research Program

The system has four parts: a backbone, a binary tree over classes, channel subsets owned by each node, and a rule for sizing those subsets.

The backbone is a standard CNN, a ResNet family model. Its early stages are shared and always executed. Its last one or two stages are path-conditional: they are computed only for the channels that some node on the input's path needs. We build the binary tree by recursively bisecting the confusion matrix of a flat model trained on the same data. This follows HD-CNN's spectral clustering, applied recursively to get a full binary decomposition. It also sidesteps the known mismatch between hand-made taxonomies and a learner's actual confusions, a mismatch HD-CNN itself notes.

Each internal node $v$ owns a channel subset $S_v$ of the conditional stages and a small binary head. Because the subsets are whole channels, the computation on a path is a dense convolution restricted to the channels in the path's subsets combined, so the savings show up as real latency. Unstructured sparsity, by contrast, often reduces counted operations without reducing wall-clock time. Subsets are learned with a differentiable channel gate per node. A penalty, adapted from HiFSRR's recursive regularization, encourages a child to reuse its parent's channels and siblings to differ. This does two jobs. It keeps the channel union along a path small, which is what makes inference cheap. It also lets data-poor deep nodes borrow the features of their better-trained ancestors, which is our answer to leaf overfitting.

The allocation rule is the core contribution. Let $\pi_v$ be the fraction of inputs that reach node $v$, estimated on held-out data. Let $c_v(b_v)$ be the measured cost of the channels node $v$ adds beyond its ancestors when its budget is $b_v$. The expected per-input cost of the tree is

$$C = \sum_v \pi_v \, c_v(b_v),$$

because an input pays for a node only if it reaches that node. End-to-end accuracy is the traffic-weighted product of correct decisions along each path. So a node with little traffic is cheap to enrich but contributes little accuracy, and the root has the opposite profile. We allocate budgets greedily. Starting from a minimal allocation, we repeatedly grow the node with the largest ratio of validation accuracy gain to $\pi_v \Delta c_v$, stopping when $C$ reaches the target. Both quantities are measured, not modeled, so the rule makes no assumption about whether the root or the leaves matter more. Which nodes end up with large budgets is itself an empirical finding about the root-versus-leaf tension.

As a secondary axis, the same rule chooses a per-node bit width from {8, 4} for weights and activations. Quantization uses the clipping and per-channel methods of Banner et al., so precision is spent where traffic-weighted sensitivity is highest. We deliberately do not binarize nodes by default, because the file gives no evidence that 1-bit convolutions hold up in this regime.

Hard routing cannot recover from early mistakes. As a cheap safeguard, when a node's confidence falls below a threshold, we descend both children and pay the extra cost. That cost is charged into $C$, so the safeguard competes for budget like any other allocation.

Baselines

The naive reference point is the uncompressed flat network: the same ResNet, trained with the same recipe and augmentation, run in full precision. It anchors accuracy and cost. For context, HD-CNN reports lowering standard-CNN top-1 error by 2.65% on CIFAR-100 and by 3.1% on ImageNet with a Network-in-Network base. That shows hierarchy can help accuracy, though at higher cost than the flat model.

The strongest competitor on the actual question, accuracy at a fixed latency, is a flat network compressed to the same measured latency. We produce it by structured channel pruning of the same backbone, in the style of the feature-channel pruning in A Tiny Model for Fast and Precise Ship Detection via Feature Channel Pruning, followed by the full 4-bit post-training quantization of Banner et al.. Banner et al. report ResNet101 at 4-bit weights and activations on ImageNet reaching 75.0% top-1 against an FP32 reference of 77.2%, up from 62.6% for naive 4-bit. That result shows a well-quantized flat network loses only a couple of points, which is why this baseline is dangerous. We found no reported result on Lacuna for channel-pruned flat CNNs at matched latency on CIFAR-100, ImageNet, or NWPU-RESISC45, so we will produce these numbers ourselves.

Fairness rules apply across all comparisons. Every method starts from the same pretrained backbone and uses the same data. Each gets the same number of GPU-hours for pruning, fine-tuning, and tuning. Each receives an equal hyperparameter-search budget. All latency is measured on the same device and kernel library.

A second strong competitor is HD-CNN-style conditional execution at matched latency. Here experts read the full shared representation, top-$k$ routing is used, and $k$ and expert width are shrunk until cost matches. This tests whether node-specific channel subsets add anything beyond routing alone. For remote sensing we also compare against Hierarchical Metric Learning at its reported 84.6% on NWPU-RESISC45 and 94% on UC-Merced. It runs a full, uncompressed VGG-16, so matching it at a fraction of the compute counts as a win.

Two ablations isolate the key ingredients. The first gives every node the same channel budget at the same total expected cost; it isolates traffic-weighted allocation. The second removes the ancestral-inheritance penalty; it isolates the effect of reuse on the channel union along a path and on accuracy at deep nodes. Beating the compressed flat network means higher top-1 accuracy at equal measured latency across at least two of the three latency targets, by a margin larger than the seed-to-seed standard deviation. We will run three seeds.

Why not just…?

Why not just compress a flat network to the same cost? This is the real threat. HD-CNN's accuracy gains were measured against uncompressed flat models at higher cost, and Banner et al. show that careful 4-bit quantization of a flat ResNet costs about two points. Nothing in the file establishes that a tree wins at equal latency. The experiment settles it by plotting an accuracy-versus-latency curve for both methods. If the flat pruned-and-quantized network matches or dominates the tree everywhere, the central claim fails, and we will report that. We do have a reason to expect the tree can win: a flat network must keep channels for every class for every input, while a path needs only its own.

Why not just prune the shared backbone globally and put a tree on top? Global reduction is known to discard fine-grained features that matter only deep in the tree. Naik et al. found that adaptive per-node selection beat a global feature count, and Goel et al. found that a tree without node-specific metrics loses much of its advantage. Both results come from shallow or frozen-feature settings, though. Our ablation that replaces per-node subsets with one globally pruned set feeding all nodes, at matched cost, tests whether the finding carries over to deep networks.

Why not just use HD-CNN's top-$k$ conditional execution? It already saves 2.8×–6.3× over running all experts. Those savings are measured against running every expert, not against a flat model, and each expert still processes the full representation. The matched-latency HD-CNN baseline answers this directly. If shrinking expert width with top-$k$ routing matches our results, then channel specialization adds nothing.

Why not just give every node the same budget? Uniform budgets are simpler and avoid tuning a large number of per-node parameters, a concern Liu et al. raise for sparsity budgets in Doubly Approximate Nearest Neighbor Classification. The uniform-budget ablation measures what traffic weighting buys. If the gain is within noise, the allocation rule should be dropped in favor of the simpler design.

Proposed Experiments

Experiments run on CIFAR-100, whose coarse and fine labels also let us compare confusion-derived trees with the given taxonomy. They also run on ImageNet with ResNet-50, and on NWPU-RESISC45 and UC-Merced for the remote-sensing comparison. Metrics are top-1 accuracy, measured single-image latency on a mobile CPU and a desktop GPU, expected channel operations per input, and peak memory.

The first decisive check is the matched-latency comparison against the flat compressed network at three latency targets, roughly 25%, 50%, and 75% of the uncompressed model. It succeeds only if the tree lies above the flat network's accuracy-latency curve. The second check examines where the allocation rule puts capacity. We report budget and bit width as a function of depth and traffic. We also test whether moving budget toward the root or toward the leaves, against the rule's choice, hurts accuracy, which directly measures the root-versus-leaf tension. The third check verifies that the counted-operation savings match measured latency. If structured channel subsets do not speed up real hardware, the efficiency argument collapses even if accuracy holds. Finally, we measure how often the low-confidence dual-descent safeguard fires and how much it recovers, to quantify how much of the tree's residual error comes from early routing mistakes.