When a camera on a farm robot, a microphone on a street pole, or a phone has to recognize what it sees, the usual choice is to run a small model locally or ship the input to a bigger model on a nearby server. Many real decisions, though, only need a coarse answer most of the time. "Weed or crop?" is enough to decide whether to spray. "Indoor or vehicle?" is enough to pick a noise profile. The exact species or scene matters only occasionally. We propose to train the small on-device encoder so that its compressed output is ordered by the label taxonomy. The first few channels of the transmitted code are trained to answer the coarsest question. Each further block of channels is trained to add what the next finer level needs. The device, or the server, can then stop the transmission at whatever level of detail the current decision requires.
The opportunity comes from two findings that have not been combined. Split computing places a shallow encoder on the device and the rest of the network on a server. Its main open problem is that the device must send a whole intermediate representation before any answer is available. Hierarchical multi-task learning, meanwhile, shows that coarse taxonomic levels are much easier and more accurate than fine ones: family-level accuracy of 94.07% versus 79.40% for species on iNatAg (iNatAg). If coarse answers need fewer bits, a representation whose prefix carries them should reach the coarse decision at a fraction of the bandwidth of a flat representation, and it should give up little fine-grained accuracy when the full code is sent.
Core Claim
A supervised compressed representation, meaning an encoder trained end to end so that its quantized output is cheap to transmit and still useful for a downstream classifier, can be trained with one classification loss per taxonomy level. Each loss sees only a nested prefix of the latent channels. We claim three things. First, at matched transmitted bits, such a code resolves coarse levels more accurately than a flat supervised code or a generically ordered code. Second, at full rate it loses at most a small margin on the leaf level. Third, it moves the hierarchy from network depth into bitstream width.
The third point matters because two constraints pull against each other. Neural Sculpting finds that networks separate hierarchical sub-functions only when their depth at least matches the depth of the task hierarchy. SplitEE finds that running beyond roughly six layers on the device costs more than offloading. Nested channel prefixes give a shallow device encoder a way to expose the hierarchy without growing deeper.
Concretely, let $z$ be the quantized latent with $C$ channels, and let $z_{1:c_k}$ be its first $c_k$ channels, with $c_1 < c_2 < \dots < c_K = C$ for taxonomy levels $k = 1 \dots K$ ordered from coarse to fine. We train
$$ \mathcal{L} = \sum_{k=1}^{K} \lambda_k \, \mathrm{CE}\big(y_k,\ h_k(z_{1:c_k})\big) + \beta \, R(z), $$
where $y_k$ is the level-$k$ label, $h_k$ is a server-side head for level $k$ that reads only the prefix, $\lambda_k$ weights each level, $R(z)$ is the estimated bit cost from a learned entropy model, and $\beta$ trades rate against accuracy. Each prefix is entropy-coded on its own, so stopping at level $k$ costs only the bits of $z_{1:c_k}$.
Relation to Prior Work
The closest work is Ladon. It trains one lightweight mobile encoder whose single compressed code feeds classification, detection and segmentation heads on a server, and it reports up to 95.4% latency and 88.2% device-energy reduction. Ladon supplies our base method and our hardware protocol. Its tasks are a flat, heterogeneous set, however, and the entire code is sent before anything is decided. We keep the shared encoder but give its code an internal order tied to a taxonomy, which allows partial transmission. Its entropy-bottleneck predecessor, the Entropic Student in the SC2 Benchmark, also supplies the method. It is the tool that shows supervised codes beating JPEG, WebP and BPG with an encoder about 40 times smaller than neural image compressors. We build on its codebase, sc2bench.
Three distributed-inference papers use a physical hierarchy of device, edge and cloud, but every tier solves the same flat task.
- SplitEE exits locally when confidence exceeds a fixed threshold and uses a bandit to choose where to split.
- DIMEE routes inputs to tiers upfront by embedding similarity to easy, moderate and hard pools.
- Resource-Constrained Edge AI with Early Exit Prediction predicts upfront whether to skip intermediate exits and adapts thresholds to bandwidth.
These papers motivate the proposal: they show that sequential exits carry a hidden compute tax and that fixed thresholds fail under changing bandwidth. They differ from it because their exits predict the full label space and their offload is all-or-nothing. BAJ3O is "hierarchical" only in network topology. It places whole models across tiers for independent vision tasks, a scheduling layer that sits on top of what we propose and does not replace it.
On the hierarchical side, the level-specific heads of HYDRA supply the head design. Its result that plain level-specific heads on a standard encoder match graph-based hierarchy models is why we do not use graph modules. Xu et al. show multi-level objectives acting as regularizers in acoustic scenes, raising accuracy from 75.19% to 77.86%. They also warn that semantic groupings can conflict with acoustic similarity. That warning constrains us, and we test it directly (see the scene-grouping check below).
The "LoRA paradox" in iNatAg is a second constraint: low-rank adaptation degrades compact backbones. We therefore fully train the small device encoder rather than adapting it with low-rank updates.
Research Program
Encoder and training. We will build the taxonomy-ordered encoder by modifying the Entropic Student and Ladon training code. We will replace the single task loss with the prefix-restricted loss above and use a separate entropy model per channel block. Prefix sizes $c_k$ will be chosen by sweeping a small grid on validation data.
Inference policies. Three policies will use the code.
- Application-driven: the application states in advance which level it needs, and only that prefix is sent.
- Device-side refinement: a tiny head on the device reads the coarse prefix $z_{1:c_1}$ directly. It is cheap because it sees only a few channels, and it decides whether more blocks are needed. This is a taxonomic early exit that costs no extra encoder depth.
- Server-pull: the server receives the coarse prefix and requests further blocks only if its coarse posterior is ambiguous among children that matter for the decision.
Consistency. Parent and child predictions can contradict each other, a problem HYDRA also notes. We will compare unconstrained fine heads with fine heads masked to the children of the decoded coarse label, and report how often errors propagate from parent to child.
Analysis. We will measure how much mutual information each channel block carries about each level. The goal is to see whether the ordering is real or whether fine information leaks into the coarse prefix.
Baselines
Naive references. Two simple points anchor the comparison. The first sends a JPEG or WebP image at several quality settings and runs a full hierarchical classifier on the server. The second runs the whole model locally on the device, where it fits. SC2 Benchmark reports that such codecs lose to supervised codes, but it evaluated only flat labels, so we rerun them here with the same server model and plot accuracy at each level against bits.
Flat supervised code. The strongest existing competitor is a flat multi-head supervised code: a Ladon-style encoder trained with all taxonomy-level heads reading the full latent, sent in full. It is the direct predecessor and already shares one code across tasks. We will train it with the same encoder architecture, data, rate range and tuning budget (the same number of hyperparameter trials over $\beta$ and learning rate) as our method.
As an accuracy ceiling on iNatAg, the uncompressed hierarchical model reports 79.40% species, 89.83% genus, 94.07% family and 92.38% crop-versus-weed accuracy (iNatAg). For Ladon's accuracy under compression, the file gives only the qualitative statement that it "outperformed or rivaled" lightweight models, so there is no reported result on Lacuna for Ladon's rate-accuracy on a taxonomy benchmark. The same holds for the Entropic Student: no reported rate-accuracy numbers on Lacuna.
We count the flat supervised code as beaten if two conditions hold. At the bit rate where the flat code reaches a given family accuracy, our prefix reaches the same accuracy with substantially fewer bits, with a target of at least 2 times fewer. At full rate, our leaf accuracy stays within one point of the flat code.
Flat early-exit offload. This baseline follows SplitEE. A local exit predicts leaf classes, coarse probabilities are obtained by summing each parent's children, and when confidence is low the full intermediate code is offloaded. It is tuned with the same threshold search as our device-side refinement policy.
Generic ordered code (key ablation). This ablation uses the same prefix structure, but every prefix is trained with the leaf loss only. Channels are ordered by nested dropout, which randomly truncates the code during training so that earlier channels become more important, but no taxonomy supervision is applied. It isolates whether the taxonomy labels matter or whether any importance ordering suffices.
Hierarchy-ablation. A second ablation shuffles the parent assignments of the leaf classes at random. It tests whether gains come from the real taxonomy or merely from having auxiliary grouped losses.
Why not just…?
Why not just marginalize a flat leaf classifier into coarse probabilities? Summing a leaf softmax over each parent's children does give a coarse answer. But the input to that classifier still has to arrive in full, so it saves compute, not transmission. It also inherits the leaf classifier's errors on hard, long-tail classes. HYDRA's +2.52 Macro-F1 gain over a flat encoder on RCV1-V2 (HYDRA) suggests explicit level supervision helps, but that result is on text with full representations. The flat early-exit offload baseline tests this alternative directly at matched bits.
Why not just order the code generically and skip the taxonomy? This is the real threat. Nested dropout alone may place the most discriminative information first, and coarse distinctions may be exactly that information. If the generic ordered code matches our coarse accuracy at matched rate, the honest conclusion is that ordering, not taxonomy, drives the gain. We would then report the method as a generic progressive split code. We expect taxonomy supervision to help most where fine visual cues cut across coarse groups. Crop-versus-weed on iNatAg is one such case: that grouping is agronomic rather than visual.
Why not just run a tiny separate coarse model on the device and offload the raw image when detail is needed? This design pays for two encoders. On a detail request it also pays a full image upload, which the SC2 Benchmark shows is inefficient compared with supervised codes. In our scheme the coarse prefix is reused as the start of the fine code, so nothing is sent twice. We will include this two-model design, matched in device FLOPs, as an extra baseline and compare total energy per decision on real traffic mixes.
Proposed Experiments
Datasets. The main benchmark is iNatAg, which has species, genus and family levels plus a crop-versus-weed label (iNatAg). We also use ILSVRC 2012 through sc2bench with a WordNet-derived grouping, and the DCASE 2016 acoustic-scene set with 3 coarse and 15 fine classes (Xu et al.). DCASE tests whether the idea carries over to audio and to a semantic grouping known to conflict with perception.
Metrics. We report accuracy at each level against transmitted bits per sample, and the area under that curve per level. We also report end-to-end latency and device energy on an NVIDIA Jetson Nano and a Raspberry Pi, matching the hardware classes used by Ladon and Resource-Constrained Edge AI. Bandwidth is emulated from 0.1 to 100 Mbit/s. Finally, we report energy per decision under traffic mixes in which 50%, 80% or 95% of requests need only the coarse level.
Decisive checks.
- Prefix rate. On iNatAg, the family-level prefix must reach within one point of the flat supervised code's full-rate family accuracy while sending at most half its bits.
- Taxonomy dependence. The shuffled-taxonomy and generic-ordered ablations must fall measurably below the real-taxonomy code at the coarse prefix. If they do not, the taxonomy claim is withdrawn.
- Scene grouping. On DCASE, we compare the semantic grouping with a data-driven grouping built by clustering the confusion matrix. If the clustered grouping yields a better coarse prefix, that would show the ordering should follow perceptual structure rather than the semantic taxonomy. This is the failure mode Xu et al. warn about.
- Energy. Under the 80% coarse-only mix, device energy per decision must beat both the SplitEE-style flat offload and the two-model design at matched accuracy.
References
- iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2, 959 Crop and Weed Species
- Neural Sculpting: Uncovering hierarchically modular task structure in neural networks through pruning and network analysis
- SplitEE: Early Exit in Deep Neural Networks with Split Computing
- A Multi-Task Supervised Compression Model for Split Computing
- SC2 Benchmark: Supervised Compression for Split Computing
- Distributed Inference on Mobile Edge and Cloud: An Early Exit Based Clustering Approach
- Resource-Constrained Edge AI with Early Exit Prediction
- Batching-Aware Joint Model Onloading and Offloading for Hierarchical Multi-Task Inference
- HYDRA: A Multi-Head Encoder-only Architecture for Hierarchical Text Classification
- Hierarchical Learning for DNN-Based Acoustic Scene Classification