Counting the Microphone: Offloading Decisions That Include the Sensor Front-End

Battery-powered wearables that listen for keywords or track heart rate have to decide, many times per second, whether to classify locally or send data to a nearby server. The mobile edge computing literature treats this as a trade between processor energy and radio energy. It assumes the data already sits in memory, ready to send. For always-on sensors that assumption is wrong where it matters most: the circuitry that samples and converts the analog signal can dominate the power budget, and a new class of analog front-ends can shrink the data by two orders of magnitude before it ever reaches a digital chip. This note proposes to treat the front-end as a decision the device makes, alongside where to classify, how fast to run the processor, and how hard to drive the radio. It also asks whether this shrinking of the data is what finally makes standard encryption of every transmitted packet affordable on such devices.

The concrete claim is this. Suppose a wearable can switch between three representations: raw 16 kHz audio, multi-bit analog band energies read out at about 80 Hz, and 1-bit analog band-presence signals. Suppose also that front-end energy, encryption energy, and an accuracy target all enter the per-slot cost. Then the best policy departs from what existing edge controllers produce. Standard AES encryption of everything that leaves the device drops from a significant cost to a negligible one. We will measure both effects rather than assume them, and the controller itself is borrowed, not invented.

Core Claim

Existing offloading controllers get their energy accounting wrong for sensing workloads, and fixing it changes their decisions. The quantitative motivation comes from three hardware papers. Sub-mW Keyword Spotting on an MCU replaces the multi-bit analog-to-digital converter (ADC) with analog filters and comparators that emit one bit per frequency band. The authors report that acquisition and conversion take up to 85% of the power budget in conventional always-on keyword spotting, and they reach 0.56 mJ per classification at 88.8% accuracy on the 10-class Google Speech Commands task, against about 94% for a full-precision digital pipeline. MSP-MFCC computes Mel-band energies in continuous time and cuts the ADC rate from 16 kHz to 80 Hz, at 0.72 µJ and 45.79 µs per frame. PEAF makes such analog features trainable end-to-end in simulation.

None of these papers considers a radio. On the other side, Power-Delay Tradeoff in Multi-User Mobile-Edge Computing Systems and Stochastic Joint Radio and Computational Resource Management give online controllers for choosing processor frequency, transmit power and offloading, with a provable trade between power and delay controlled by a single parameter. Both define a task by its digital size and nothing upstream of it.

We supply the missing term and the missing decision. We reuse the method of the second group of papers, and the first group supplies both the cost figures and the motivation.

The representation choice also governs privacy. Scrambling-based speech encryption via compressed sensing shows that compressed speech measurements still leak the speech envelope (relevance about 0.96). Their digital scrambling cure costs about 200 ms. We expect low-rate analog features to leak less than raw audio but not nothing, so they still need encryption. Because they are tiny, encrypting them should be cheap.

Research Program

We build a per-slot cost model and plug it into the drift-plus-penalty controller of Mao et al. That controller minimizes a queue-weighted drift plus $V$ times energy each slot, where $V$ is the knob trading energy against delay. The energy spent in slot $t$ is

$$E(t) = E_{\mathrm{fe}}(m) + e_{\mathrm{enc}} D_m \mathbb{1}[\text{offload}] + \begin{cases} \kappa f^2 c_m & \text{local} \\ P_{\mathrm{tx}} D_m / R(h,P_{\mathrm{tx}}) & \text{offload.} \end{cases}$$

The symbols are defined as follows. $m$ is the front-end mode. $E_{\mathrm{fe}}(m)$ is its acquisition energy per slot. $D_m$ is the resulting payload in bits. $e_{\mathrm{enc}}$ is the measured authenticated-encryption energy per bit. $\kappa f^2 c_m$ is the usual model of processor energy at clock frequency $f$ for $c_m$ cycles. $R(h,P_{\mathrm{tx}})$ is the achievable link rate under channel state $h$.

Accuracy differs by mode and by where inference runs. A small binary network runs on the device; a larger model runs on the server. We therefore add a virtual queue, which grows whenever the delivered accuracy falls below a target. This is the standard Lyapunov device for long-term constraints, applied here to accuracy.

Energies per mode come from two sources. First, the published figures above. Second, circuit-level simulations of PEAF-style filter banks with Monte Carlo component mismatch, so that analog variation shows up as accuracy spread and is not ignored. We measure $e_{\mathrm{enc}}$ ourselves on a Cortex-M-class microcontroller running AES in an authenticated mode. The file contains no reported per-bit figure for this. Heart-rate monitoring with the analog envelope front-end of PuLsE (5x bandwidth reduction, 5.8 mW) serves as a second workload, to test whether conclusions carry beyond audio.

Baselines

The naive reference is the fixed analog-binary pipeline of Sub-mW Keyword Spotting on an MCU, which always classifies locally and never transmits. Its reported point is 88.8% accuracy at 0.56 mJ per classification on 10-class Speech Commands V2. The proposal only matters if an accuracy target above that level can be met at a lower energy per correct classification than simply using a larger local front-end.

The strongest existing controller is the Lyapunov scheme of Power-Delay Tradeoff in Multi-User Mobile-Edge Computing Systems, run with its native cost model, a fixed raw-digital front-end and unencrypted payloads. Its headline result is a drop in execution delay from about 1000 ms (local-only) to 1.05 ms in its own simulator. For keyword spotting under our workload, no reported result was found on Lacuna for this controller. It will be run in the same multi-user simulator, with the same channel traces, the same arrival processes and the same sweep of $V$. We will then charge it, after the fact, the true front-end and encryption energy of the actions it chose. Beating it means that, at matched average delay and accuracy, our policy uses measurably less true energy across the $V$ sweep.

An ablation isolates the key ingredient. It gives the Mao et al. controller the front-end mode as a choice but hides front-end energy from its cost, so any gain can be attributed to accounting rather than to having more options. A second ablation replaces the Lyapunov controller with a myopic per-slot minimizer that uses our full cost model. All policies get the same tuning budget over $V$ and accuracy targets.

Why not just…?

Why not just always use the cheapest analog front-end and classify locally? This is a real threat. If the 1-bit pipeline's accuracy is acceptable, offloading adds nothing. The gap between 88.8% and about 94% reported by Cerutti et al. suggests applications with stricter targets will need either richer features or a server model. The experiment sweeps the accuracy target and reports the threshold below which the naive pipeline wins outright. We will state that threshold plainly rather than hide it.

Why not just a greedy per-slot rule with the corrected cost model? It may do nearly as well. With 80 Hz features, transmission may be so cheap that queues rarely build up. That is why we make the accounting, not the controller, the claim. The myopic ablation will show whether queue awareness matters. We expect it to matter mainly under deep fading and heavy multi-user load, where Mao et al. show that controllers unaware of queues let backlogs grow. If it does not matter, we will report that the correct cost model plus a simple rule suffices.

Why not lightweight or chaotic ciphers instead of AES? The chaos-based Huffman scheme of Usama et al. cuts compress-then-encrypt time on the Calgary bib file from 319.84 s to 137.32 s. Our hypothesis is that once payloads shrink by roughly 200x, standard AES is already negligible, so nonstandard primitives are unnecessary. Measuring AES energy as a share of total energy per mode settles this directly.

Why not stream raw audio and let the server do everything? This option stays in the action set as the raw-digital mode, so the controller can pick it whenever the channel is good enough. The leakage evaluation shows what the option costs in exposure.

Proposed Experiments

The keyword workloads are Google Speech Commands V2 (10- and 35-class) and TI-DIGITS, where MSP-MFCC reports 98.2%. Wake-word detection uses Mozilla Howl; PEAF reports only qualitative power gains there, so our numbers for it are new. For each front-end mode we train a small local binary network and a larger server model, and we inject simulated filter mismatch.

Metrics are energy per correct classification, deadline-violation rate, delivered accuracy against target, and encryption's share of total energy. Leakage is measured by training an eavesdropper to recognize words and speakers from intercepted plaintext payloads of each mode.

The first decisive check is whether our policy's energy–delay curve lies strictly below the post-hoc-charged Mao et al. curve at matched accuracy. The second is whether the accounting-only ablation recovers most of that gain. The third is whether AES costs under a few percent of energy for the analog modes but a material share for raw streaming. The fourth is whether eavesdropper accuracy on unencrypted analog features stays well above chance, which would justify encrypting even tiny payloads. Repeating the first check on the ultrasound heart-rate workload tests generality.