Do Cryptographic Benchmark Rankings Survive the Network? Hierarchical Encryption Under Emulated and Real Radio Conditions

Hierarchical identity-based encryption (HIBE) lets an organization derive a user's public key from a name, such as an email address or a role, instead of a certificate. It also lets each level of an authority tree (root, domain, department) issue private keys to the level below it, so no single server has to be online for every request. Papers in this area almost always show their advantage with a microbenchmark: they time the cryptographic operations in a software library on a desktop CPU and compare those times against an earlier scheme. Nobody checks whether that advantage still holds once the protocol runs over a real network, on a weak client device, with packet loss. This matters because the claimed gains are often a few milliseconds. For example, 2.34 ms versus 43.50 ms for signing in E-Tenon. A wireless round trip can take tens to hundreds of milliseconds, and E-Tenon's own design needs an extra communication round.

We propose a controlled study of when benchmark rankings between hierarchical encryption schemes reverse under deployment conditions. We will run the same scheme implementations at three levels of realism. The first is library-only timing. The second is software network emulation. The third is hardware-in-the-loop radio emulation on Colosseum, a testbed that routes real radio transmissions through programmable hardware reproducing fading and interference. From these runs we will fit a simple predictive cost model: device-calibrated compute time, plus rounds times round-trip time, plus a loss and fragmentation penalty. We will then report where that model is enough and where only the hardware tier reveals the truth. The deliverable is a measured answer to the question "is library timing a valid proxy?", together with a cheap correction that authors of cryptography papers could adopt.

Core Claim

Our hypothesis is that, for interactive hierarchical protocols, the end-to-end ranking between schemes is set mostly by three things. The first is the number of network rounds. The second is the message size relative to the link's packet size, because pairing-group elements are large and can fragment. The third is client CPU speed. Raw pairing speed on a desktop matters much less. Two consequences follow. The first is that at least one published "faster" scheme will lose end-to-end under realistic wide-area or wireless round-trip times. E-Tenon's multi-signature variant, with its extra round, is the leading candidate. The second is that an additive model calibrated on the target device will predict median handshake latency closely. Its failures will cluster in low-signal, bursty-loss conditions, where retransmission timers interact with fragmented large messages.

If that second point holds, most papers do not need a hardware testbed; they need the corrected model. If it fails, hardware-in-the-loop evaluation is actually necessary, and we can say for which regimes. Both outcomes are useful, and the experiments are designed to tell them apart.

Positioning Against Prior Work

The motivating evidence comes from the hierarchical encryption papers themselves.

Some papers validate only with isolated operation timings. Liu et al. build a master-key-free hierarchical attribute-based scheme and time it in the Java Pairing-Based Cryptography library (JPBC). They report roughly 2–3 ms decryption at 30 attributes versus about 7 ms for prior schemes. E-Tenon pairs multi-level attribute-based encryption with multi-signatures and reports 761.28 ms decryption versus 1249.28 ms for Zhang et al. E-Tenon's authors acknowledge an extra round they never measure over a network.

Others rely on analysis alone. Wan, Ren and Preneel give an HIBE roaming protocol that finishes in 4 local messages without contacting the home network. They evaluate it only by counting operations, never over the air.

None of these papers is contradicted by our study. Their security proofs and operation counts stand. We test only whether their performance rankings transfer.

The underlying primitives, Boneh–Boyen selective-ID IBE and constant-size-ciphertext HIBE, supply the key-delegation building blocks we implement. Their message sizes as a function of tree depth are exactly what our fragmentation term needs.

Colosseum supplies the method for the hardware tier. It has 128 radio nodes, a channel emulator with 65,535 paths, and a measured 3.18 dB average signal-quality loss from Wi-Fi interference. It has never been used for identity-based handshakes. Its 1×1 km delay bound limits what distances we can emulate.

The broader reality-gap literature (tool-use sandboxes in Agarwal et al., uniform-link storage simulation in TFR-LRC, simulated oracle committees in DecTest) warns about the same failure in other fields. Those papers document the gap but do not quantify it.

The closest thing to our idea would be "run a protocol on a testbed." What we add is a three-tier comparison with a prediction model, so the result generalizes beyond one testbed run.

Research Program

Implementations. We will write one implementation per scheme on a single pairing library, so that differences reflect the protocols rather than the code bases. The schemes are:

  • the Wan et al. roaming handshake built on constant-size HIBE, at hierarchy depths 2 through 5;
  • the Liu et al. hierarchical attribute-based access scheme;
  • E-Tenon, in both its multi-signature form and a single-signature variant;
  • a conventional reference: elliptic-curve signatures with short-lived X.509 certificates and cached certificate chains.

Each protocol runs as a real client–server program over TCP/TLS, with explicit timeouts and retransmission. Before any network run, each reimplementation will be checked against its paper's reported library timings in ratio terms. If our speed ratios do not match the published ones, we report that mismatch first.

Test tiers. Tier A reproduces the papers' library-only timing. Tier B adds Linux traffic shaping, using fixed delay, random loss and bandwidth caps drawn from Tier C measurements. Tier C runs on Colosseum: clients and authorities sit on separate radio nodes, the IP link is carried over srsRAN cellular, and scenarios include clean channels, mobility fading, and the Wi-Fi co-channel interference case.

Client CPU conditions. The client runs at full speed, under frequency and cgroup throttling, and separately on physical low-power ARM boards over a lab Wi-Fi link. Colosseum hosts are not constrained devices, so the throttling is only a proxy and we validate it against the boards.

Hierarchy stress test. We degrade only the link to one intermediate domain authority, then measure key-issuance stalls and handshake failures for users below it. This tests the untested assumption that delegation removes bottlenecks when the links in the tree are uneven.

Cost model. For each scheme we predict end-to-end latency as

$$T = C_{\text{dev}} + R \cdot \text{RTT} + \lceil S/M \rceil \cdot p \cdot \tau,$$

where the terms are:

  • $C_{\text{dev}}$: the scheme's compute time measured on the client device;
  • $R$: the number of sequential network rounds;
  • $\text{RTT}$: the measured round-trip time;
  • $S$: total handshake bytes;
  • $M$: the path's maximum packet size;
  • $p$: per-packet loss probability;
  • $\tau$: the retransmission timeout.

The loss term assumes independent losses, so each fragment adds an expected $p\tau$ of delay. Where the model under-predicts, that points to bursty or correlated loss that it does not capture.

Baselines

The naive reference point is the published library timings taken at face value. Examples include E-Tenon's 2.34 ms signing and verification against 43.50 ms and 25.84 ms for Zhang et al., its 761.28 ms decryption against 1249.28 ms (E-Tenon), and Liu et al.'s 2–3 ms against about 7 ms (Liu et al.). We will reproduce these in Tier A on identical hardware. The model "beats" them if its ranking of schemes agrees better with Tier C, measured by Kendall rank correlation over all scheme pairs and conditions.

The strongest existing evaluation method is software network emulation with realistic delay and loss, our Tier B. It is the tool a careful systems reviewer would ask for. I found no reported result on Lacuna for HIBE or attribute-based encryption handshakes under network emulation, so we establish this baseline ourselves. Tier B receives parameters measured from Tier C at matched signal levels, which is generous to it. Our proposal beats it only if Tier C reveals ranking reversals or p99 latency errors above 25% that Tier B, with the best-fit parameters, does not reproduce.

The deployment baseline is short-lived-certificate PKI with cached chains. It runs on the same devices and links and gets the same tuning effort on timeouts and caching. I found no reported result on Lacuna for this comparison. HIBE "wins" only if it completes roaming authentication faster or more reliably when the home authority is far away or unreachable.

Two ablations isolate the model's ingredients. The first drops device calibration and uses desktop compute times. The second drops the fragmentation term. Each ablation shows how much of the prediction's accuracy that term contributes.

Why not just…?

Why not just use ns-3 or traffic shaping and skip hardware? This is the real threat, and Tier B exists to test it directly. If Tier B with measured parameters matches Tier C rankings and tail latencies, we will say so, and the contribution becomes the calibrated model plus the finding that software emulation suffices. The reason to expect some divergence is that Colosseum's interference and fading produce correlated, bursty loss (Colosseum), which independent-loss models miss. That matters most for large, fragmented pairing messages.

Why not just use standard PKI instead of HIBE? It may well win on a well-connected network. Wan et al.'s argument for HIBE rests on not contacting the home network online during roaming (Wan et al.), and that advantage only appears at high inter-domain round-trip times. We run PKI as a baseline and report where the crossover occurs, rather than assuming HIBE is better.

Why not just count rounds analytically? Counting operations and rounds is what Wan et al. already do, and it is the model with the fragmentation and device terms removed. The ablations show whether those extra terms reduce prediction error enough to matter.

Proposed Experiments and Decisive Checks

We sweep:

  • hierarchy depth (2–5);
  • concurrent clients (1–64);
  • round-trip time (10–200 ms in Tier B, native in Tier C);
  • signal condition (clean, fading, Wi-Fi interference);
  • client CPU class.

For each cell we record median and 99th-percentile handshake completion time, failure and timeout rate, and client energy on the ARM boards.

Three results would be decisive. The first is whether E-Tenon's multi-signature variant loses to its single-signature form, or to Zhang-style designs, once the round-trip time exceeds some threshold, and whether the model predicts that threshold. The second is whether the Tier B versus Tier C divergence in tail latency concentrates in the interference scenario at larger hierarchy depths, as the fragmentation hypothesis implies. The third is whether degrading one intermediate authority's link raises failure rates only in its subtree, which is the containment that delegation promises, or spreads to other branches.

We will release the harness and all traces so future scheme papers can report end-to-end numbers alongside library timings.