When a large language model is split across many small chips ("chiplets") sharing one package, every layer must stop and exchange partial results so that each chiplet holds the full hidden state again. On a wired grid of chiplets this exchange takes many hops. One prior study found that inter-chiplet communication consumes about 72.6% of Transformer execution time once a package holds 18 chiplets. A radio antenna built into the package can reach every chiplet in a single transmission. The catch is that the radio channel inside a metal package is lossy and noisy, and making it reliable costs bandwidth and power. This note proposes to stop paying that cost uniformly. Language-model activations are highly non-uniform: a small, fixed set of outlier channels and the per-block scale factors carry most of the information that matters for accuracy, and the bulk tolerates noise. So the outliers and scales should travel over protected wired links, while the bulk travels as low-bit values over a lightly protected wireless broadcast that is allowed to make bit errors.
In two sentences: in tensor-parallel decoding, the "gather everyone's shard to everyone" step is a broadcast, which in-package radio does in one hop. We make the radio cheap enough to matter by deciding, per activation channel, how much error the model can absorb, instead of protecting every bit equally. The claim is narrow and testable. For latency-bound decode collectives on packages of roughly 9–18 chiplets, error-tolerant wireless broadcast plus outlier-protected wired transport should beat the best wired alternative, which is wired transport with activation compression. During bandwidth-bound prefill we expect it to lose to that same alternative. Measuring the crossover is part of the contribution.
What exists and what is missing
The closest work, Exploring the Potential of Wireless-enabled Multi-Chip AI Accelerators, adds millimeter-wave transceivers to a 3×3 chiplet grid. It sends multicasts and long-distance transfers over the radio according to a fixed hop threshold and a random injection probability, and reports a 10% average and roughly 20% peak speedup at 96 Gb/s. It treats the radio as an ideal, error-free pipe. It does not model autoregressive decoding, and its routing ignores what the bits mean.
Its companion characterization, Communication Characterization of AI Workloads for Large-scale Multi-chiplet Accelerators, motivates this proposal. It shows that multicast traffic drives congestion as chiplet count grows. Gemini supplies the simulator and the strongest purely wired competitor: its mapping co-exploration cuts Transformer hop counts by 34.2% and die-to-die traffic by 74%.
On the model side, Communication Compression for Tensor Parallel LLM Inference quantizes activations block-wise before the reduction step in Llama 2 inference over wired links. It notes that the compression overhead stops paying off when links are fast. We reuse its block quantizer.
Two papers supply the outlier-protection idea. Cambricon-LLM protects only outliers with on-die error correction for flash storage and keeps 92–95% accuracy at a raw flash error rate of 2×10⁻⁴. We transplant that idea from storage errors to radio errors. ASVD and Task-Circuit Quantization independently show that accuracy hinges on a small fraction of channels or weights.
The physical constraints come from Channel Characterization for Chip-scale Wireless Communications within Computing Packages and Millimeter-Wave Propagation within a Computer Chip Package. Both are full-wave simulations, not hardware measurements. They find that how fast signal strength falls with distance (the path loss exponent $n$, which is 2 in free space) depends heavily on packaging:
| Package type | Path loss exponent $n$ |
|---|---|
| Optimized single-chip package | 0.75–0.93 |
| 2.5D silicon interposer with air gaps | 1.55 |
| Multi-chip module (signals pass through bulk silicon) | 4.27 |
Worst-case coupling between antennas can sit near −81 dB until the silicon is thinned and the design is co-optimized. These numbers constrain the proposal. Bit-error rate will differ by receiver location, and on multi-chip modules the radio may be unusable. We treat package type as an experimental axis rather than assuming it away.
Core mechanism
In tensor-parallel decoding with $N$ chiplets, each all-reduce of a hidden vector of size $H$ splits into two phases. First, a reduce-scatter leaves each chiplet with a $1/N$ shard. Second, an all-gather makes every chiplet broadcast its shard to every other chiplet. On one shared radio channel, the all-gather's airtime is
$$T_{\text{wl}} \approx \frac{H\,b}{R_{\text{eff}}(\epsilon)} + N\,t_{\text{ovh}},$$
where $b$ is bits per activation after quantization, $t_{\text{ovh}}$ is per-transmission preamble and turnaround time, and $R_{\text{eff}}(\epsilon)$ is the useful data rate after error-correction overhead needed to reach target bit-error rate $\epsilon$. The data term does not grow with $N$. That is the structural advantage over a wired ring or mesh, whose latency grows with the hop count.
The lever is $\epsilon$. Accepting a higher bit-error rate on bulk activations raises $R_{\text{eff}}$ by allowing weaker coding or denser modulation at the same transmit power. It can also allow lower power, which matters thermally. The channel indices known to carry massive activations are fixed per model, so they can be identified once offline. These channels and all block scale factors go over the wired mesh with strong protection. The low-bit integer bulk goes over the radio.
Because a broadcast is heard by every chiplet at different signal strengths, each receiver gets an independently corrupted copy. Chiplets therefore compute their next shards from slightly different hidden states. This is still a valid noisy forward pass, but errors can accumulate through the key-value cache over long generations. We measure this effect directly rather than assume it away.
Research Program
The work has three parts.
Channel to error model. We take S-parameters from full-wave package models built to the published configurations: an interposer with an aluminum nitride heat spreader and thinned dies, plus a multi-chip module counterpart. From these we derive a per-receiver bit-error rate as a function of coding rate and modulation, using a standard link budget. Because the source channel studies are simulation-only, every downstream result inherits that limitation, and we will state it with each result.
Model tolerance. We inject receiver-specific bit flips into block-quantized activations during PyTorch inference of the Llama 2 family. We sweep bit width, $\epsilon$, which channels are protected, and generation length. The output is accuracy as a function of $\epsilon$ for each policy.
System simulation. We extend the Gemini simulator with autoregressive prefill and decode traffic, including tensor-parallel collectives and key-value cache placement. We plug in the measured $R_{\text{eff}}(\epsilon)$ curves and run 3×3 and 6×3 chiplet configurations. The routing policy is fixed by design, not tuned online: outliers and scales go wired; bulk goes wireless with $\epsilon$ chosen as the largest value whose accuracy loss stays under a preset budget.
Baselines
The naive reference is a wired mesh carrying uncompressed FP16 tensor-parallel collectives, with the default Gemini mapping. Every speedup is reported relative to it.
The strongest competitor is wired transport with block-wise activation quantization following Communication Compression for Tensor Parallel LLM Inference. That paper reports a 3.5–4.5× reduction in communication volume, up to 2× lower time-to-first-token on bandwidth-limited links, and less than 3% WikiText-2 perplexity increase in most cases. It is the strongest competitor because it attacks the same bytes with no new hardware. We will run it with the same quantizer, bit widths, calibration data, and Gemini mapping search budget as our method, and combine it with the best Gemini mapping. Beating it means lower per-token decode latency at matched accuracy, with perplexity within the same 3% band and MMLU within one point, on at least the 6×3 configuration.
Gemini's wired mapping co-exploration, reported at 1.98× over Simba and Tangram, gets the same annealing budget as our setup.
The prior wireless design (Exploring the Potential of Wireless-enabled Multi-Chip AI Accelerators) is re-implemented with its hop threshold and injection probability. It has a 10% average speedup at 96 Gb/s on non-LLM-decode workloads, and no reported result was found on Lacuna for it on LLM decoding. It runs with an ideal, error-free channel and again with our realistic channel.
Two ablations isolate the key ingredient. The first is uniform protection: all bits over wireless at a conservative $\epsilon$. The second keeps the same number of protected channels but chooses them at random rather than by outlier index. If the random-channel ablation matches the full method, the outlier-awareness claim fails.
Why not just…?
Why not just compress activations and keep the wires? This is the real threat. Compression shrinks the data term, but in decode the messages are small and the cost is dominated by per-hop latency, which compression does not remove. The same paper also notes that compression overhead can cancel gains on fast links. The decode-versus-prefill split settles this. We predict wired compression wins prefill. If it also matches us on decode at 18 chiplets, the proposal's claim is refuted, and we will report that.
Why not just choose a better parallel mapping, such as pipeline instead of tensor parallelism? Gemini-style mapping reduces traffic substantially, and pipeline splitting avoids per-layer collectives. However, it raises single-stream decode latency through pipeline bubbles. We include pipeline and hybrid mappings in the Gemini search so the wired side gets its best shot.
Why not just use strong error correction on the radio and treat it as a perfect pipe? That is the uniform-protection ablation. If the gain in $R_{\text{eff}}$ from tolerating errors is small at the measured coupling levels, the uniform version will match ours, and the contribution reduces to the prior wireless design applied to LLMs.
Why not just avoid wireless because multi-chip-module packages ruin the channel? With $n = 4.27$ this may be true. We run the multi-chip module configuration explicitly and expect it may fail. Then the result becomes a packaging recommendation: the method needs interposer-style packages.
Decisive checks
Benchmarks are WikiText-2 perplexity, MMLU, and GSM8k for longer generations, where error accumulation would show, on Llama 2 models sharded over 9 and 18 chiplets. We report per-token decode latency, time-to-first-token, energy per token, and accuracy versus $\epsilon$.
Three outcomes decide the proposal. First, the accuracy-versus-error curve for outlier-protected transport must stay flat to a bit-error rate meaningfully higher than for random protection; otherwise the core idea is wrong. Second, on the 6×3 interposer configuration, decode latency must beat wired-plus-compression at matched accuracy. Third, the prefill crossover must appear where the airtime relation predicts. If the second check fails, we will publish the tolerance curves and the crossover analysis as a bound on when in-package radio can help LLM serving.
References
- Exploring the Potential of Wireless-enabled Multi-Chip AI Accelerators
- Communication Characterization of AI Workloads for Large-scale Multi-chiplet Accelerators
- Gemini: Mapping and Architecture Co-exploration for Large-scale DNN Chiplet Accelerators
- Communication Compression for Tensor Parallel LLM Inference
- Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM
- ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
- Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression
- Channel Characterization for Chip-scale Wireless Communications within Computing Packages
- Millimeter-Wave Propagation within a Computer Chip Package