When two radios share a band and their receivers report what they heard back to the transmitters, the transmitters can cancel past interference. The report turns a collision into shared knowledge, and the next transmission can undo the collision over the air. A "cognitive" helper radio that has overheard the primary user's message can push this further by shaping its own signal to line up with the primary's. Both tricks only work if the two transmitters' outputs stay tightly correlated. Information theory has a precise and underused result here: codes whose outputs depend on very long stretches of their input destroy exactly this kind of correlation. The dependency spectrum, meaning how an encoder's output variance splits across single input bits, pairs of bits, and larger groups, bounds how correlated two distributed encoders can be.
We propose to make the dependency spectrum a measurable quantity for neural feedback codes, and to test one sharp prediction. In a causal cognitive interference channel, the cognitive transmitter learns the primary message over a noisy overhearing link, so it only ever holds a degraded copy. The best cognitive encoder should therefore depend on the primary message through low-order, short-window functions. Their optimal order and window should shrink predictably as the overhearing link gets noisier. If this holds, the spectrum measured on a trained code can tell us how long a memory window the code needs without sweeping it, and it explains why learned codes cooperate when they do. If it fails, we learn that learned codes escape the correlation bounds in a way the theory does not capture.
Core Claim
Three lines of work motivate the question without answering it. Shirani and Pradhan define effective length and the dependency spectrum for Boolean functions, and prove that random single-letter codes put their variance either on single letters or on huge blocks. As a result, those codes cannot preserve correlation between encoders (On the Sub-Optimality of Single-Letter Coding Over Networks; On the sub-optimality of single-letter coding in multi-terminal communications; On the correlation between Boolean functions of sequences of random variables). Heidari et al. use this to show that in constructed three-user interference channels with feedback, any optimal code must have bounded effective length. Their result is existence only: it gives no formula for the right length and no Gaussian or learned-code instance (Bounds on the Effective-length of Optimal Codes for Interference Channel with Feedback).
Separately, causal cognition work models the helper learning the primary message over a noisy link (On the Capacity of the Two-User Gaussian Causal Cognitive Interference Channel; On the Gaussian Interference Channel with Half-Duplex Causal Cognition). Mirmohseni et al. go further and parameterize the helper's head start by a scalar look-ahead $L$, from strictly causal to non-causal (On the Capacity of Interference Channel With Causal and Noncausal Generalized Feedback at the Cognitive Transmitter). All of this uses asymptotic random codes.
Finally, learned feedback codes empirically discover cooperation. Deepcode is an RNN feedback code that beats linear Schalkwijk–Kailath schemes by about three orders of magnitude in bit error rate (Deepcode: Feedback Codes via Deep Learning). MBAF is a Transformer code for the two-user multiple-access channel whose transmitters learn to decode each other from feedback (Do not Interfere but Cooperate: A Fully Learnable Code Design for Multi-Access Channels with Feedback). Nobody has asked what these learned encoders look like in dependency-spectrum terms, or placed them in a causal cognitive topology.
The claim rests on one quantitative mechanism. Suppose the cognitive transmitter's estimate of each primary bit agrees with the true bit with correlation $\rho$, set by the overhearing SNR. Write $f$ for the part of the cognitive encoder's output that depends on the primary bits, and $g$ for the primary encoder's output. Both decompose into orthogonal components by interaction order, and the correlation they can reach obeys a bound of the form
$$\mathrm{Corr}(f,g) \le \sum_{k\ge 1} \rho^{k}\sqrt{P_k(f)\,P_k(g)},$$
where $P_k(\cdot)$ is the fraction of a function's variance carried by its order-$k$ components. Its dependency spectrum is these $P_k$, refined by which positions each component touches. The $\rho^k$ factor arises because a product of $k$ independently noisy bits keeps only $\rho^k$ of its correlation. This is the classical noise-stability form that the Shirani–Pradhan bounds refine. The consequence is that order-$k$ structure in the cognitive encoder is worth roughly $\rho^k$ as much for alignment as order-1 structure. So as $\rho$ falls, a good code must shift its dependence on primary bits toward low orders and short windows.
Research Program
We will build a simulated two-user Gaussian interference channel with three links. The primary transmitter and receiver form one pair. The cognitive pair overhears the primary transmitter over an AWGN link with delay parameter $L$ and overhearing SNR $\sigma_o$. Both receivers send passive output feedback, with optional feedback noise. Block length matches MBAF ($K=51$ bits) so we can reuse its architecture and training recipe.
We will train MBAF-style Transformer encoders and decoders end-to-end over a grid of $(L, \sigma_o)$, interference strength, and feedback SNR.
The measurement tool extends the dependency spectrum beyond Boolean functions. Each transmitted symbol is a real-valued function of independent primitives: own message bits, the other user's message bits, and the channel noise draws. For independent inputs of any type, a real-valued function has a unique orthogonal variance decomposition by input subset, known as the Hoeffding or Efron–Stein decomposition. We estimate the order-wise and window-wise variance fractions with standard pick-freeze Monte Carlo estimators, the Sobol sensitivity indices used in sensitivity analysis. For binary message inputs this reduces to the Shirani–Pradhan spectrum. We report three quantities per trained code: the spectrum with respect to the primary message, an effective length (the smallest window holding 95% of that variance), and the measured correlation between the two transmitters' signals against the bound above. Before any scientific use, we will validate the estimator on codes with known spectra: uncoded transmission, parity codes over fixed windows, and random single-letter-style codes.
We then use the spectrum to design. From the spectrum of an unconstrained code trained at one operating point, we read off a predicted window $w^\star$. We then train a code whose attention is masked to $w^\star$ past symbols of the overheard stream. A second variant adds a penalty on variance mass above order $k^\star$. This directly addresses Gap 4 in the file: $L$ is a scalar, whereas the spectrum shows which look-ahead symbols the code actually uses.
Baselines
The naive reference point is the same channel with no cognition and no feedback, trained with the same architecture. Every cooperative gain is measured against it.
The orthogonal reference is GBAF-TD, the time-division learned feedback code used as MBAF's baseline. On the two-user AWGN multiple-access channel at $K=51$, 1 dB SNR, MBAF reports block error rate (BLER) $1.7\times10^{-7}$ against $1.8\times10^{-3}$ for GBAF-TD (Do not Interfere but Cooperate). We will first reproduce both numbers on that multiple-access channel as a sanity check of our training pipeline, and trust the new-channel results only if we come close.
The strongest existing competitor is MBAF itself, ported to the cognitive interference channel with unrestricted attention over all overheard and feedback history. It already learns cooperation from feedback, so any win from spectrum-guided design must beat it rather than a weak strawman. No reported result was found on Lacuna for learned codes on an interference channel with a causal cognitive transmitter, so the comparison numbers will be ours. Both models receive the same data generator, parameter count, training steps, and hyperparameter search budget.
We also run a hand-designed linear reference. This is the two-stage amplify-and-forward scheme of Suh and Tse (Feedback Capacity of the Gaussian Interference Channel to within 2 Bits), with the cognitive node forwarding its decoded primary estimate.
The upper reference is a genie cognitive node with the primary message known non-causally ($L=\infty$), the setting of Kang et al.'s cognitive-relay DoF analysis (The Degrees of Freedom of the Interference Channel with a Cognitive Relay under Delayed Feedback).
The ablation that isolates the key ingredient is a brute-force window sweep: masked-attention codes at every window size, with the best chosen on validation BLER. The spectrum-guided method "beats" the field if it satisfies three conditions. Its predicted $w^\star$ must land within one sweep step of the swept optimum at most operating points. It must match unconstrained MBAF's BLER within a factor of two at equal budget. And it must outperform unconstrained MBAF when the training budget is cut or when feedback is noisy.
Why not just…?
Why not just train an unconstrained Transformer and let it find whatever structure it needs? This is the real threat. If unconstrained MBAF matches every constrained variant at every operating point, the spectrum adds nothing as a design tool. The proposal survives this outcome only as a scientific claim: it should still predict how the learned code's spectrum shifts with $\rho$, and whether measured transmitter correlation sits near the bound. We say in advance that the design benefit is expected mainly under reduced training budgets and noisy feedback. Those are the regimes where MBAF has never been tested, since it assumes perfect feedback. If there is no benefit there either, we will report that.
Why not just sweep the memory window and skip the spectrum? A sweep finds the best window at one operating point but costs one training run per candidate. It also treats all positions in a window as equally used. The spectrum is estimated from a single trained model and reveals uneven dependence within a window. The decisive test is whether spectrum-predicted $w^\star$ tracks the swept optimum across the $(L,\sigma_o)$ grid. If it does not, the predictive claim is false.
Why not just assume non-causal knowledge, since causal cognition sometimes matches it? Cardone et al. show that equality holds only in some regimes. Elsewhere their half-duplex causal schemes sit within gaps of roughly 1 to 8.5 bits of the non-causal bounds (On the Gaussian Interference Channel with Half-Duplex Causal Cognition). More to the point, the mechanism we test is exactly what vanishes at $L=\infty$: with a perfect message copy, $\rho=1$ and high-order dependence costs nothing. The genie is kept as an upper curve, not as the model.
Why not just orthogonalize the users in time and avoid interference? Time division is included as a baseline. The four-orders-of-magnitude MBAF versus GBAF-TD gap on the multiple-access channel (Do not Interfere but Cooperate) and Suh and Tse's unbounded feedback gain at $\alpha=1/2$ (Feedback Capacity of the Gaussian Interference Channel to within 2 Bits) suggest it will lose at moderate interference. We will report where, if anywhere, it does not.
Proposed Experiments
The first decisive check is monotonicity. Across overhearing SNRs, the unconstrained learned cognitive encoder's order-wise spectrum with respect to primary bits should shift toward low orders as $\rho$ falls, and its effective length should shrink. A flat spectrum across $\rho$ would contradict the core claim.
The second check tests the bound. Measured signal correlation between the two transmitters should lie below the spectrum bound. The ratio of measured correlation to the bound should rise in regimes where BLER approaches the genie curve. If correlation is far below the bound in good codes, cooperation runs through some other channel and the explanation is wrong.
The third check tests prediction. Spectrum-predicted windows should match swept optima across the grid of $L\in\{0,1,2,4,\infty\}$, three overhearing SNRs, and two interference levels. It should do so at a fraction of the sweep's training cost.
The fourth check tests robustness. With feedback SNR reduced from noiseless to noisy, we compare constrained and unconstrained codes. Our hypothesis is that bounded effective length degrades more gracefully, because it limits how many noisy symbols each output amplifies.
Metrics throughout are BLER and bit error rate at fixed transmit power and blocklength, measured transmitter correlation, and estimated spectra with bootstrap confidence intervals. Each BLER point uses at least $10^8$ blocks so that errors near $10^{-7}$ are resolvable.
References
- On the Sub-optimality of Single-Letter Coding over Networks
- On the sub-optimality of single-letter coding in multi-terminal communications
- On the Correlation between Boolean Functions of Sequences of Random Variables
- Bounds on the Effective-length of Optimal Codes for Interference Channel with Feedback
- On the Capacity of the Two-User Gaussian Causal Cognitive Interference Channel
- On the Gaussian Interference Channel with Half-Duplex Causal Cognition
- On the Capacity of Interference Channel With Causal and Noncausal Generalized Feedback at the Cognitive Transmitter
- Deepcode: Feedback Codes via Deep Learning
- Do not Interfere but Cooperate: A Fully Learnable Code Design for Multi-Access Channels with Feedback
- Feedback Capacity of the Gaussian Interference Channel to Within 2 Bits
- The Degrees of Freedom of the Interference Channel with a Cognitive Relay under Delayed Feedback