Wireless systems at millimeter-wave frequencies increasingly skip expensive channel measurement. Instead, they predict the radio channel, or the best beam, directly from where the user is. These location-to-channel predictors are usually trained on ray-tracing simulators, and nobody knows how much of their accuracy survives contact with real hardware. Acoustics has already met this problem and measured it. Sound localizers trained on simulated rooms drop from 89.2% to 52.4% accuracy on real recordings (RealMAN). Acoustics also has dense, real, position-labeled channel measurements of a kind that radio largely lacks. We propose using acoustics as a measurable stand-in for the radio problem to answer one question: is the simulation-to-reality gap of a location-to-channel map mostly a property of the transceiver hardware, which is shared across all locations, or of the environment, which varies with location?
The answer determines what a fix should look like. If most of the gap comes from the array's real response (gain and phase errors per element, directional sensitivity), then a small calibration term shared across every location can be fit from a few dozen real measurements. A location-to-channel model rebuilt around that term should close the gap far more cheaply than fine-tuning a monolithic network. We build such a factored model, test it on acoustic datasets where the true channel is known everywhere, and then transfer it to real 60 GHz vehicular beam data.
Core Claim
A location-to-channel predictor can be split into three parts. A simulator supplies the geometry of the dominant propagation paths. A learned array response is shared across all locations. A small, spatially smooth residual absorbs unmodeled scattering and material effects. Concretely, for user location $x$ and frequency $f$, the predicted channel across the receive array is
$$ \hat{h}(x,f) = \sum_{k=1}^{K} \alpha_k(x)\, g_k(x)\, a_\phi\big(\theta_k(x), f\big)\, e^{-j2\pi f \tau_k(x)} + r_\psi(x,f). $$
Here $\theta_k, \tau_k, g_k$ are the arrival direction, delay and gain of the $k$-th strongest simulated path. The vector $a_\phi(\theta,f)$ is the array's response to a plane wave from direction $\theta$; it is learned once and shared by all locations. The factor $\alpha_k(x)$ is a bounded, smooth correction to path gains that stands in for frequency-dependent absorption. The term $r_\psi$ is a low-rank diffuse residual field.
The testable claim is that $a_\phi$ alone recovers most of the real-world performance lost by a simulation-trained predictor, using tens rather than hundreds of real measurements. Writing delays as phase shifts in the frequency domain follows Acoustic Volume Rendering (AVR). AVR showed that time-domain coordinate networks produce non-physical phase, and that moving to the frequency domain cut phase error on the real MeshRIR benchmark from 1.61 to 0.85. We reuse that technique.
Relation to Prior Work
Three groups of papers motivate the hardware hypothesis. Realistic sources, receivers and walls added measured transducer directivity and frequency-dependent walls to an image-source simulator, which models reflections as mirror-image sources. This cut real-world room-geometry estimation error by over 70%. The same paper also found that overly specific microphone models hurt transfer to unseen hardware, a constraint we respect by learning $a_\phi$ per device rather than assuming a database pattern. VAST showed the complementary environment story: adding non-specular scattering to synthetic training improved real azimuth error from 12.2° to 4.5°. RealMAN quantifies the overall collapse but does not attribute it to a cause. None of these papers separates the hardware contribution from the environment contribution for a location-conditioned predictor, which is what we do.
On the radio side, Kilinc et al. train location-based beam selection entirely in Sionna ray tracing and explicitly leave real-world validation open. Mollah et al. train a GPS-to-beam classifier directly on real DeepSense6G data, avoiding simulation but requiring full real supervision. Channel charting learns pseudo-positions from real channel measurements without labels, but the result has no metric scale and no link to a simulator. Our model sits between these: simulation provides structure, and a small amount of real data provides calibration.
Two neural-field papers come closest in mechanism and need a direct contrast. AVR and INFER both map position to a complex frequency response, and AVR even builds its simulator on Sionna. Both fit each scene from dense real measurements and do not separate hardware from environment. As a result, nothing they learn transfers when the room changes but the device does not, or the reverse. Our contribution is that separation and the sample-efficiency it is supposed to buy, not a better field renderer.
Research Program
The work has three stages.
First, we build an acoustic location-to-channel task that mirrors radio beamforming. The quantity predicted from source location is the relative transfer function, meaning the per-microphone channel normalized to a reference microphone. The predicted channel then defines a beamformer with no pilot and no speech-based estimation. We score it by the output power the beamformer achieves relative to an oracle beamformer built from the measured channel. This is the acoustic counterpart of the received-power ratio used on DeepSense6G.
Second, we pretrain on simulation and adapt with $N$ real measurements, sweeping $N$ over 0, 10, 30, 100 and 300. We compare the factored model against monolithic alternatives and against ablations that drop one term at a time. For each method we report the fraction of the sim-to-real gap closed, defined as (adapted score − simulation-only score) / (real-trained score − simulation-only score).
Third, we carry the factored model to DeepSense6G. There, the shared term becomes the realized response of the 64-beam phased-array codebook, and the predicted beam power is $|w_b^H \hat{h}(x)|^2$ for codebook beam $w_b$. Simulated path geometry comes from a coarse Sionna reconstruction of the street scene. If that reconstruction proves too inaccurate, we fall back to pretraining on Kilinc-style synthetic street canyons; this is the main engineering risk.
Baselines
The naive reference is a predictor trained purely in simulation and deployed unchanged. In acoustics it is an image-source shoebox simulator with ideal omnidirectional microphones; in radio it is a Sionna-trained location-to-beam regressor in the style of Kilinc et al. Every other method is measured by how much of its loss to the real-trained ceiling it recovers.
The ceiling is a model trained only on real data with the full real training split, as in RealMAN's real-to-real setting and Mollah et al.'s 1D convolutional classifier. On DeepSense6G that classifier reports an 84.58% average Top-1 received power ratio (Mollah et al.). The factored model beats this ceiling only if it matches or exceeds 84.58% using a small fraction of the real labels. We will report the label fraction at which it crosses 95% of that ceiling's performance.
The strongest competitor is monolithic fine-tuning: the same simulation-pretrained network with all weights updated on the $N$ real samples. It gets an identical architecture budget, identical real samples, and an equal hyperparameter search, chosen on a held-out real validation split of the same size for every method. We beat it if the factored model closes at least twice the gap at $N \le 30$ with overlapping or better results at $N=300$.
The second strong competitor is realistic-simulator training in the style of Srivastava et al. It injects measured directivity and frequency-dependent walls before training, using dEchorate's known geometry. It uses no real channel labels, so the fair comparison is against the factored model at $N=0$ and at small $N$.
The third competitor is randomized-simulator training in the style of VAST, which augments simulations with scattering and transducer variability.
The key ablations keep only $a_\phi$, keep only $r_\psi$, and replace the learned $a_\phi$ with the ideal analytic array response. Together they isolate whether the hardware term carries the effect.
Why not just…?
Why not just fine-tune the simulation-trained network on whatever real data exists? This is the most serious alternative. Acoustic modeling for Google Home got up to 28% relative error reduction from real-data fine-tuning, and multi-condition training studies found the gap persists until real data is added. The proposal does not dispute that real data is needed. Its claim is about how efficiently that data is used. A monolithic network must relearn the array response at each location it sees, whereas the factored model learns it once. If the sample-efficiency curves overlap at small $N$, the factorization adds nothing, and we will report that as the finding.
Why not just make the simulator realistic, as Srivastava et al. did? Their over-70% gain shows this works when good measured patterns of the actual hardware exist. Their own overfitting result shows it fails when the patterns come from different hardware. Radio arrays in the field drift from their datasheet patterns, and vehicle mounting changes them further. Learning $a_\phi$ from a few in-situ samples addresses exactly that case. The two approaches are also compatible, so we will test realistic-simulator pretraining followed by factored adaptation as a combined condition.
Why not just randomize the simulator heavily, as VAST did? Randomization buys robustness to environment variation and gave a large real-world gain (12.2° to 4.5°). It does not tell a deployed system what its own hardware looks like, and averaging over random array errors should blur fine phase structure that beamforming gain depends on. If randomized training already reaches the real-trained ceiling, the hardware hypothesis is wrong and the experiments below will show it.
Proposed Experiments and Decisive Checks
On dEchorate, which provides precise geometry and positions plus wall panels that are physically swapped between sessions, we fit $a_\phi$ in one wall configuration and freeze it. We then adapt only $\alpha$ and $r_\psi$ in the others. The hardware hypothesis predicts that a frozen $a_\phi$ transfers across wall configurations with little loss, while a frozen residual does not. If the reverse holds, the gap lives mostly in the environment and the core claim fails.
On RealMAN, whose 32-channel array is fixed while sources move across 32 scenes, we fit $a_\phi$ in a few scenes and test zero-shot in the held-out scenes against crude shoebox simulations. On MeshRIR's dense real grid we report AVR-comparable phase error, so the field component can be checked against the 0.85 AVR reports. No reported result was found on Lacuna for location-predicted transfer-function beamforming gain on any of these datasets, so our simulation-only and real-trained references will define that scale.
On DeepSense6G Scenarios 36–39 we report Top-1 received-power ratio against the 84.58% reference. We also inject GPS noise, since both Mollah et al. and Kilinc et al. leave sensitivity to positioning error uncharacterized; the shared array term should be unaffected by it, while the location-dependent terms should degrade.
The single decisive plot is fraction of gap closed versus $N$ for the factored model, monolithic fine-tuning, and the $a_\phi$-only ablation. The proposal succeeds if the $a_\phi$-only curve rises steeply and accounts for most of the full model's gain at small $N$ in both acoustics and radio.
References
- RealMAN: A Real-Recorded and Annotated Microphone Array Dataset for Dynamic Speech Enhancement and Localization
- Acoustic Volume Rendering for Neural Impulse Response Fields
- Realistic Sources, Receivers and Walls Improve The Generalisability of Virtually-Supervised Blind Acoustic Parameter Estimators
- VAST: The Virtual Acoustic Space Traveler Dataset
- Beam Training in mmWave Vehicular Systems: Machine Learning for Decoupling Beam Selection
- Position Aware 60 GHz mmWave Beamforming for V2V Communications Utilizing Deep Learning
- Wireless Channel Charting: Theory, Practice, and Applications
- INFER: Learning Implicit Neural Frequency Response Fields for Confined Acoustic Environments
- Acoustic Modeling for Google Home.
- Effectiveness of dereverberation, feature transformation, discriminative training methods, and system combination approach for various reverberant environments