Spatial audio sounds convincing only when it is filtered through the listener's own head-related transfer function (HRTF). An HRTF is the direction-dependent filter that a person's head, ears and torso apply to incoming sound. Measuring one takes an anechoic chamber and hundreds of loudspeaker directions, so public HRTF collections hold roughly 45 to 200 people. Machine-learned personalization is therefore starved of examples. The Human Connectome Project (HCP), meanwhile, holds high-resolution structural MRI of about a thousand adult heads, scanned with one protocol and processed with one pipeline. Nobody has used these scans as acoustic objects.
We propose to turn HCP structural scans into a large population of physically simulated HRTFs and use them as a pretraining prior for personalization from very few measurements. The design is deliberately limited to what MRI can support. A 0.7 mm scan pins down head size and shape well. It probably does not resolve pinna folds reliably, and in-scanner padding and earphones may deform the ears. We therefore simulate only the head-dominated, low-frequency part of the HRTF: the timing and level differences between the ears, below roughly 5 kHz. Within that band, wavelengths are long relative to the MRI voxel size, and fast acoustic solvers still work well. The central question is concrete. If a model is pretrained on about a thousand simulated MRI heads, does it predict a new listener's interaural time difference better from only three measured directions than the best published method trained on measured data alone?
Core Claim
Existing HRTF personalization methods learn their prior over human heads from a small number of measured subjects. Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization (RANF) is the clearest case. It retrieves acoustically similar people from the 200-subject SONICOM set and adapts a neural field to them, a neural field being a network that maps a direction to a filter value. With 3 measurements it reaches an ITD error of 20.5 μs. We claim that the low-frequency part of this prior is limited by how few heads it has seen, not by the model. A prior built from roughly a thousand anatomically real but acoustically simulated heads should lower sparse-measurement ITD error and low-band spectral error. It should leave the high-frequency pinna band unchanged, and that band remains the job of measured data.
The claim is falsifiable in both directions. If the simulated prior helps only as much as randomly scaled ellipsoid heads do, real anatomy adds nothing. If it also helps above 8 kHz, our assumption that MRI cannot capture the pinna was too pessimistic, and the frequency cutoff should be pushed up.
Relation to Prior Work
Three lines of work supply the method.
The first is the solver. Lightning-fast Boundary Element Method speeds up the boundary element method, which computes how sound scatters off a surface mesh. Its preconditioner reduces iteration counts to 7–13 on million-unknown problems. The same paper constrains our design: at high frequencies the "screening" effect behind the speed-up weakens. We read this as a reason to stay in the low band rather than as an obstacle to work around.
The second is the representation. HRTF Field shows that one neural field conditioned on a per-subject latent vector can absorb ten measured databases with different sampling grids, and that it transfers nearly as well to an unseen eleventh. We add a simulated corpus as one more "database" within that same setup.
The third is the adaptation recipe: RANF's retrieval plus low-rank adaptation, and Parameter-Transfer Learning for Low-Resource Individualization of Head-Related Transfer Functions, which pretrains on the larger PKU&IOA set and fine-tunes the last layer for each listener. Both pretrain only on measurements. Our proposal changes where the pretraining data come from and leaves the adaptation method alone.
The motivation comes from the data-scarcity problem that every personalization paper reports. Sparsity-Constrained Weight Mapping for Head-Related Transfer Functions Individualization from Anthropometric Features has to synthesize 4,960 training pairs from 30 CIPIC subjects by leave-k-out recombination. Its inputs are caliper measurements, not 3D anatomy.
On the HCP side, earlier work treats the scans purely as brain data: parcellation in Geometric Deep Learning of the Human Connectome Project Multimodal Cortical Parcellation, and audiovisual decoding in SIM and A Shared Neural Encoding Model for the Prediction of Subject-Specific fMRI Response.
We deliberately do not pursue the more obvious idea of adding personalized HRTFs to HCP movie-fMRI encoding models. Those participants heard the soundtrack through headphones, so their own pinnae never filtered it. Personalized HRTF features would describe a stimulus they did not actually receive.
Research Program
The first stage is an audit, and it can end the project early. Public HCP structural volumes are processed to protect identity. Before any simulation, we will measure in what fraction of subjects the ears and the full scalp boundary survive. We will also check whether the field of view clips the lower head and whether ear padding visibly flattens the pinnae. The low-band design only needs an intact head surface. Clipped or deformed pinnae are tolerable if we replace them with a fixed template ear, and that replacement becomes one of the ablations.
The second stage extracts a watertight scalp mesh per subject from the T1w/T2w volumes. Each mesh is decimated to the element size needed up to about 5 kHz, which is much coarser than the sub-millimetre meshes the pinna band would require. Because the scans contain no torso, we attach a generic torso and shoulders to every head. Torso-related variation is therefore absent from the prior by construction.
The third stage runs the boundary element solver on a dense spherical grid for each head. This produces complex-valued low-band HRTFs, including phase and therefore ITD. Most learned HRTF models discard phase. A simulator provides it directly.
The fourth stage pretrains an HRTF Field–style neural field. The training mixture combines the simulated corpus, with its loss restricted to the simulated band, and the pooled measured databases at full band. Personalization then follows the RANF and PTL recipes unchanged. Retrieval runs over the combined real and simulated pool, using RANF's ITD-based similarity.
Baselines
The naive reference point is a generic, non-individualized HRTF evaluated on the same held-out listeners. The file shows how much this costs perceptually. HRTF Estimation in the Wild reports front-back confusion of 29.0% with a generic HRTF versus 14.8% with an estimated personal one. Any method that fails to beat the generic HRTF on ITD error has failed outright.
The strongest existing competitor is RANF, run on its own SONICOM split (179 training subjects, 20 test subjects) with 3 and 100 measured directions. Its reported results are an ITD error of 20.5 μs at 3 measurements, against 28.6 μs for a plain neural field, and an LSD of 3.0 dB at 100 measurements, against 3.8 dB. LSD is log-spectral distortion, the average decibel gap between predicted and true spectra. RANF is the right target because it is built for the extreme-sparsity regime where a prior matters most, and because ITD is its own retrieval signal. To keep the comparison fair, we rerun RANF with its published recipe and our extended pool, using the same test listeners, the same measurement directions, an equal hyperparameter-search budget, and matched total pretraining steps. We count it as beaten if mean ITD error at 3 measurements drops reliably below the rerun RANF figure across seeds, ideally toward 15 μs, with no loss in LSD above 8 kHz.
The second competitor is multi-database pretraining on measured data alone, following HRTF Field, which pools ten datasets. This baseline asks whether simply using all available real data is enough. The file gives no reported number on Lacuna for its cross-dataset error, so we will establish one ourselves on a leave-one-database-out split. For CIPIC we also report PTL-L's published 4.95 dB LSD.
The third baseline is an analytic spherical-head ITD model, with radius fitted from the same three measured directions. Physics alone may explain most of what the simulation adds, so this is a necessary check. No reported result was found on Lacuna for it on SONICOM, so it will be measured here.
The key ablation keeps the entire pipeline identical but replaces the HCP heads with ellipsoids. These are randomly scaled to match the mean and spread of HCP head dimensions and carry a template ear. This isolates the one ingredient we claim matters: real anatomical shape variation beyond overall size. A second ablation varies the number of HCP heads (100, 300, 1,000) to test whether the benefit scales with population size, as the data-scarcity explanation predicts.
Why not just…?
Why not just pretrain on all ten measured databases? HRTF Field shows this transfers well. It is the most likely way for our proposal to turn out unnecessary. Even pooled, the measured databases cover a few hundred people with different apparatus. If the measured-only multi-database prior matches ours at 3 measurements, the simulated corpus adds nothing and we will report that. The head-count scaling curve tells the two explanations apart: a gain that grows with the number of HCP heads points to population coverage, while a flat curve points to redundancy with the measured data.
Why not just use a spherical-head formula for ITD? For ITD alone, head radius plus a known relation between geometry and timing may capture most of the individual variation, so this is a real threat. It cannot represent ear position, an asymmetric or elongated skull, or low-band ILD. The ellipsoid ablation and the spherical-head baseline together measure exactly how much error remains after size alone is accounted for. If that remainder is under a few microseconds, we will narrow the claim to ILD and low-band spectra.
Why not just simulate from synthetic or morphable head shapes instead of MRI? Parametric heads are cheap and need no data audit. Their variation, however, comes from whatever the designer built in, not from the population. The ellipsoid ablation is a minimal version of this alternative. Beating it shows that real shape variation matters. Failing to beat it means real anatomy is not worth the cost of using HCP scans.
Why not just simulate the pinna too and get the whole HRTF? The solver paper reports that its speed advantage fades at high frequencies. MRI resolution and in-scanner ear deformation also make the simulated pinna band suspect. We test this directly rather than assume it: one ablation extends the simulated loss to 12 kHz. If LSD above 8 kHz gets worse relative to the low-band-only prior, the cutoff is justified.
Proposed Experiments and Decisive Checks
The primary experiment compares ITD error and band-split LSD (below 4 kHz, 4–8 kHz, above 8 kHz) on the 20 held-out SONICOM listeners at 3, 10 and 100 measured directions. The methods compared are generic, RANF rerun, measured multi-database pretraining, spherical-head, ellipsoid ablation, and our HCP prior. Results are averaged over five seeds with paired tests across listeners. The 4–8 kHz band, which The role of spectral cues in vertical plane elevation perception identifies as the main cue for elevation, is reported separately. We expect no gain there, and a gain would change how we interpret the results.
A secondary transfer test uses CIPIC with PTL-L's protocol. Because CIPIC includes anthropometry, we can also check whether head widths in the simulated corpus span the CIPIC range. Simulation validity is checked without paired MRI and measured HRTFs from the same people. Instead, we compare the distribution of simulated ITD and low-band ILD across the thousand HCP heads with the measured distributions in SONICOM and CIPIC. A systematic offset would reveal a mesh or boundary-condition error before it contaminates the prior.
The decisive result needs three things together. The HCP prior must beat RANF's rerun ITD error at 3 measurements. It must beat the ellipsoid ablation by more than the spread across seeds. Its gain must grow with the number of heads. If all three hold, large neuroimaging archives become a practical new source of acoustic training data. If the ellipsoid ablation matches it, the honest conclusion is that head size, not MRI anatomy, was the missing information.
References
- Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization
- Lightning-fast Boundary Element Method
- HRTF Field: Unifying Measured HRTF Magnitude Representation with Neural Fields
- Parameter-Transfer Learning for Low-Resource Individualization of Head-Related Transfer Functions
- Sparsity-Constrained Weight Mapping for Head-Related Transfer Functions Individualization from Anthropometric Features
- Geometric Deep Learning of the Human Connectome Project Multimodal Cortical Parcellation
- SIM: Surface-based fMRI Analysis for Inter-Subject Multimodal Decoding from Movie-Watching Experiments
- A Shared Neural Encoding Model for the Prediction of Subject-Specific fMRI Response
- HRTF Estimation in the Wild
- The role of spectral cues in vertical plane elevation perception