HRTFformer: A Spatially-Aware Transformer for Personalized HRTF Upsampling in Immersive Audio Rendering

Xuyi Hu, Jian Li, Shaojie Zhang, Stefan Goetz, Lorenzo Picinali, Özgür B. Akan, Aidan O. T. HoggarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces HRTFformer, a novel transformer-based model for upsampling Head-Related Transfer Functions (HRTFs). By leveraging self-attention in the spherical harmonic domain, HRTFformer effectively captures long-range spatial correlations, leading to more accurate and perceptually realistic HRTF reconstruction, especially from sparse measurements.

What Problem Does This Paper Address?

Creating personalized HRTFs is crucial for realistic immersive audio experiences, as individual anatomy significantly shapes how sound is perceived. However, the process of measuring HRTFs is time-consuming and requires specialized equipment, making it impractical for widespread adoption. HRTF upsampling aims to solve this by reconstructing high-resolution HRTFs from a limited number of measurements. Existing methods often struggle to preserve local spatial variations and generalize well at high upsampling factors, leading to a disconnect between objective spectral distortion metrics and actual perceptual performance.

Methods

The core of HRTFformer is a transformer architecture adapted for HRTF upsampling. The model operates in the spherical harmonic (SH) domain, which provides a compact and physically meaningful representation of directional acoustic information.

  • Encoder-Decoder Architecture: The model employs an encoder-decoder structure. The encoder processes low-resolution SH coefficients, extracting hierarchical features and compressing them into a latent representation. The decoder then reconstructs high-resolution SH coefficients from this latent representation.
  • Transformer Blocks: Both the encoder and decoder utilize transformer blocks with self-attention mechanisms. This allows the model to capture global contextual relationships between SH coefficients, addressing the limitations of models that focus only on local features.
  • Rotary Position Embedding (RoPE): To effectively model the ordering of SH coefficients, RoPE is used. This method incorporates relative positional information by rotating query and key vectors, offering better generalization to variable input resolutions compared to absolute position encodings.
  • Neighbour Dissimilarity Loss (NDL): A novel loss function is introduced to enforce spatial continuity in the HRTF magnitude spectrum. This loss encourages smooth variations across neighboring source directions, promoting more perceptually coherent and realistic upsampling.
  • Token Scaling: Instead of traditional layer normalization, token scaling is used. This approach preserves the relative energy distribution across frequency bins, which is critical for magnitude-sensitive SH coefficients.

Datasets

The paper evaluates HRTFformer on the SONICOM HRTF dataset. Experiments were conducted under four sparsity levels: 3, 5, 19, and 100 initial sampling points.

Key Insights and Ideas

  • Transformers for Spatial Correlations: The paper demonstrates that transformer architectures, with their self-attention mechanism, are highly effective at capturing long-range spatial dependencies in the SH domain, which is crucial for accurate HRTF upsampling.
  • The Importance of Perceptual Coherence: A key insight is that minimizing spectral distortion (like LSD) doesn't always translate to better perceptual performance. The introduction of the neighbour dissimilarity loss directly addresses this by promoting spatial smoothness, leading to more perceptually realistic results.
  • RoPE for Positional Awareness: Using rotary position embeddings proves more effective than standard position encodings for capturing the inherent ordering and relationships within SH coefficients, leading to better generalization.
  • Token Scaling's Advantage: The empirical finding that token scaling outperforms layer normalization highlights its benefit in preserving the physical characteristics of SH coefficients, which are sensitive to magnitude.

Key Results

HRTFformer consistently outperforms existing state-of-the-art methods, including both traditional interpolation and deep learning approaches, across various metrics:

  • Spatial Cue Evaluation (Table I): Under the most challenging sparsity levels (3 and 5), HRTFformer achieves significantly lower ITD, ILD, and LSD errors. For instance, at sparsity level 3, it achieves an LSD of 4.20 dB, compared to the next best method (IOA3D) at 4.67 dB.
  • Perceptual Evaluation (Tables III & IV): HRTFformer demonstrates superior performance in perceptual metrics like Polar Accuracy Error, Polar RMS Error, and Quadrant Error, especially at low sparsity levels. At sparsity level 3, it achieves a Polar Accuracy Error of 3.83 degrees and a Polar RMS Error of 35.37 degrees, significantly outperforming baselines.
  • Robustness: The model shows robust reconstruction performance across different sparsity levels.
  • Visualizations (Fig. 3): LSD distributions show that HRTFformer maintains significantly lower errors, particularly at high sparsity levels, where other methods exhibit noticeable increases in reconstruction errors.

Limitations and Future Work

  • Dataset Specificity: The current work is limited to experiments conducted on a single HRTF dataset (SONICOM). Future research needs to address generalizability across datasets acquired with different measurement protocols.
  • Subjective Evaluation: While the paper uses a Bayesian auditory localization model for perceptual evaluation, the authors acknowledge the need for subjective evaluations with real human listeners to confirm spatial realism, externalization, and individual localization.
  • Data Diversity: The limited diversity of HRTF datasets can impact generalization. Future work could explore using synthetic HRTFs and transfer learning to improve generalization to unseen subjects and conditions.
  • Computational Efficiency: HRTFformer has a larger model size and inference time compared to some GAN-based baselines. While not a primary concern for offline use, optimizing efficiency could be a future direction.

Summary updated

BibTeX
@misc{hu2025hrtfformer,
  author = {Xuyi Hu and Jian Li and Shaojie Zhang and Stefan Goetz and Lorenzo Picinali and Özgür B. Akan and Aidan O. T. Hogg},
  title = {{HRTFformer: A Spatially-Aware Transformer for Personalized HRTF Upsampling in Immersive Audio Rendering}},
  year = {2025},
  eprint = {2510.01891},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01891},
}
Figure