High Fidelity Generative Audio at Professional Sampling Rates

This direction is split into other directions: high-fidelity audio synthesis. Its last version stays readable and citable here.

Version 1 of 1203 papers226 conceptsfrozen 2026-10-04

For years, neural audio research has been confined by a "16 kHz ceiling." While 16,000 samples per second is sufficient for intelligible speech, it lacks the spectral brilliance and harmonic detail required for professional music production, cinema, or broadcast. Professional standards demand 44.1 kHz (CD quality) or 48 kHz (studio quality) to capture the full human hearing range.

Bridging this gap is not simply a matter of upsampling; it requires architectures that can model the complex high-frequency transients and phase relationships that define high-fidelity sound. Recent research has shifted toward generative frameworks—specifically diffusion models, discrete tokenizers, and hybrid DSP systems—to overcome the computational and structural barriers of full-band audio synthesis.

The Resolution Bottleneck

The primary challenge in high-fidelity synthesis is the sheer volume of data. At 48 kHz, a single second of stereo audio contains 96,000 data points. Traditional autoregressive models struggle with these long-range dependencies, often resulting in audible glitches or "scars."

To address this, researchers like Prem Seetharaman and Ethan Manilow developed Make it Sound Good (MSG) MSG, a generative post-processor designed to "heal" the artifacts left by lower-resolution models. Rather than generating high-res audio from scratch, MSG uses a GAN-based architecture to restore missing harmonics and transients, effectively acting as a neural "varnish" for separated music stems.

Latent Diffusion and Full-Band Synthesis

Diffusion models have emerged as a dominant solution for high-fidelity generation because they model the underlying distribution of audio rather than predicting individual samples. Haohe Liu and colleagues introduced AudioLDM AudioLDM, which operates in a compressed latent space. By training on continuous latent representations instead of raw waveforms, the model achieves high-quality generation and zero-shot manipulation on consumer-grade hardware.

For general audio synthesis, Santiago Pascual, Jordi Pons, and the team at Dolby Laboratories developed DAG (Diffusion Audio Generator) DAG. DAG moves away from autoregressive constraints to achieve full-band (48 kHz) synthesis. The intuition here is that score-based diffusion can better capture the high-frequency information necessary for realistic soundscapes, which lower-bandwidth models typically muffle.

Scaling via Discrete Audio Tokens

Another breakthrough involves treating audio like text through discrete tokenization. By compressing high-resolution audio into a sequence of "tokens," researchers can leverage the power of Large Language Models (LLMs) for audio tasks.

Luca A. Lanzendörfer and Roger Wattenhofer demonstrated this with DAC-SE1 DAC-SE1, a 1-billion parameter model that performs speech enhancement at 44.1 kHz. Similarly, Shengpeng Ji and Ziyue Jiang developed WavTokenizer WavTokenizer, which achieves extreme compression while maintaining the semantic richness needed for 24kHz to 48kHz reconstruction. These models work by learning a "codebook" of acoustic patterns, allowing the system to reconstruct high-fidelity audio from a very small number of tokens per second.

Hybrid Neural-DSP Architectures

A third direction involves injecting domain knowledge from Digital Signal Processing (DSP) into neural networks. Purely neural vocoders often struggle with the "text-to-phase" mapping problem, leading to metallic artifacts.

Yongmao Zhang and Lei Xie addressed this in VISinger2 VISinger2 by integrating a differentiable DSP synthesizer. By forcing the model to generate audio through a harmonic-plus-noise framework, the system ensures that the output adheres to the physical properties of human singing. This hybrid approach allows for 44.1 kHz synthesis with 30% fewer parameters than traditional models.

Similarly, ESTVocoder ESTVocoder by Xiao-Hang Jiang provides the network with a "head start" by using a pitch-based excitation signal, significantly improving training convergence and output clarity at high sampling rates.

Professional Applications and Control

The push for higher fidelity is driven by the need for practical tools in professional workflows.

  • Foley and Sound Design: Models like MultiFoley MultiFoley and Tri-Ergon Tri-Ergon generate 44.1/48 kHz stereo audio synchronized to video, using parameters like LUFS (Loudness Units relative to Full Scale) to give sound editors professional-grade control.
  • Music Production: Systems like DrumGAN DrumGAN and StyleWaveGAN StyleWaveGAN allow musicians to sculpt percussion sounds using intuitive features like "brightness" and "hardness" at CD-quality resolution.

By moving beyond the 16 kHz research standard, these models are transitioning from academic curiosities to viable tools for the creative industries, where acoustic brilliance and high-frequency detail are non-negotiable.

Papers in this version

Showing 60 of 203 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_high_fidelity_generative_audio_at_profes_40704cfe,
  title = {High Fidelity Generative Audio at Professional Sampling Rates},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-fidelity-generative-audio-at-professional-sampling-rates/rdr_40704cfe8b4d492ea0078d5a90bcbf86},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_high_fidelity_generative_audio_at_profes_40704cfe_v1,
  title = {High Fidelity Generative Audio at Professional Sampling Rates},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-fidelity-generative-audio-at-professional-sampling-rates/rdr_40704cfe8b4d492ea0078d5a90bcbf86/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}