Learning Neural Vocoder from Range-Null Space Decomposition

Andong Li, Tong Lei, Zhihang Sun, Rilin Chen, Erwei Yin, Xiaodong Li, Chengshi ZhengIJCAI 2025 · DOI

TL;DR

This paper introduces RNDVOC, a novel neural vocoder that decomposes the speech synthesis problem into "range-space" and "null-space" components. This approach leverages classical signal processing theory to achieve state-of-the-art speech quality with significantly fewer parameters and faster inference speeds compared to existing methods.

What Problem Does This Paper Address?

Neural vocoders are crucial for generating realistic speech from acoustic features, powering applications like text-to-speech (TTS) and speech enhancement. While recent neural vocoders have achieved impressive naturalness, they often face a trade-off between model size (and thus computational cost) and performance. Large models like BigVGAN, despite their quality, are computationally expensive and impractical for resource-constrained environments. Furthermore, existing time-frequency (T-F) domain vocoders, while faster, often lag behind their time-domain counterparts in quality, partly because they don't fully exploit the inherent structure of speech signals. This paper aims to bridge this gap by developing a more efficient and high-quality neural vocoder that addresses both parameter efficiency and performance.

Methods

The core innovation of RNDVOC lies in its application of Range-Null Space Decomposition (RND) theory to the neural vocoder task. The authors reframe the problem of reconstructing a target spectrogram from a mel-spectrogram. They observe that the mel-spectrogram is a linearly compressed representation of the magnitude spectrum, with phase information discarded. RND theory allows them to decompose the reconstruction into two parts:

  1. Range-Space Module (RSM): This module leverages the linear degradation of the mel-spectrogram. It uses a pseudo-inverse operation to transform the mel-spectrogram into the corresponding linear-scale domain, effectively recovering the "range-space" component of the original signal. This part is designed to preserve the fundamental acoustic information.
  2. Null-Space Module (NSM): This module is responsible for reconstructing the "null-space" component, which includes the spectral details and phase information that were lost or compressed in the mel-spectrogram. This is achieved through a learnable network.

The RNDVOC architecture features a dual-path framework for hierarchical encoding and decoding. It employs:

  • Hierarchical Spectral Encoding Module (HSEM): Splits the spectral information into multiple regions and processes them hierarchically.
  • Dual-Path Blocks (DPB): These blocks consist of a Cross-Band Module (CBM) for modeling correlations between neighboring spectral sub-bands and a Narrow-Band Module (NBM) for capturing temporal dependencies using ConvNeXt v2 blocks.
  • Hierarchical Magnitude/Phase Decoding Modules (HMDM/HPDM): Reconstruct the magnitude and phase spectra.

A novel omnidirectional phase loss is also introduced to better capture phase relationships across T-F bins, improving spectral detail reconstruction.

Datasets

The paper uses two standard datasets for training and evaluation:

  • LJSpeech: A single-speaker dataset for objective and subjective evaluations.
  • LibriTTS: A multi-speaker dataset used for comprehensive objective and subjective comparisons.

Key Insights and Ideas

  • RND Theory for Vocoders: Applying classical signal decomposition theory (RND) to neural vocoders provides a more interpretable and structured approach to speech synthesis, moving beyond black-box modeling.
  • Decoupling Range and Null Spaces: Separating the reconstruction into a deterministic range-space component (leveraging the linear mel-spectrogram degradation) and a learnable null-space component allows for more efficient parameter usage and better performance.
  • Hierarchical and Cross-Band Modeling: Processing spectral information hierarchically and using cross-band modules to model relationships between sub-bands is crucial for capturing the complex structure of speech.
  • Lightweight Design: The RND framework inherently supports creating very lightweight models (down to 0.08M parameters) without a drastic drop in quality, making it suitable for edge devices.
  • Omnidirectional Phase Loss: A more effective phase loss function significantly contributes to better spectral detail reconstruction.

Key Results

  • State-of-the-Art Performance: RNDVOC achieves comparable or better objective and subjective performance than leading vocoders like BigVGAN, even when using significantly fewer parameters.
  • Parameter Efficiency: The proposed method achieves this with drastically reduced model sizes. For instance, a version with only 2.8% of BigVGAN's parameters (3.14M vs. 112M) yields competitive results.
  • Inference Speed: RNDVOC offers a significant speed-up, with nearly 10x faster inference on a CPU compared to BigVGAN.
  • Lightweight Models: RNDVOC-Lite and RNDVOC-UltraLite, with as few as 0.08M parameters, outperform HiFiGAN-V2 in PESQ and VISQOL metrics, demonstrating the effectiveness of the RND approach for extreme lightweight scenarios.
  • Subjective Quality: MUSHRA tests on LibriTTS show RNDVOC is statistically superior to BigVGAN in subjective speech quality.

Limitations and Future Work

  • Limitations: While the paper highlights impressive results, the authors acknowledge that the current framework is primarily designed for single-speaker scenarios. The "fixed" nature of the A matrix in the range-space decomposition, while providing interpretability, might limit its adaptability to highly diverse acoustic conditions or speaker characteristics compared to fully learnable approaches.
  • Future Work: The authors suggest exploring more advanced techniques for the null-space modeling and investigating multi-speaker extensions of the RNDVOC framework. Further research into optimizing the range-space transformation for even greater adaptability could also be a promising direction.

Summary updated

BibTeX
@inproceedings{li2025learning,
  author = {Andong Li and Tong Lei and Zhihang Sun and Rilin Chen and Erwei Yin and Xiaodong Li and Chengshi Zheng},
  title = {{Learning Neural Vocoder from Range-Null Space Decomposition}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/904},
}
Figure