Time-Graph Frequency Representation with Singular Value Decomposition for Neural Speech Enhancement

Tingting Wang, Tianrui Wang, Meng Ge, Qiquan Zhang, Zirui Ge, Zhen YangICASSP 2025 · DOI

TL;DR

This paper introduces a novel "Time-Graph Frequency Representation" (GFT-SVD) that uses Singular Value Decomposition to create a real-valued representation of speech signals. This approach effectively aligns amplitude and phase information, overcoming a key limitation in existing neural speech enhancement methods and leading to improved speech quality.

What Problem Does This Paper Address?

Neural speech enhancement aims to clean up noisy audio recordings. Many successful methods rely on time-frequency (T-F) representations like the Short-Time Fourier Transform (STFT). However, STFT produces complex-valued representations, meaning amplitude and phase are intertwined. To handle this, researchers often use "decoupling" strategies, where separate neural network streams predict amplitude and phase. The core problem is that these separate streams struggle to perfectly align their predictions, leading to "phase wrapping" issues and degraded speech quality. This paper tackles the challenge of achieving better amplitude-phase alignment in neural speech enhancement.

Methods

The paper proposes a new representation called the "Graph Fourier Transform with Singular Value Decomposition" (GFT-SVD). Instead of relying on the traditional STFT or even a standard Graph Fourier Transform (GFT) which still produces complex values, GFT-SVD uses Singular Value Decomposition (SVD) on a graph representation of the speech signal. This process transforms the raw waveform into a real-valued time-graph representation.

Here's a breakdown:

  • Graph Representation: Speech signals are modeled as graph signals, where each sample is a node and dependencies between samples are represented by edges. This is captured by an adjacency matrix.
  • Singular Value Decomposition (SVD): The adjacency matrix is decomposed using SVD. The resulting unitary matrices are used to define a new graph Fourier basis.
  • Real-Valued Transformation: This new basis allows the noisy speech signal to be transformed into a real-valued spectrogram, effectively encoding both amplitude and phase information in a single, aligned representation.
  • Neural Network Architecture: The GFT-SVD representation is used as input to a standard neural network (the "mask estimator"). The network predicts a mask, which is then applied to the noisy signal. Finally, an inverse GFT-SVD (iGFT-SVD) reconstructs the enhanced speech. This pipeline avoids the need for dual-stream networks.

Datasets

The paper uses two datasets for evaluation:

  • DNS-2020: This is a large dataset from the 2020 Deep Noise Suppression Challenge, sampled at 16 kHz. The authors generated 100 hours of noisy data with Signal-to-Noise Ratios (SNRs) ranging from -5dB to 20dB.
  • VCTK+DEMAND: This is a simulated dataset. Clean speech is taken from the Voice Bank corpus, and noisy speech is generated by mixing it with noise from the Diverse Environments Multichannel Acoustic Noise Database (DEMAND).

Key Insights and Ideas

  • Unified Amplitude and Phase: The GFT-SVD representation inherently aligns amplitude and phase information by producing a single real-valued output. This bypasses the need for complex decoupling strategies in neural networks.
  • Improved Masking: The real-valued, unsymmetrical nature of the GFT-SVD spectra (compared to the symmetrical real/imaginary parts of STFT) is found to be more conducive to mask training, leading to better performance.
  • Graph Structure Matters: The choice of the "k" value (representing the number of neighbors considered in the graph structure) impacts performance. The paper suggests that k=3 or k=5 yields optimal results, indicating that capturing local dependencies is crucial.
  • Computational Efficiency: The GFT-SVD approach, despite introducing a new representation, shows comparable or even slightly better computational efficiency (MACs, parameters, and Real-Time Factor) compared to STFT-based baselines.

Key Results

The paper demonstrates significant improvements across various metrics:

  • Objective Metrics: Models using GFT-SVD consistently outperform their STFT-based counterparts and GFT-EVD (Graph Fourier Transform with Eigenvector Decomposition) based methods. For instance, DPCRNG (DPCRN with GFT-SVD) achieved over 1dB gains in SI-SDR compared to DPCRN with STFT on the DNS-2020 dataset.
  • Subjective Metrics: Improvements are also observed in perceptual metrics like PESQ and STOI, indicating better speech quality.
  • Generalizability: The GFT-SVD approach shows consistent benefits when applied to various backbone neural network architectures (NSnet2, DPCRN, DCRN, DCCRN, MTFAA, UNet, G-UNet).

Limitations and Future Work

  • Graph Structure Optimization: While the paper explores different 'k' values, further research could investigate more sophisticated graph construction methods beyond simple k-nearest neighbors.
  • Real-World Complexity: The experiments are primarily conducted on simulated or benchmark datasets. Evaluating the GFT-SVD approach on more diverse and challenging real-world noisy conditions could be a valuable next step.
  • Further Exploration of GFT-SVD: The authors suggest that the GFT-SVD framework could be extended to other signal processing tasks beyond speech enhancement.

Summary updated

BibTeX
@inproceedings{wang2025time,
  author = {Tingting Wang and Tianrui Wang and Meng Ge and Qiquan Zhang and Zirui Ge and Zhen Yang},
  title = {{Time-Graph Frequency Representation with Singular Value Decomposition for Neural Speech Enhancement}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10888429},
}
Figure