Abstract
State-of-the-art signal decomposition often relies on low-rank factorization of time-frequency (t-f) transforms, most notably Nonnegative Matrix Factorization (NMF) of spectrograms. However, NMF is primarily an analysis-based approach that discards phase information. This paper proposes Low-Rank Time-Frequency Synthesis (LRTFS), a generative modeling paradigm that imposes low-rankness directly on the synthesis coefficients of a signal. By bridging t-f synthesis modeling with traditional NMF, LRTFS enables more robust signal recovery, handles phase implicitly, and excels in applications like denoising and compressive sensing.
Introduction
In audio signal processing, NMF has become a standard tool for source separation and feature extraction. Despite its success, standard NMF operates on the squared magnitude of the Short-Time Fourier Transform (STFT), treating it as an "analysis" step. This approach ignores the underlying phase of the signal, which must be heuristically reconstructed. Conversely, traditional "synthesis" models, such as sparse coding, focus on the raw time-domain signal but typically rely on sparsity priors. While sparsity is useful, it fails to capture the inherent low-rank structure of audio—where specific spectral patterns repeat over time.
LRTFS addresses these gaps by applying a low-rank prior to the synthesis coefficients within a generative framework. This allows for a more principled treatment of the signal's structure and phase, facilitating superior performance in inverse problems.
Methodology
The LRTFS framework models a signal $x$ as: $$x = \Phi\alpha + e$$ where $\Phi$ is a time-frequency dictionary (e.g., a Gabor frame), $\alpha$ are the synthesis coefficients, and $e$ represents additive white noise.
Low-Rank Prior
Unlike sparse models, LRTFS assumes that the variance of the coefficients $\alpha$ follows a low-rank structure: $$|\alpha_{fn}|^2 \approx [WH]_{fn}$$ Here, $W$ represents the spectral templates (dictionary of patterns) and $H$ represents the temporal activations, mirroring the structure of NMF.
Estimation and Algorithms
The authors employ Maximum Joint Likelihood Estimation (MJLE) to simultaneously estimate the coefficients $\alpha$ and the NMF factors $W$ and $H$. The optimization is handled via:
- Majorization-Minimization (MM): Used for updating $W$ and $H$ based on Itakura-Saito NMF principles.
- Iterative Shrinkage Algorithm (ISA): A fast, proximal gradient method used to update the synthesis coefficients $\alpha$.
This generative approach can be extended to multi-layer models, allowing the separation of distinct signal components, such as "tonal" parts (captured by low-rank structures) and "transient" parts (captured by sparse structures).
Experimental Results
The model was evaluated on audio datasets, including a piano recording and a musical excerpt ("Mamavatu").
Signal Decomposition and Denoising
As shown in Figure 1, LRTFS demonstrates a significant advantage over standard IS-NMF in decomposing a noisy piano signal. While IS-NMF often splits single notes into multiple components and fails to isolate transient "hammer attacks," LRTFS successfully recovers clean, distinct components. Because LRTFS models the raw signal, it maintains phase consistency, leading to more natural-sounding reconstructions.
Compressive Sensing (CS)
A key finding is the application of low-rankness to Compressive Sensing. Traditional CS relies on signal sparsity, but the authors show that the repetitive spectral structure of audio makes low-rankness a more powerful prior. In tests recovering signals from as little as 1% to 10% of original samples, LRTFS outperformed sparsity-based methods (like $\ell_1$ regularization) by several decibels in output SNR.
Discussion and Conclusion
LRTFS provides a robust generative alternative to analysis-based NMF. By modeling synthesis coefficients with a low-rank prior, it bridges the gap between structured matrix factorization and raw signal processing.
Key Advantages: * Implicit Phase Handling: Operates on the time-domain signal, avoiding the "phase problem" of magnitude-only NMF. * Structural Flexibility: Multi-layer extensions allow for the simultaneous modeling of tonal and transient components. * Superior CS Recovery: Demonstrates that low-rankness is a more effective prior than sparsity for musical audio.
Future work will focus on reducing the computational complexity of the joint estimation and incorporating temporal correlations into the variance model to better capture the dynamic evolution of sound.
Summary updated