Joint pitch-analysis formant-synthesis framework for CS recovery of speech

Srikanth Raj Chetupally, Thippur V. SreenivasInterspeech 2012

TL;DR

This paper introduces a novel joint analysis-synthesis framework for recovering speech signals from compressed sensing measurements. By simultaneously estimating both the speech production model (formant and pitch filters) and the sparse residual signal, they achieve significantly improved speech reconstruction quality compared to previous methods.

What Problem Does This Paper Address?

Compressive Sensing (CS) relies on signals being sparse in some transform domain for efficient recovery from undersampled measurements. While many signals are sparse in standard transforms like Fourier or wavelets, speech signals are often only sparse when modeled by specific audio production models. Traditional CS approaches for speech often assume a known model or a "synthesis sparse" model where the signal is a sparse linear combination of basis functions. However, the actual speech production model (like linear prediction filters) is unknown and time-varying, making direct application of CS challenging. This paper addresses the problem of recovering speech signals from CS measurements when the underlying sparse representation model itself needs to be estimated from the data.

Methods

The core contribution is a joint pitch-analysis, formant-synthesis model (PAFSM). This framework extends previous work by:

  • Incorporating both Formant and Pitch Prediction: Formant filters model the spectral envelope (all-pole system), while pitch filters capture the long-term periodicity in voiced speech segments. This dual modeling aims to induce greater sparsity.
  • Analysis-Synthesis Duality: The paper leverages the idea that a signal can be sparse either in its "analysis" domain (e.g., after passing through an analysis filter) or in its "synthesis" domain (e.g., as a sparse combination of basis functions generated by a synthesis filter). They propose a PAFSM that uses a pitch analysis filter and a formant synthesis filter.
  • Iterative Reconstruction Algorithm: A key aspect is an iterative algorithm that alternates between:
  • Reconstruction: Solving the CS recovery problem (minimizing the L1 norm of the sparse residual) given current estimates of the model filters.
  • Model Update: Estimating the parameters of the formant and pitch filters (and the pitch period) based on the currently reconstructed sparse residual and signal, using a Minimum Mean Square Error (MMSE) approach.
  • Blind Estimation: The entire process is designed to work in a "blind" scenario, meaning it estimates both the signal and the model parameters without prior knowledge of either.

Datasets

The experiments were conducted using a BBC broadcast news clip of 30 seconds duration, sampled at 8 kHz. The signal was analyzed in non-overlapping frames of 40 msec. No new datasets were introduced; standard speech data was used for evaluation.

Key Insights and Ideas

  • Modeling Periodicity is Crucial: Explicitly modeling the long-term periodicity of voiced speech using a pitch filter significantly improves the sparsity of the residual signal, leading to better reconstruction.
  • Joint Estimation is Powerful: Estimating the signal and the model parameters (filters) simultaneously, rather than sequentially, is essential for achieving high-quality reconstruction when the model is unknown.
  • Analysis vs. Synthesis Sparsity: The paper highlights that for signals with potentially unstable synthesis filters, an "analysis sparse" model can offer a more robust approach. Their PAFSM combines the benefits by using a pitch analysis filter and a formant synthesis filter.
  • Iterative Refinement: The iterative nature of the algorithm allows for progressive refinement of both the signal estimate and the underlying speech production model, leading to a more accurate final reconstruction.

Key Results

  • Improved Reconstruction SNR (RSNR): The proposed PAFSM method consistently outperformed a baseline greedy algorithm and other single-model approaches (FSM, FAM) across various sub-sampling factors (M/N = 0.25, 0.5, 0.75). For M/N = 0.75, PAFSM achieved up to a 5 dB improvement in RSNR over the baseline.
  • Reduced Log-Likelihood Ratio (LLR): The PAFSM method also demonstrated significantly lower LLR values compared to the baseline, indicating a much closer match between the estimated and original speech model parameters. This suggests improved perceptual quality. For example, LLR was reduced by more than 50% compared to the baseline.
  • Qualitative Improvement: Spectrogram analysis showed that the PAFSM-recovered signals closely matched the original signal's spectra, particularly during voiced segments, outperforming other methods.

Limitations and Future Work

  • Computational Complexity: The iterative nature of the algorithm, especially the L1 minimization and model parameter estimation steps, can be computationally intensive.
  • Frame-Based Processing: The current approach processes speech in frames. Extending it to a more continuous, sample-by-sample or block-adaptive framework could be beneficial.
  • Model Order Selection: The paper uses fixed orders for the formant (p=10) and pitch (q=2) filters. Investigating adaptive model order selection based on the signal characteristics could further enhance performance.
  • Extension to Unvoiced Speech: The current focus is on voiced segments due to the pitch filter. Future work could explore extending the framework to effectively handle unvoiced speech segments.

Summary updated

BibTeX
@inproceedings{chetupally2012joint,
  author = {Srikanth Raj Chetupally and Thippur V. Sreenivas},
  title = {{Joint pitch-analysis formant-synthesis framework for CS recovery of speech}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure