Diagnostic prediction of transmitted speech quality: a new framework for signal-based and parametric models

Sebastian Möller, Marcel Wältermann, Nicolas CôtéInterspeech 2012

TL;DR

This paper introduces a novel framework for predicting speech quality that goes beyond a single "overall quality" score. By analyzing speech through key perceptual dimensions like "coloration," "noisiness," and "discontinuity," their models offer more insightful diagnostic information about why speech quality might be suboptimal, aiding in network troubleshooting and planning.

What Problem Does This Paper Address?

In today's complex telecommunication networks, especially with the rise of Voice-over-IP (VoIP), ensuring high speech quality for users is paramount. Traditional speech quality prediction models, like PESQ or the E-model, provide a single "Mean Opinion Score" (MOS) that summarizes overall quality. While useful, this single number offers little insight into the specific reasons behind a perceived quality degradation. If a call has a low MOS, it's hard to tell if the problem is due to background noise, audio artifacts, or dropped packets. This lack of diagnostic detail makes it challenging for network engineers to pinpoint and fix the root causes of poor speech quality. This paper aims to bridge this gap by developing models that not only predict overall quality but also diagnose the specific perceptual dimensions contributing to it.

Methods

The authors propose a "dimension-based" framework that breaks down speech quality into perceptually relevant dimensions. They identify three core dimensions: coloration (related to frequency response), noisiness (background or signal-correlated noise), and discontinuity (interruptions or artifacts). Loudness is also considered as a fourth dimension, particularly in the signal-based model.

They implement this framework in two complementary ways:

  1. Signal-based Model (DIAL): This model analyzes the actual input and output speech signals. It estimates the maximum achievable quality and then estimates the impairment for each of the identified perceptual dimensions. These dimension-specific estimates are then combined to predict the overall MOS. This approach is suitable for real-time, per-sample quality assessment.
  2. Parametric Model (DNC): This model uses network and codec parameters (like those used in the E-model) to predict the perceptual dimension impairments. Instead of directly mapping parameters to an overall MOS, it first estimates the impairment for each dimension and then combines these to predict the overall quality. This is useful for network planning before a channel is even set up.

Both models aim to provide both an integral quality score and a diagnostic profile of dimension estimates.

Datasets

The authors evaluated their models using two mixed narrowband and wideband databases (labeled A and B). These databases were chosen because they contained not only subjective integral quality judgments (MOS) but also subjective judgments on perceptual dimensions. The paper notes that these specific databases were also used for the derivation of the DNC model, which is a caveat for its performance evaluation.

Key Insights and Ideas

  • Beyond the Single Score: The core idea is that a single MOS score is insufficient for effective troubleshooting. Understanding why quality is degraded (e.g., "it's noisy" vs. "it's choppy") is crucial.
  • Perceptual Dimensions as Building Blocks: Decomposing speech quality into orthogonal perceptual dimensions (coloration, noisiness, discontinuity) provides a more structured and interpretable way to assess quality.
  • Complementary Models: The development of both signal-based (DIAL) and parametric (DNC) models demonstrates the framework's versatility, catering to different stages of network management (real-time monitoring vs. planning).
  • Diagnostic Value: The dimension-based approach demonstrably provides diagnostic information that standard models lack, correlating well with subjective dimension judgments.

Key Results

  • Integral Quality Prediction: Both DIAL and DNC models achieved high correlations (R > 0.8, often > 0.9) with subjective integral quality judgments, performing comparably to or better than standard models like WB-PESQ and the WB-E-model in many cases.
  • Diagnostic Power: The models showed good performance in predicting individual perceptual dimensions. For DIAL, coloration estimation was particularly strong (>0.95 correlation), while noisiness estimation was moderate. DNC showed high correlations across all three dimensions.
  • Visualization: Figure 1 illustrates how the DNC model's integrated impairment score (Itot) aligns with subjective ratings, showing it can accurately weight different types of degradations (codec, linear distortions, packet loss, noise).

Limitations and Future Work

  • Database Dependency: The parametric DNC model's performance evaluation was conducted on databases used for its derivation, meaning its superiority over the WB-E-Model might not be fully proven on independent data.
  • Noisiness Estimation: The signal-based DIAL model still faces challenges in accurately estimating the noisiness dimension.
  • Future Directions:
    • Develop a non-intrusive signal-based implementation (relying only on the degraded signal).
  • Validate the approach with independent databases and compare it to the new standard POLQA.
  • The DIAL model, with improvements, could be a candidate for a future ITU-T standard for dimension-based quality assessment.
  • Extend the framework to other quality estimation problems, such as synthesized speech or audio-visual communication.

Summary updated

BibTeX
@inproceedings{moller2012diagnostic,
  author = {Sebastian Möller and Marcel Wältermann and Nicolas Côté},
  title = {{Diagnostic prediction of transmitted speech quality: a new framework for signal-based and parametric models}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure