Multi-Dimensional Decomposition of Perceived Audio Quality

62 concepts34 papers4 proposals

When humans listen to a degraded audio signal—whether it is a choppy cell phone call, a heavily compressed MP3, or an AI-generated synthetic voice—we rarely evaluate it as a single, monolithic "good" or "bad" experience. Instead, our auditory system naturally separates different types of degradation. We notice that a voice sounds muffled (spectral filtering), that there is a hiss in the background (additive noise), or that the audio is cutting out (temporal interruption).

Historically, automated audio evaluation has compressed this rich, multi-faceted human experience into a single scalar metric, such as a Mean Opinion Score (MOS). While a single score is convenient for benchmarking, it acts as a black box. If a speech enhancement algorithm receives a low MOS score, the score alone cannot tell an engineer whether the algorithm failed to suppress background noise or if it suppressed the noise perfectly but heavily distorted the underlying speech.

To solve this, a major research direction has emerged around perceptual dimension decomposition. Instead of predicting a single aggregate quality score, this paradigm breaks down audio quality into its constituent, orthogonal perceptual axes. By modeling these individual dimensions, researchers can build evaluation tools that are not only more accurate but also inherently diagnostic.


The Core Perceptual Axes of Audio Degradation

The foundation of this approach lies in identifying the fundamental dimensions that define audio degradation. Early psychophysical research established that human listeners evaluate telecommunication channels along distinct, identifiable axes rather than a vague continuum of quality.

As shown by Wältermann et al. in their study on modern telephone connections, human perception of speech quality can be robustly mapped to three primary dimensions:

  1. Noisiness: The perception of additive, unwanted background signals or ambient noise.
  2. Continuity: The temporal smoothness of the signal, capturing whether the audio is fluid or suffers from packet loss and "choppy" interruptions.
  3. Directness / Frequency Content (Coloration): The spectral fidelity of the voice, capturing whether it sounds natural, muffled, or overly metallic.
                  ┌─────────────────────────┐
                  │ Perceived Audio Quality │
                  └────────────┬────────────┘
                               │
         ┌─────────────────────┼─────────────────────┐
         ▼                     ▼                     ▼
┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐
│    Noisiness    │   │   Continuity    │   │   Coloration    │
│ (Additive Hiss) │   │ (Temporal Gaps) │   │ (Muffled/Tinny) │
└─────────────────┘   └─────────────────┘   └─────────────────┘

By isolating these dimensions, instrumental models can target specific acoustic features. For instance, Scholz et al. demonstrated that isolating the "noisiness" dimension alone and mapping it to specific acoustic features like signal-correlated noise and spectral composition yields an incredibly high correlation (>0.95) with human subjective ratings for that specific attribute.


From Dimensions to Overall Quality

Why does decomposing quality into these dimensions work so well for predicting overall human satisfaction? The answer lies in how our brains integrate multiple distinct sensory inputs.

The authors of several foundational studies explored how these individual dimensions combine to form an overall impression of quality. Wältermann et al. demonstrated that a simple linear model incorporating these three perceptual dimensions can predict overall listening quality with remarkable accuracy, explaining approximately 90% of the variance in human scores. This suggests that, under normal conditions, our overall judgment of quality is largely an additive function of independent perceptual impairments.

However, this linear assumption has boundaries. Berger et al. discovered that when multiple severe impairments are present simultaneously—such as extreme background noise combined with severe packet loss—the dimensions begin to interact in non-linear ways. In these scenarios, one catastrophic failure dimension can dominate the user experience, a nuance that modern diagnostic frameworks attempt to capture by first establishing a baseline of maximum achievable quality and then sequentially subtracting dimension-specific degradations (Möller et al.).


Deep Learning and Multi-Output Diagnostics

With the advent of deep learning, the principle of perceptual decomposition has transitioned from hand-crafted acoustic features to multi-task neural network architectures.

Instead of training a neural network to map a spectrogram directly to a single human MOS score, modern architectures are designed with multiple output heads or specialized sub-networks. A prominent example is the NISQA architecture introduced by Mittag et al., which uses a deep CNN-Self-Attention model to simultaneously predict overall quality alongside explicit scores for noisiness, coloration, discontinuity, and loudness.

                          ┌───────────────────┐
                          │  Audio Spectrogram│
                          └─────────┬─────────┘
                                    │
                                    ▼
                        ┌───────────────────────┐
                        │ Shared Latent Feature │
                        │      Extraction       │
                        └───────────┬───────────┘
                                    │
         ┌──────────────┬───────────┴───┬──────────────┐
         ▼              ▼               ▼              ▼
   ┌───────────┐  ┌───────────┐   ┌───────────┐  ┌───────────┐
   │ Noisiness │  │Coloration │   │Discontinu-│  │ Overall   │
   │   Head    │  │   Head    │   │ ity Head  │  │ MOS Head  │
   └───────────┘  └───────────┘   └───────────┘  └───────────┘

This multi-task formulation provides a powerful regularizer during training. By forcing the network's internal representations to explicitly account for distinct physical phenomena (like background noise versus speech distortion), the network is prevented from relying on spurious correlations or "shortcut learning."

This exact mechanism is what makes metrics like DNSMOS P.835, developed by Gopal et al., so effective for evaluating noise suppressors without a clean reference signal. By outputting separate scores for speech distortion and background noise, it allows developers to immediately see the trade-offs of their models—such as an algorithm that achieves perfect noise suppression at the cost of destroying voice naturalness. Similarly, in generative speech enhancement, Yuan et al. leveraged this concept by designing a multi-branch discriminator that splits the frequency spectrum to evaluate high-frequency noise intrusiveness and low-frequency speech distortion independently.


Generalizing the Paradigm Beyond Speech Codecs

The intuition behind perceptual dimension decomposition is not restricted to telecommunication noise; it has successfully generalized across the entire spectrum of audio and speech evaluation tasks.

  • Text-to-Speech (TTS): Rather than relying on traditional signal-fidelity metrics, the Text-to-Speech Distribution Score (TTSDS) proposed by Minixhofer et al. decomposes synthetic speech evaluation into statistical distributions of distinct feature dimensions, namely prosody, speaker identity, and intelligibility.
  • Singing Voice Evaluation: In the PESnQ framework introduced by Gupta et al., complex expert human judgment of singing is broken down into discrete, measurable musical traits like intonation accuracy and vibrato consistency before being aggregated.
  • Music Aesthetics: For full-length AI-generated music, benchmarks like SongEval (Yao et al.) score audio across structural coherence, melodic memorability, and vocal naturalness to capture the nuances of human musical appreciation.
  • Audio Captioning: Even when evaluating text descriptions of audio, Wang et al. showed that breaking evaluation down into four distinct semantic dimensions—sound events, sources, attributes, and relations—pinpoints exactly where audio language models fail in a way that aggregate semantic similarity scores cannot.

By moving away from monolithic scoring and embracing the multi-dimensional nature of human perception, these frameworks provide engineers with the granular, diagnostic feedback required to systematically isolate and fix the root causes of audio degradation.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_perceptual_dimension_decomposition_52ce23ec,
  title = {perceptual dimension decomposition},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/perceptual-dimension-decomposition/rdr_52ce23ec1bf6403db078c369e5e3f69f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_perceptual_dimension_decomposition_52ce23ec_v1,
  title = {perceptual dimension decomposition},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/perceptual-dimension-decomposition/rdr_52ce23ec1bf6403db078c369e5e3f69f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake