Perceptual Audio Coding: A 40-Year Historical Perspective

Jürgen Herre, Schuyler Quackenbush, Minje Kim, Jan SkoglundICASSP 2025 · DOI

TL;DR

This paper provides a comprehensive historical overview of perceptual audio coding, tracing its evolution from early psychoacoustic models to modern neural network-based approaches. It highlights the key technological advancements and standardization efforts that have made high-quality audio compression ubiquitous across digital media, while also looking towards future challenges and opportunities.

What Problem Does This Paper Address?

The paper addresses the challenge of efficiently representing audio signals for digital storage, transmission, and playback without sacrificing perceived quality. Traditional digital audio, like uncompressed PCM, is highly redundant and requires significant bandwidth. Perceptual audio coding aims to overcome this by exploiting the limitations of human hearing to discard information that is imperceptible or less perceptible, thereby reducing the data rate. The paper chronicles the journey of developing and refining these techniques over four decades, from basic psychoacoustic principles to sophisticated integrated systems.

Methods

The paper outlines the fundamental principles of perceptual audio coding, which revolve around psychoacoustics and masking.

  • Masking: This phenomenon, where louder sounds obscure softer sounds in both frequency (simultaneous masking) and time (temporal masking), is the cornerstone of perceptual coding. The paper explains how understanding these masking thresholds allows coders to strategically introduce quantization noise below the audible limit.
  • Filterbank Architecture: A common approach involves analyzing the audio signal in the frequency domain using filterbanks. The spectral coefficients are then quantized based on the perceptual model's masking thresholds.
  • Transform Coding: Techniques like the Modified Discrete Cosine Transform (MDCT) are crucial for achieving good time-frequency resolution, balancing the need to capture transient sounds (requiring good time resolution) and tonal sounds (requiring good frequency resolution).
  • Predictive Coding: For speech, Linear Predictive Analysis-by-Synthesis (LPAS) and its variants (like CELP) were early successes, modeling speech as an excitation signal filtered by a linear predictor.
  • Hybrid Approaches: More recent developments combine waveform coding with parametric side information for tasks like Bandwidth Extension (BWE) and spatial audio coding, achieving significant compression at very low bitrates.
  • Neural Audio Coding: The paper introduces the emerging paradigm of neural audio coding, often formulated as autoencoders, which learn to compress and reconstruct audio signals end-to-end, potentially achieving even higher compression gains.

Datasets

Not discussed in the paper. The paper focuses on the historical development of algorithms and standards rather than specific empirical evaluations on particular datasets.

Key Insights and Ideas

  • The "Virtuous Cycle": Perceptual audio coding's success is attributed to a strong collaboration between academia and industry, leading to standardization and widespread commercial adoption.
  • Masking is King: The entire field is built upon understanding and exploiting the psychoacoustic phenomenon of masking to hide coding artifacts.
  • Trade-offs are Inherent: Achieving transparency at very low bitrates is often impossible. Perceptual coding strategically prioritizes perceived quality over objective error metrics like SNR.
  • Evolution Beyond Bitrate: Modern codecs go beyond simple compression, incorporating features like low-delay operation for communication, error resilience for wireless, and sophisticated spatial and object-based coding for immersive experiences.
  • Neural Networks as the Next Frontier: Data-driven approaches, particularly neural networks, are showing promise for even greater compression efficiency, though challenges in complexity and generalization remain.

Key Results

The paper highlights the remarkable progress in reducing bitrates for "good quality" stereo audio over time. For instance, early codecs like MP3 (1992) required 128-256 kbps, while modern codecs can achieve similar quality at 20 kbps and below. This progress has enabled the ubiquitous presence of digital audio in devices ranging from mobile phones to high-fidelity playback systems. The paper also notes the development of codecs that can handle complex multi-channel and object-based audio for immersive applications like AR/VR.

Limitations and Future Work

  • Limitations of Neural Coders: While promising, current neural audio coding systems often exhibit increased computational complexity, making them expensive for resource-constrained devices. Many are also specialized for speech, requiring more generalization for general audio.
  • Future Directions: The paper points to the increasing integration of data-driven and machine learning approaches as a significant future direction. This includes exploring neural autoencoders for end-to-end compression and generative models for synthesis. Further research is needed to address the complexity and generalization challenges of neural codecs. The paper also mentions the ongoing development of codecs for immersive audio experiences (AR/VR) and the continued need for efficient coding solutions for a growing array of applications.

Summary updated

BibTeX
@inproceedings{herre2025perceptual,
  author = {Jürgen Herre and Schuyler Quackenbush and Minje Kim and Jan Skoglund},
  title = {{Perceptual Audio Coding: A 40-Year Historical Perspective}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10887760},
}
Figure