Abstract
Recent advances in audio declipping have substantially improved the state of the art. Yet, practitioners need guidelines to choose a method, and while existing benchmarks have been instrumental in advancing the field, larger-scale experiments are needed to guide such choices. First, we show that the clipping levels in existing small-scale benchmarks are moderate and call for benchmarks with more perceptually significant clipping levels. We then propose a general algorithmic framework for declipping that covers existing and new combinations of variants of state-of-the-art techniques exploiting time-frequency sparsity: synthesis vs. analysis sparsity, with plain or structured sparsity. Finally, we systematically compare these combinations and a selection of state-of-the-art methods. Using a large-scale numerical benchmark and a smaller scale formal listening test, we provide guidelines for various clipping levels, both for speech and various musical genres. The code is made publicly available for the purpose of reproducible research and benchmarking.
Introduction: The Challenge of Audio Clipping
Audio clipping, or saturation, occurs when a signal exceeds the maximum capacity of a recording device or software, resulting in "squared-off" waveforms and harsh harmonic distortion. While light clipping causes minor clicks, severe clipping destroys the signal's dynamics and timbre.
The authors identify two major gaps in the field:
- Benchmark Limitations: Most existing research relies on the "SMALL" dataset—a tiny collection of twenty 5-second clips. These benchmarks often focus on moderate clipping levels that don't reflect the "torture tests" required by real-world applications.
- Algorithmic Fragmentation: There are many competing declipping strategies (e.g., analysis vs. synthesis sparsity), but they are rarely compared under a single, consistent framework.
Methodology: A Unified Framework
The authors propose a Generic Declipping Algorithm that can be configured to mimic or extend most state-of-the-art sparsity methods. The core idea is to treat declipping as an underdetermined linear inverse problem, using time-frequency sparsity as a regularizer.
Key components include:
- Analysis vs. Synthesis: Synthesis models assume a signal is built from a few "atoms" in a dictionary, while analysis (or cosparse) models assume the signal becomes sparse when a specific operator is applied. The authors show that the analysis variant is often more computationally efficient.
- Social Sparsity: Unlike "plain" sparsity, which treats every time-frequency point independently, social sparsity looks at neighborhoods. It assumes that if a frequency is active at one moment, it likely persists in time or has related harmonics nearby.
- Adaptive Selection: They introduce an "Adaptive Social" variant that automatically selects the best time-frequency neighborhood pattern (e.g., tonal vs. percussive) based on the signal's local entropy.
- Consistency Constraints: The algorithm ensures that "reliable" (unclipped) samples remain unchanged while "clipped" samples are forced to stay above the original clipping threshold.
Large-Scale Evaluation
The study moves beyond the standard SMALLbox toolbox to a much larger scale:
- RWC Music Database: Over 13,000 seconds of audio across Pop, Jazz, and Classical (Vocals, Chamber, and Orchestra).
- TIMIT Database: 600 seconds of speech data.
- MUSHRA Listening Tests: Formal perceptual evaluations conducted with 26 human listeners to validate numerical results.
All data was down-sampled to 16 kHz for the numerical benchmarks to manage computational load, while the listening tests used full 44.1 kHz audio.
Key Insights and Metrics
One of the most important findings is that the "best" algorithm depends on the input Signal-to-Distortion Ratio (SDR). For severe clipping (Input SDR < 5 dB), plain cosparse models are the most robust. For mild clipping (Input SDR > 10 dB), structured social sparsity models significantly outperform plain ones in both numerical and perceptual scores.
The authors argue that the clipping threshold (τ) is a poor measure of degradation because it doesn't account for a signal's dynamic range. They advocate for using Input SDR as the primary metric for benchmarking. Furthermore, social sparsity acts as a powerful prior that prevents "musical noise" artifacts, but it can struggle if the signal is so corrupted that the algorithm selects the "wrong" neighborhood pattern during initialization.
Results and Discussion
- Quantitative Gains: For most datasets, the sparsity-based methods provided SDR improvements of 8 dB or more.
- Perceptual Superiority: In MUSHRA listening tests, all sparsity-based methods significantly outperformed traditional autoregressive interpolation. At 3 dB input SDR (heavy clipping), listeners slightly preferred the Adaptive Social algorithms.
- Computational Efficiency: The study highlights that while synthesis models have historically been the "go-to," analysis-based (cosparse) models are often faster and provide comparable, if not better, results for declipping.
Limitations and Future Outlook
The algorithms performed notably worse on the "Pop" category of the RWC database. The authors attribute this to the fact that modern pop music is often already heavily compressed or contains intentionally saturated guitar sounds, which violates the "clean signal is sparse" assumption. Additionally, while some variants are fast, the most sophisticated adaptive social models still require significant iterations, making them difficult to implement in true real-time low-latency systems.
Future directions include extending this framework to multichannel audio and exploring how these methods could be combined with deep learning-based "blind" clipping detection.
Summary updated