Audio declipping with social sparsity

Kai Siedenburg, Matthieu Kowalski, Monika Dörfler

TL;DR

This paper tackles the problem of "audio declipping," where parts of an audio signal have been clipped (truncated due to exceeding amplitude limits). They introduce a novel unconstrained convex formulation that allows for efficient iterative algorithms. By incorporating "social sparsity" and structured shrinkage operators, they achieve significantly better audio reconstruction quality compared to existing methods, particularly in preserving signal details.

What Problem Does This Paper Address?

Digital audio often suffers from clipping, a distortion that occurs when the signal's amplitude exceeds the maximum representable value, leading to truncation. This results in unpleasant artifacts. While traditional methods exist for audio restoration, declipping specifically requires not only reconstructing missing samples but also ensuring the reconstructed values respect the original signal's amplitude limits. Existing sparsity-based approaches, while promising, often lead to computationally expensive optimization problems when incorporating these clipping constraints. This paper aims to bridge this gap by proposing a more efficient and effective method for audio declipping.

Methods

The core of the paper lies in two main contributions:

  1. Unconstrained Convex Formulation: The authors propose a new way to formulate the declipping problem as an unconstrained convex optimization problem. They achieve this by using a "squared hinge function" to penalize reconstructed samples that fall below the clipping threshold. This clever reformulation allows them to leverage efficient iterative shrinkage/thresholding algorithms (ISTA family), which are generally faster and simpler to implement than methods requiring complex constrained optimization.
  2. Social Sparsity and Structured Shrinkage Operators: To further improve reconstruction quality, the paper incorporates the concept of "social sparsity." This principle allows the algorithm to model dependencies between coefficients in time-frequency representations. Specifically, they employ structured shrinkage operators like the Windowed Group-Lasso (WGL) and, more importantly, the Persistent Empirical Wiener (PEW). These operators go beyond simple soft-thresholding by considering temporal relationships between coefficients, effectively capturing signal components that persist over time (like harmonics) while discarding transient artifacts.

The proposed algorithm, based on a relaxed version of ISTA, iteratively refines the signal's representation using these structured operators.

Datasets

The authors evaluated their approach using audio data from the http://small-project.eu/ website. This dataset consists of speech and music audio files, sampled at 16kHz, each 5 seconds in duration. They created various clipping scenarios by normalizing the signals and then clipping them at different levels (0.1, 0.2, ..., 0.9). This approach allows for a systematic evaluation of declipping performance across different signal types and severities of clipping.

Key Insights and Ideas

  • Unconstrained is Better (for efficiency): Reformulating the declipping problem into an unconstrained convex optimization problem is a key enabler for using efficient ISTA-like algorithms, significantly reducing computational burden compared to constrained methods.
  • Social Sparsity for Temporal Structure: Exploiting "social sparsity" by considering temporal dependencies between time-frequency coefficients is crucial for high-quality audio reconstruction. This allows the algorithm to better distinguish between persistent signal components and transient distortions.
  • PEW Operator's Power: The Persistent Empirical Wiener (PEW) operator, which combines energy preservation (Empirical Wiener) with temporal persistence (Persistent), emerges as a particularly effective tool for audio declipping. It significantly outperforms simpler shrinkage methods.
  • Warm-Starting for Speed: The paper highlights the benefit of a "warm-start" strategy for the hyperparameter tuning. By starting with a large regularization parameter and gradually decreasing it, the algorithm achieves good performance much faster, simulating a small parameter without the slow convergence issues.

Key Results

  • Improved SNR: The proposed method, particularly using the PEW operator, consistently achieves higher Signal-to-Noise Ratio (SNRm) compared to state-of-the-art methods like OMP (Orthogonal Matching Pursuit) and HT (Iterative Hard Thresholding). Improvements of up to 5 dB SNRm were observed.
  • Qualitative Improvements: The PEW-declipped signals exhibit remarkably few audible artifacts, even in cases of severe clipping. They more closely resemble the original signal's profile beyond the clipped regions.
  • Efficiency: The algorithm is described as fast and simple to implement, outperforming OMP in terms of computational time while achieving better results.
  • Neighborhood Influence: The choice of neighborhood size for the persistent operators (WGL, PEW) impacts performance. For speech, smaller neighborhoods (length 3) were found to be beneficial, while for music, larger neighborhoods (length 5 or 7) yielded better results.

Limitations and Future Work

  • Formal Listening Tests: The authors acknowledge that formal listening experiments were not conducted due to resource limitations, although they noted subjectively that PEW estimates had few audible artifacts.
  • Neighborhood Optimization: The choice of neighborhood size and shape for the persistent operators is noted as being "delicate" and signal-dependent, suggesting further research into optimal neighborhood selection.
  • Generalization to Inpainting: The authors express interest in applying their approach to the broader "audio inpainting" problem, which involves reconstructing larger missing segments of audio. They believe exploiting neighborhood persistence in the time-frequency domain will be valuable for handling significant data loss.
  • Theoretical Characterization: Future theoretical work is planned to further characterize the PEW operators in conjunction with ISTA-type algorithms.

Summary updated

Abstract

We consider the audio declipping problem by using iterative thresholding algorithms and the principle of social sparsity. This recently introduced approach features thresholding/shrinkage operators which allow to model dependencies between neighboring coefficients in expansions with time-frequency dictionaries. A new unconstrained convex formulation of the audio declipping problem is introduced. The chosen structured thresholding operators are the so called windowed group-Lasso and the persistent empirical Wiener. The usage of these operators significantly improves the quality of the reconstruction, compared to simple soft-thresholding. The resulting algorithm is fast, simple to implement, and it outperforms the state of the art in terms of signal to noise ratio.
BibTeX
@inproceedings{siedenburg2014audio,
  author = {Kai Siedenburg and Matthieu Kowalski and Monika Dörfler},
  title = {{Audio declipping with social sparsity}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2014},
  url = {https://openreview.net/forum?id=dTSL1qv5S2},
}
Figure