Reverse Engineering of Music Mixing Graphs with Differentiable Processors and Iterative Pruning

Sungho Lee, Marco A. Martínez Ramírez, Wei-Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, Yuki Mitsufuji

Abstract

Reverse engineering music mixes involves uncovering how dry source signals are processed and combined to produce a final mix. This paper presents a method to search for a compositional graph of audio processors by constructing a dense mixing console and applying differentiable signal processing. By optimizing parameters via gradient descent and employing iterative pruning, we can identify the underlying structure and settings of a mix. Our approach preserves the quality of the full mixing console while removing approximately two-thirds of the processors. This method facilitates large-scale graph data collection for downstream tasks like automatic mixing. We also introduce an efficient batch-processing method to accelerate the search, alongside extensive quantitative and qualitative evaluations.

Introduction

Music mixing is a complex, creative process where "dry" tracks—such as isolated vocals, drums, and guitars—are processed through various effects and grouped into subgroups to create a polished "wet" mix. For researchers and engineers, reverse engineering this process is a significant challenge. The goal is to identify the specific processors (EQs, compressors, reverbs), their order, their grouping, and their exact parameter settings.

Previous research often relied on fixed, linear effect chains. However, real-world mixing is inherently compositional, resembling a directed acyclic graph (DAG). This paper addresses the massive search space of these graphs by seeking the sparsest possible representation that accurately reconstructs a target mix.

Methodology

The Dense Mixing Console

The proposed system begins with a "Dense" Mixing Console ($G_c$). In this stage, the system assumes an over-complete architecture where every input track and every subsequent subgroup is assigned a full chain of seven differentiable processors: equalizer, compressor, noise gate, stereo imager, gain/panning, multitap delay, and reverb.

Differentiable Optimization

By implementing these processors as differentiable modules in PyTorch, the system can leverage gradient descent. This allows for the simultaneous optimization of thousands of parameters across the entire graph to minimize the difference between the generated output and the target professional mix.

Iterative Pruning

To move from a dense, unrealistic console to a sparse, human-like mixing structure, the system employs iterative pruning—a technique borrowed from neural network optimization. The "dry/wet" parameter of each effect serves as a proxy for its importance. If the optimizer reduces the "wet" signal of a processor to near zero, that processor is deemed negligible. The system iteratively removes these components and fine-tunes the remaining ones to maintain sonic integrity.

Computational Efficiency

Processing complex graphs with over 100 tracks is computationally expensive. To solve this, the authors developed a custom scheduling algorithm for batch processing. This allows the GPU to compute multiple independent nodes of the mixing graph in parallel, significantly reducing the time required for optimization.

Experimental Setup

Datasets

The study utilized a robust dataset of 1,129 songs from three primary sources:

  • MedleyDB & MixingSecrets: Standard public multitrack datasets for academic evaluation.
  • Internal Dataset: A private collection of 579 songs crowdsourced from professional engineers, ensuring the model is tested against high-quality Western music production standards.

Evaluation Metrics

The primary objective metric used was the Multi-Resolution Short-Time Fourier Transform (MRSTFT) loss. The researchers found that MRSTFT correlated most strongly with human subjective listening tests compared to other metrics like FAD or SI-SDR.

Results and Discussion

Sparsity and the "67% Rule"

One of the most significant findings was that approximately 67% of the processors in the initial dense console could be pruned without a perceptible loss in quality. This indicates that professional mixes are often much sparser than a "maximalist" approach would suggest, and the pruning algorithm effectively identifies the essential components of a mix.

Perceptual Transparency

In MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) subjective listening tests, participants were often unable to distinguish between the original full console and the pruned versions (using a tolerance of $\tau=0.01$). This confirms that the iterative pruning successfully preserves the artistic intent of the mix.

Consistency and Stochasticity

When tested against known synthetic graphs, the system achieved a high accuracy of 0.902 in processor detection. Interestingly, the pruning process exhibited stochastic behavior; different runs on the same song could yield different graph structures that were perceptually identical. This mirrors the real-world reality where different engineering paths can lead to the same sonic result.

Limitations and Future Work

Despite its success, the model has limitations:

  • Processor Variety: The current suite lacks non-linear effects such as saturation, distortion, or pitch correction.
  • Static Nature: The model assumes parameters are static, failing to account for "automation" (changes in faders or EQ over time).
  • Phase Issues: Current loss functions can occasionally lead to temporal "smearing" in time-based effects like reverb.
  • Topology: The search is currently constrained to pruning a pre-defined tree. Future iterations may utilize Neural Architecture Search (NAS) to allow the AI to propose entirely original routing and side-chaining paths.

Conclusion

This research provides a powerful framework for the automated analysis and reconstruction of music mixes. By combining differentiable signal processing with iterative pruning, the system can efficiently navigate the complex search space of mixing graphs, providing a foundation for future AI-driven mixing tools and large-scale musicological studies.

Summary updated

Abstract

Reverse engineering of music mixes aims to uncover how dry source signals are processed and combined to produce a final mix. We extend the prior works to reflect the compositional nature of mixing and search for a graph of audio processors. First, we construct a mixing console, applying all available processors to every track and subgroup. With differentiable processor implementations, we optimize their parameters with gradient descent. Then, we repeat the process of removing negligible processors and fine-tuning the remaining ones. This way, the quality of the full mixing console can be preserved while removing approximately two-thirds of the processors. The proposed method can be used not only to analyze individual music mixes but also to collect large-scale graph data that can be used for downstream tasks, e.g., automatic mixing. Especially for the latter purpose, efficient implementation of the search is crucial. To this end, we present an efficient batch-processing method that computes multiple processors in parallel. We also exploit the "dry/wet" parameter of the processors to accelerate the search. Extensive quantitative and qualitative analyses are conducted to evaluate the proposed method's performance, behavior, and computational cost.
BibTeX
@article{lee2025reverse,
  author = {Sungho Lee and Marco A. Martínez Ramírez and Wei-Hsiang Liao and Stefan Uhlich and Giorgio Fabbro and Kyogu Lee and Yuki Mitsufuji},
  title = {{Reverse Engineering of Music Mixing Graphs with Differentiable Processors and Iterative Pruning}},
  journal = {Journal of the Audio Engineering Society},
  year = {2025},
  doi = {10.17743/jaes.2022.0212},
}
Figure