Reverse Engineering of Music Mixing Graphs With Differentiable Processors and Iterative Pruning

Sungho Lee, Marco A. Martínez-Ramírez, Wei‐Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, Yuki Mitsufuji

TL;DR

Researchers have developed a way to reverse engineer music mixes by reconstructing the complex graph of audio processors (EQs, compressors, reverbs) used to create them. By combining differentiable signal processing with iterative pruning techniques borrowed from neural network optimization, the system can identify both the structure of a mixing console and the specific parameters needed to match a target mix.

What Problem Does This Paper Address?

Music mixing is a highly compositional and creative process where "dry" tracks (vocals, drums, guitars) are processed through chains of effects and grouped together to create a final "wet" mix. For researchers building automatic mixing tools or analyzing professional engineering practices, "reverse engineering" this process is a holy grail.

The challenge is that the search space is massive. You aren't just trying to find the right setting for a single knob; you're trying to figure out which processors exist, in what order, and how they are grouped. Previous attempts often assumed a fixed, simple chain of effects. This paper tackles the "compositional" nature of mixing—the fact that it is a directed acyclic graph (DAG) of processors—and seeks to find the simplest (sparsest) graph that still sounds like the original.

Methods

The authors propose a two-stage strategy:

  1. The "Dense" Mixing Console: They start by constructing a massive, over-complete mixing console ($G_c$). Every input track and every subgroup is assigned a full chain of seven differentiable processors: equalizer, compressor, noise gate, stereo imager, gain/panning, multitap delay, and reverb.
  2. Differentiable Optimization: Because the processors are implemented as differentiable modules (using PyTorch), the system can use gradient descent to optimize thousands of parameters simultaneously to minimize the difference between the generated mix and the target.
  3. Iterative Pruning: To find a realistic, sparse graph, they borrow a technique from neural network compression. They iteratively remove "negligible" processors—those that don't contribute much to the final sound—and then fine-tune the remaining ones. They use the "dry/wet" parameter of each effect as a proxy for its importance.
  4. Efficient Batching: To make this computationally feasible, they developed a custom scheduling algorithm that allows the GPU to process multiple independent nodes of the mixing graph in parallel, rather than one by one.

Datasets

The study utilized a massive collection of 1,129 songs across three main sources:

  • MedleyDB & MixingSecrets: Well-known public multitrack datasets used for evaluation.
  • Internal Dataset: A private collection of 579 songs crowdsourced from professional engineers to provide a robust training and testing ground for Western music styles.

Key Insights and Ideas

  • The 67% Rule: The researchers found that approximately two-thirds of the processors in their dense starting console could be pruned away without a significant loss in perceptual quality. This suggests that real-world mixes are often much sparser than a "kitchen sink" approach.
  • Dry/Wet as an Importance Score: One of the "aha" moments was treating the dry/wet knob as a learnable gate. If the optimizer turns the "wet" signal down to near zero, the pruning algorithm knows that specific processor is a prime candidate for removal.
  • Stochasticity in Mixing: Interestingly, the pruning process is stochastic. Running the algorithm multiple times on the same song can result in slightly different mixing graphs that all sound nearly identical. This mirrors the real world, where two different engineers might achieve a similar "sound" using different tools.
  • MRSTFT is King: Among various objective metrics (like FAD or SI-SDR), the Multi-Resolution Short-Time Fourier Transform (MRSTFT) loss showed the strongest correlation with human subjective listening tests, making it the most reliable "compass" for the optimization.

Key Results

  • Perceptual Transparency: In MUSHRA (subjective listening) tests, participants found that the pruned graphs (with a tolerance of $\tau=0.01$) were perceptually indistinguishable from the full, unpruned mixing consoles.
  • High Accuracy: In a "consistency check" (where the system tried to recover a known synthetic graph), it achieved an average accuracy of 0.902 in detecting whether a specific processor type was present or not.
  • Efficiency: The batched processing method provided significant speedups, allowing the system to process complex graphs with over 100 tracks in a reasonable timeframe (averaging 36 minutes per song on a single RTX 3090).

Limitations and Future Work

While the system is a major step forward, the authors acknowledge several hurdles:

  • Missing Effects: The current set of seven processors doesn't include non-linear "character" effects like saturation, distortion, or pitch correction (Auto-Tune).
  • Static Parameters: The model assumes mixing parameters are constant throughout the song. It cannot yet account for "automation"—the common practice of faders or EQ settings moving over time.
  • Phase and Smearing: The loss functions used don't perfectly model phase, which can sometimes lead to "smearing" artifacts in time-heavy effects like reverbs and delays.
  • Topology Constraints: The search is currently limited to pruning a pre-defined tree structure. Future work could involve "Neural Architecture Search" (NAS) to allow the AI to propose entirely new, creative routing paths (like complex side-chaining) from scratch.

Summary updated

Abstract

Reverse engineering of music mixes aims to uncover how dry source signals are processed and combined to produce a final mix. In this paper, prior works are extended to reflect the compositional nature of mixing and search for a graph of audio processors. First, a mixing console is constructed, applying all available processors to every track and subgroup. With differentiable processor implementations, their parameters are optimized with gradient descent. Next, the process of removing negligible processors and fine-tuning the remaining ones is repeated. This way, the quality of the full mixing console can be preserved while removing approximately two-thirds of the processors. The proposed method can be used not only to analyze individual music mixes but also to collect large-scale graph data for downstream tasks such as automatic mixing. Especially for the latter purpose, efficient implementation of the search is crucial. To this end, an efficient batch-processing method that computes multiple processors in parallel is presented. Also, the “dry/wet” parameter of each processor is exploited to accelerate the search. Extensive quantitative and qualitative analyses are conducted to evaluate the proposed method’s performance, behavior, and computational cost.
BibTeX
@article{lee2025reverse,
  author = {Sungho Lee and Marco A. Martínez-Ramírez and Wei‐Hsiang Liao and Stefan Uhlich and Giorgio Fabbro and Kyogu Lee and Yuki Mitsufuji},
  title = {{Reverse Engineering of Music Mixing Graphs With Differentiable Processors and Iterative Pruning}},
  journal = {Journal of the Audio Engineering Society},
  year = {2025},
  doi = {10.17743/jaes.2022.0212},
}

Loading related content...

Figure