Abstract
Reverse engineering music mixes involves uncovering how dry source signals are processed and combined to produce a final mix. This paper presents a method to search for a compositional graph of audio processors by constructing a dense mixing console and applying differentiable signal processing. By optimizing parameters via gradient descent and employing iterative pruning, we can identify the underlying structure and settings of a mix. Our approach preserves the quality of the full mixing console while removing approximately two-thirds of the processors. This method facilitates large-scale graph data collection for downstream tasks like automatic mixing. We also introduce an efficient batch-processing method to accelerate the search, alongside extensive quantitative and qualitative evaluations.
Introduction
Music mixing is a complex, creative process where "dry" tracks—such as isolated vocals, drums, and guitars—are processed through various effects and grouped into subgroups to create a polished "wet" mix. For researchers and engineers, reverse engineering this process is a significant challenge. The goal is to identify the specific processors (EQs, compressors, reverbs), their order, their grouping, and their exact parameter settings.
Previous research often relied on fixed, linear effect chains. However, real-world mixing is inherently compositional, resembling a directed acyclic graph (DAG). This paper addresses the massive search space of these graphs by seeking the sparsest possible representation that accurately reconstructs a target mix.
Methodology
The Dense Mixing Console
The proposed system begins with a "Dense" Mixing Console ($G_c$). In this stage, the system assumes an over-complete architecture where every input track and every subsequent subgroup is assigned a full chain of seven differentiable processors: equalizer, compressor, noise gate, stereo imager, gain/panning, multitap delay, and reverb.
Differentiable Optimization
By implementing these processors as differentiable modules in PyTorch, the system can leverage gradient descent. This allows for the simultaneous optimization of thousands of parameters across the entire graph to minimize the difference between the generated output and the target professional mix.
Iterative Pruning
To move from a dense, unrealistic console to a sparse, human-like mixing structure, the system employs iterative pruning—a technique borrowed from neural network optimization. The "dry/wet" parameter of each effect serves as a proxy for its importance. If the optimizer reduces the "wet" signal of a processor to near zero, that processor is deemed negligible. The system iteratively removes these components and fine-tunes the remaining ones to maintain sonic integrity.
Computational Efficiency
Processing complex graphs with over 100 tracks is computationally expensive. To solve this, the authors developed a custom scheduling algorithm for batch processing. This allows the GPU to compute multiple independent nodes of the mixing graph in parallel, significantly reducing the time required for optimization.
Experimental Setup
Datasets
The study utilized a robust dataset of 1,129 songs from three primary sources:
- MedleyDB & MixingSecrets: Standard public multitrack datasets for academic evaluation.
- Internal Dataset: A private collection of 579 songs crowdsourced from professional engineers, ensuring the model is tested against high-quality Western music production standards.
Evaluation Metrics
The primary objective metric used was the Multi-Resolution Short-Time Fourier Transform (MRSTFT) loss. The researchers found that MRSTFT correlated most strongly with human subjective listening tests compared to other metrics like FAD or SI-SDR.
Results and Discussion
Sparsity and the "67% Rule"
One of the most significant findings was that approximately 67% of the processors in the initial dense console could be pruned without a perceptible loss in quality. This indicates that professional mixes are often much sparser than a "maximalist" approach would suggest, and the pruning algorithm effectively identifies the essential components of a mix.
Perceptual Transparency
In MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) subjective listening tests, participants were often unable to distinguish between the original full console and the pruned versions (using a tolerance of $\tau=0.01$). This confirms that the iterative pruning successfully preserves the artistic intent of the mix.
Consistency and Stochasticity
When tested against known synthetic graphs, the system achieved a high accuracy of 0.902 in processor detection. Interestingly, the pruning process exhibited stochastic behavior; different runs on the same song could yield different graph structures that were perceptually identical. This mirrors the real-world reality where different engineering paths can lead to the same sonic result.
Limitations and Future Work
Despite its success, the model has limitations:
- Processor Variety: The current suite lacks non-linear effects such as saturation, distortion, or pitch correction.
- Static Nature: The model assumes parameters are static, failing to account for "automation" (changes in faders or EQ over time).
- Phase Issues: Current loss functions can occasionally lead to temporal "smearing" in time-based effects like reverb.
- Topology: The search is currently constrained to pruning a pre-defined tree. Future iterations may utilize Neural Architecture Search (NAS) to allow the AI to propose entirely original routing and side-chaining paths.
Conclusion
This research provides a powerful framework for the automated analysis and reconstruction of music mixes. By combining differentiable signal processing with iterative pruning, the system can efficiently navigate the complex search space of mixing graphs, providing a foundation for future AI-driven mixing tools and large-scale musicological studies.
Summary updated