Open-Unmix - A Reference Implementation for Music Source Separation

Fabian-Robert Stöter, Stefan Uhlich, Antoine Liutkus, Yuki MitsufujiJ. Open Source Softw 2019 · DOI · DBLP

Introduction

Open-Unmix (UMX) is the first open-source, state-of-the-art reference implementation for music source separation based on deep learning. By providing a high-performance, modular baseline in PyTorch, the authors aim to solve the "reproducibility crisis" in audio research and provide an accessible starting point for both researchers and artists.

The Challenge of Music Source Separation

Music source separation—the process of isolating individual "stems" (vocals, drums, bass) from a mixed track—is a mathematically "ill-posed" problem because there is no single closed-form solution for extracting multiple sources from a stereo signal.

While deep learning has recently pushed the field toward high-quality results, a significant hurdle remained: the lack of a common, high-performing open-source baseline. Most state-of-the-art (SOTA) systems were closed-source, forcing researchers to compare their new methods against their own (often weaker) internal baselines. This made it nearly impossible to track absolute progress in the field or for newcomers to enter the space without reinventing the wheel.

Methodology and Design Philosophy

Open-Unmix is built on a bi-directional Long Short-Term Memory (BLSTM) network architecture. Rather than creating a monolithic, complex package, the authors focused on a "MNIST-like" design philosophy:

  • Modular Architecture: The code isolates pre-processing, data loading, model definition, and post-processing. This allows researchers to swap out the model architecture while keeping the rest of the pipeline constant.
  • Framework Accessibility: While the reference implementation is in PyTorch, the authors provided ports to NNabla and planned support for TensorFlow, ensuring the core logic is accessible across different deep learning ecosystems.
  • Signal Processing Integration: The system bridges the gap between "classical" signal processing and modern machine learning, utilizing verified procedures for audio handling that have proven effective in literature.

Data and Benchmarking

The paper relies heavily on the MUSDB18 dataset (and its high-quality successor, MUSDB18-HQ). For decades, source separation was hindered by copyright issues; researchers couldn't share the "ground truth" stems of commercial music. MUSDB18 consists of 150 full-length tracks (~10 hours of music) with isolated stems released under Creative Commons licenses. While small compared to speech recognition datasets, it is currently the largest freely available dataset for music separation.

Results and Performance

The primary result is that Open-Unmix achieves SOTA performance comparable to the best-known systems in the field. In evaluations against entries from the 2018 Signal Separation Evaluation Campaign (SiSEC), UMX performed on par with TAK1, the top-performing system at the time.

Performance comparison of Open-Unmix (UMX) against other state-of-the-art methods in terms of Signal-to-Distortion Ratio (SDR) across four audio stems.

As shown in the figure above, there was no statistically significant difference between UMX and the best closed-source models, effectively making UMX the new "gold standard" for open-source separation. Unlike previous open-source tools which focused on classical methods or older neural architectures, UMX provides the performance levels required for professional-grade audio restoration and remixing.

Conclusion and Future Work

Open-Unmix is intended to be a baseline, not the final word in separation technology. The project does not aim to constantly chase the absolute highest leaderboard scores itself. Instead, it provides a stable foundation for other researchers to build upon. The authors encourage the community to use UMX to experiment with new input representations (beyond standard spectrograms) and more advanced neural architectures (like Transformers or CNN-based models) while using the UMX pipeline for fair comparison.

Summary updated

BibTeX
@article{stoter2019open,
  author = {Fabian-Robert Stöter and Stefan Uhlich and Antoine Liutkus and Yuki Mitsufuji},
  title = {{Open-Unmix - A Reference Implementation for Music Source Separation}},
  journal = {J. Open Source Softw.},
  year = {2019},
  doi = {10.21105/joss.01667},
}
Figure