TL;DR
This position paper outlines the necessity of a new, dedicated research community at the intersection of computer systems and machine learning. To bridge the gap between isolated academic models and complex real-world deployments, the authors introduce the MLSys conference, establishing a roadmap for full-stack research targeting hardware/software co-design, efficiency, privacy, and accessibility.
What Problem Does This Paper Address?
While machine learning (ML) models have achieved historic breakthroughs on curated benchmarks, deploying them in production remains a massive bottleneck. The authors argue that modern ML represents a radical paradigm shift—often called "Software 2.0"—where developers spend more time collecting, labeling, and reshaping data than writing traditional code.
Traditional ML research focuses almost exclusively on maximizing predictive accuracy. However, real-world deployment introduces severe full-stack bottlenecks that accuracy alone cannot solve:
- Skyrocketing Costs: Compute costs for training large models are immense, and data annotation can cost millions.
- Resource Constraints: Deploying models on edge devices requires strict adherence to latency, power, and memory limits.
- Trust and Safety: Production systems must navigate complex, system-wide concerns like privacy, adversarial robustness, fairness, and interpretability.
Because these challenges span both hardware and software, they cannot be solved by the traditional ML community or the traditional systems community working in isolation.
Methods
As a position paper, this work does not introduce a new algorithm or technical artifact. Instead, it proposes an organizational and intellectual framework to catalyze a new field. The authors define a research agenda partitioned into three core pillars:
- Software Systems for ML: Designing tools to support the entire ML lifecycle, including high-level programming interfaces, data preprocessing pipelines, debugging tools, and production monitoring.
- Hardware Systems for ML: Developing specialized, heterogeneous hardware and distributed architectures tailored to the unique data access patterns and stochastic nature of ML workloads.
- Optimizing for Metrics Beyond Accuracy: Engineering systems that inherently satisfy constraints like power efficiency, federated privacy, cost, and fairness.
Additionally, the paper highlights the reciprocal domain of Machine Learning for Systems—applying ML techniques to replace heuristics and optimize low-level system components like compilers, operating systems, and databases.
To operationalize this, the authors established the MLSys conference, featuring a program committee split evenly between systems and ML experts to ensure true cross-disciplinary peer review and a strong emphasis on artifact evaluation for reproducibility.
Datasets
Not discussed in the paper (as this is a position and community-building white paper, no experimental evaluations or datasets were utilized).
Key Insights and Ideas
- The "Software 2.0" Reality: Programming in the ML era is fundamentally about data curation and pipeline management rather than writing explicit logic. Systems must adapt to treat data, features, and labels as first-class citizens.
- Full-Stack Bottlenecks Require Full-Stack Solutions: Issues like model latency or training cost cannot be solved purely at the algorithmic level; they require co-designing hardware, compilers, and frameworks simultaneously.
- A Two-Way Street: The relationship between systems and ML is symbiotic. We need better systems to train and serve ML, but we also need ML to build smarter, self-tuning computer systems.
Key Results
The primary result of this paper is the successful institutionalization of the MLSys community. Originally emerging from successful satellite workshops at venues like NeurIPS, ICML, OSDI, and SIGMOD, the standalone MLSys conference was formally established in 2018. The paper outlines its structural success, noting a highly selective review process and the widespread adoption of modern artifact evaluation to spur rapid, reproducible progress in the field.
Limitations and Future Work
The authors frame the entire paper as an open-ended invitation to tackle future challenges. They explicitly note that the community must eventually look beyond single-model pipelines and address larger, systemic issues:
- Ecosystem Management: How do we manage, maintain, and evaluate entire ecosystems of models that interact with each other in complex, unpredictable ways?
- Long-Term Dynamics: How do we monitor and safely maintain systems designed to pursue long-term, evolving goals?
- Societal Scale: How can we facilitate the sharing and reuse of data and models at a societal scale while navigating strict economic, legal, and privacy boundaries?
Summary updated