Hidden Technical Debt in Machine Learning Systems

D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, Dan DennisonNeurIPS 2015

Introduction

Building and deploying a machine learning (ML) model is often fast and relatively cheap, but maintaining it over time is notoriously difficult and expensive. This paper argues that ML systems possess a unique capacity for incurring "technical debt." In these systems, short-term wins in model accuracy frequently lead to massive, long-term maintenance costs due to system-level complexities and hidden dependencies.

In traditional software engineering, "technical debt" refers to the long-term cost of choosing an easy, short-term solution over a better, more robust approach that would take longer to implement. While traditional code-level debt can be managed through refactoring and unit tests, ML systems introduce a new, more insidious category of debt.

The ML Paradox and System-Level Debt

The authors address the "ML Paradox": why is it so easy to create a powerful prediction system, yet so hard to keep it running reliably? The core challenge is that ML systems are driven by data, which inherently erodes the clean abstraction boundaries that software engineers usually rely on to keep systems modular and maintainable.

Rather than presenting a new algorithm or relying on a specific experimental dataset, this paper provides a qualitative framework and a taxonomy of risk factors based on broad industry observations, particularly from the authors' experiences at Google. They apply software engineering principles to the specific architectural patterns of ML, identifying several "debt-inducing" categories:

  • Boundary Erosion: ML models inherently mix signals, making it nearly impossible to change one part of the system without affecting everything else.
  • Data Dependencies: Relying on external, evolving data streams is significantly more dangerous and unpredictable than relying on static code libraries.
  • Configuration Debt: The complexity of the settings that govern how models are trained and deployed is often overlooked but highly fragile.

The "Tiny Box" Reality and Key Insights

One of the most famous insights from this paper is the realization that the actual ML code is a minuscule fraction of a real-world system.

Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surrounding infrastructure is vast and complex.

The vast majority of the "plumbing"—data collection, verification, configuration, serving infrastructure, and monitoring—is where the technical debt actually accumulates.

Compounding this infrastructure bloat is the CACE Principle: "Changing Anything Changes Everything." Because ML models are highly entangled, you cannot tweak a single hyper-parameter, update a data pipeline, or add a single feature without potentially shifting the entire distribution of the model's behavior. This entanglement makes isolated, modular improvements nearly impossible.

Anti-Patterns in ML Systems

The paper identifies several common architectural "smells" or anti-patterns that plague ML deployments:

  • Glue Code & Pipeline Jungles: Using generic, open-source ML packages often requires massive amounts of "glue code" to format data. Similarly, data preparation often evolves into a "jungle" of scrapes, joins, and transformations that is impossible to test or reproduce reliably.
  • Correction Cascades: When a model designed for Problem A is slightly off when applied to Problem B, engineers often learn a "correction" model on top of it. This creates a brittle chain where improving the base model can actually break the downstream corrections.
  • Undeclared Consumers: Because ML outputs are often written to logs or files, other systems might start using those predictions as inputs without the original developers knowing. This creates "visibility debt," where you cannot safely update your model because you do not know who is relying on its exact current outputs.

Conclusions and Moving Forward

The paper successfully shifted the industry conversation from "how do we make models more accurate?" to "how do we make ML systems sustainable?" The authors conclude that while ML code is the easy part, the surrounding infrastructure is the hard part. Furthermore, technical debt in ML is often hidden at the system level, making it invisible to standard static analysis tools.

While the authors acknowledge they do not provide a "silver bullet" or a strict metric for measuring this debt (as ML lacks equivalents to software engineering metrics like cyclomatic complexity), they emphasize that paying down debt requires a cultural shift. Teams must value "boring" tasks—like deleting unused features, tightening APIs, and improving documentation—just as much as they value increasing AUC or accuracy.

Future work in this domain must focus on developing better abstractions for ML, creating automated tools for the static analysis of data dependencies, and fostering a team culture that actively rewards the reduction of system complexity.

Summary updated

BibTeX
@inproceedings{sculley2015hidden,
  author = {D. Sculley and Gary Holt and Daniel Golovin and Eugene Davydov and Todd Phillips and Dietmar Ebner and Vinay Chaudhary and Michael Young and Jean-François Crespo and Dan Dennison},
  title = {{Hidden Technical Debt in Machine Learning Systems}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2015},
}

Loading related content...

Figure