TL;DR
This paper introduces Vamsa, a novel system for automatically tracking which columns in a dataset are used to train machine learning models directly from Python scripts. This is crucial for debugging, ensuring fairness, and meeting regulatory compliance in ML applications, a task that is currently manual and error-prone.
What Problem Does This Paper Address?
As machine learning models become increasingly integrated into critical domains like finance and healthcare, understanding their behavior, ensuring fairness, and debugging them effectively are paramount. A key challenge is knowing precisely what data went into training a specific model. This "data provenance" is essential for answering questions like: "Which columns from the original dataset were used to derive the features and labels for this model?" Currently, obtaining this information requires manual inspection of complex data science scripts, which is time-consuming, error-prone, and doesn't scale. Existing ML pipeline management tools often track the overall pipeline but lack the granular detail of column-level data-to-model relationships.
Methods
Vamsa tackles this problem using a three-module architecture:
- Derivation Extractor: This module performs static analysis on Python scripts to build a "workflow intermediate representation" (WIR). This WIR captures the dependencies between variables, functions, and operations within the script, essentially creating a program dependency graph.
- KB-Based Annotator: This is the core of Vamsa's intelligence. It annotates the variables in the WIR with semantic information (e.g., "this variable is a model," "this variable is a feature set"). It achieves this by querying an external, pluggable Knowledge Base (KB) that stores information about ML library APIs (like scikit-learn, CatBoost, LightGBM). This KB-driven approach allows Vamsa to be agnostic to specific ML libraries and extensible to new ones.
- Provenance Tracker: This module uses the annotated WIR to pinpoint the exact columns used for features and labels. It intelligently traverses the WIR, guided by rules from the KB, to identify operations like
dropor slicing that explicitly include or exclude columns.
A key design principle is that Vamsa works on unmodified Python scripts, requiring no user input or code changes.
Datasets
The authors evaluated Vamsa on two large datasets of Python scripts:
- Kaggle Dataset: 1.2K scripts downloaded from Kaggle.
- NTBK Dataset: 24.8K scripts from a corpus of publicly available Python notebooks crawled from GitHub.
For accuracy evaluation, they randomly selected 150 scripts from each dataset for which they manually extracted the ground truth provenance information.
Key Insights and Ideas
- Static Analysis is Sufficient (Mostly): The paper demonstrates that for a significant portion of data science scripts, static analysis of the code is enough to automatically extract valuable ML provenance, avoiding the need for runtime execution or intrusive instrumentation.
- Knowledge Base is Key to Extensibility: The reliance on an external KB for API semantics is a crucial design choice. It allows Vamsa to adapt to the rapidly evolving ML landscape without code modifications, making it future-proof.
- Column-Level Provenance is Highly Valuable: The real-world use case highlights how precise column-level provenance can quickly pinpoint the source of data corruption issues, saving significant debugging time and effort across large organizations.
dropoperations are easier to track than complex transformations: The results suggest that explicit column exclusion (e.g., usingdrop) is detected with higher precision and recall than column inclusion, likely due to the more direct nature of these operations.
Key Results
Vamsa demonstrates impressive performance:
- Accuracy: Achieves high precision and recall, ranging from 90.4% to 99.1% across various tasks (feature/label exclusion/inclusion, model/dataset identification) on manually labeled datasets.
- Coverage: The Derivation Extractor covers a high percentage of scripts, with limitations primarily due to unhandled Python grammar constructs (which are noted as implementation issues, not design flaws). The KB-based Annotator and Provenance Tracker also show good coverage, with failures often attributable to commented-out code, unusual import patterns, or the need for KB expansion.
- Efficiency: The system is highly efficient, with component latencies in the order of milliseconds for average-sized scripts. Even for larger scripts (up to 900 lines of code), the end-to-end latency remains manageable.
Limitations and Future Work
- Undecidability: The authors acknowledge that static analysis has inherent limitations with control flow constructs like loops and conditionals, making some provenance tracking undecidable. They suggest future work to incorporate runtime information for these cases.
- KB Population: While the KB-driven approach is flexible, manual population of the KB is currently required. Automating this process is a key direction for future work.
- Finer-Grained Provenance: The paper suggests exploring finer-grained provenance, such as identifying specific partitions or subsets of data used for training, which could further enhance upstream applications.
- Handling Complex Data Transformations: The current system sometimes struggles with complex data transformations that involve merging and splitting data, leading to occasional misidentification of training datasets.
Summary updated