Introduction
Data scientists often spend the majority of their time on "data janitoring"—the tedious process of cleaning and preparing messy datasets. Real-world machine learning (ML) pipelines frequently ingest data from disparate sources, resulting in missing values, inconsistent formatting, and outliers. These "domain value violations" can introduce significant bias, leading to poor model performance or even erroneous predictions.
Traditionally, cleaning is a manual process involving custom scripts. This is not only time-consuming but also risky; naive cleaning operations, such as imputing averages or deleting records, can inadvertently destroy useful signals or introduce new biases. BoostClean addresses this by automating the process and ensuring that cleaning operations demonstrably improve the final model's predictive accuracy.
Methodology: The Boost-and-Clean Algorithm
BoostClean frames data cleaning as a statistical boosting problem. Its core innovation is the Boost-and-Clean algorithm, which treats cleaning operations as "weak learners" in an ensemble.
Cleaning as Boosting
The system defines a "conditional cleaning script" as a pair consisting of an error detector and a repair function. In each iteration, BoostClean searches an extensible library to find the cleaning operation that yields the greatest improvement in model accuracy on a validation set. Following the logic of AdaBoost, the system then re-weights the data to focus on records that remain difficult to predict, guiding subsequent cleaning rounds.
Detector and Repair Libraries
BoostClean utilizes a pre-populated library of detectors and repairs:
- Detectors: These identify issues such as null values, type mismatches, or statistical outliers.
- Repairs: These apply fixes like mean/mode imputation or record discarding.
- IsoDetect: To eliminate manual threshold tuning, the authors developed
IsoDetect, which employs Isolation Forests to automatically identify outliers in feature vectors. - Word2Vec for Categorical Data: A novel contribution uses Word2Vec to detect errors in categorical data. By treating records as documents and attribute values as words, the system learns typical co-occurrence patterns. Values appearing in unlikely contexts are flagged as potential errors.
Operational Flow
BoostClean distinguishes between cleaning performed during offline training and repairs required during online inference. This ensures that the model remains robust to "dirty" data encountered in production environments.
Evaluation and Results
The authors evaluated BoostClean across 12 diverse datasets, including eight binary classification tasks from Kaggle and UCI, two data analytics datasets, and two proprietary production datasets from a lead-scoring company.
Key Findings
- Accuracy Improvements: BoostClean improved absolute prediction accuracy by up to 9% over non-ensembled baselines. On proprietary datasets characterized by high class imbalance, it improved the Area Under the Curve (AUC) by 13-14%.
- Detection Quality: The automated detector library achieved 81% accuracy in identifying errors, rivaling labor-intensive manual rules.
- Computational Performance: Through optimizations such as parallelization and prediction materialization, the system achieved a 22.2x end-to-end speedup on a 16-core machine.
Discussion
A fundamental insight of BoostClean is its accuracy-centric approach. Rather than aiming for "perfect" data in isolation, it prioritizes "useful" data for the specific model. By leveraging clean labels—which are often more reliable than input features—as ground truth, BoostClean builds a sequence of complementary cleaners.
Limitations and Future Work
Currently, BoostClean relies on the availability of clean labels for validation and focuses primarily on single-cell "domain value violations." It does not yet address more complex issues like entity resolution or functional dependency violations. Future research may expand the library to include parameterized cleaning (e.g., regex-based extractors) and explore scalability in the context of unstructured or evolving "data lakes."
Summary updated