TL;DR
This paper introduces Pyramid, a novel system that enhances big data protection by selectively managing data access for machine learning workloads. By leveraging count featurization and differential privacy, Pyramid significantly reduces the amount of raw data exposed to potential attacks while maintaining high model accuracy and performance.
What Problem Does This Paper Address?
Organizations are accumulating massive amounts of data, creating a "toxic asset" that is vulnerable to attacks and insider threats. A common practice is to collect data "just in case," leading to vast, accessible data lakes. However, many modern machine learning (ML) applications require periodic retraining, necessitating access to this entire data store, thereby increasing exposure. The challenge is to protect this data effectively without hindering the performance and utility of ML workloads, especially those that rely on large, high-dimensional datasets.
Methods
Pyramid builds upon count featurization, a technique that summarizes historical data by counting feature occurrences with different labels. This drastically reduces dimensionality, making ML training more efficient. Pyramid enhances this by:
- Limited Exposure Data Management: It maintains a small, rolling "hot window" of recent raw data for active use and a "historical statistics store" containing differentially private aggregates (counts).
- Differential Privacy Infusion: Noise is added to the count tables to protect individual observations that are no longer in the hot window but still within a defined retention period.
- Weighted Noise Infusion: To mitigate the impact of differential privacy noise on accuracy, Pyramid distributes the privacy budget unevenly across features, applying less noise to features with lower counts.
- Private Count-Median Sketches: For better accuracy under high noise levels, Pyramid uses count-median sketches instead of standard count-min sketches, which are prone to bias.
- Count Selection: An automatic process identifies potentially useful feature combinations to count, reducing the need to access raw historical data for evolving workloads.
- Data Retention Policies: Pyramid supports data expiration by managing count tables in windows and retraining models to forget old data.
Datasets
The paper evaluates Pyramid on three real-world applications using the following datasets:
- Criteo: Used for ad targeting classification. Two versions were used: a full dataset with 1.2 billion observations and a Kaggle version with 45 million observations.
- MovieLens: Used for movie recommendation (classification and regression).
- MSN.com: Used for news personalization (regression).
Key Insights and Ideas
- Leveraging ML Techniques for Security: The core idea is to adapt ML techniques like training set minimization (count featurization) not just for performance, but as a mechanism for data protection.
- "Just-in-Case" Data is the Real Risk: The paper highlights that the most significant security risk comes from data collected for speculative future use, which is often stored in large, accessible data lakes.
- Count Featurization as a Privacy Enabler: By summarizing data into counts, Pyramid creates a more compact and manageable representation that is amenable to privacy-preserving techniques.
- Differential Privacy is Scalable with Smart Techniques: While differential privacy can be noisy, techniques like weighted noise infusion and private count-median sketches significantly reduce its accuracy impact.
- Workload Evolution is Key: Pyramid's ability to support model changes and feature additions without accessing raw historical data is crucial for practical adoption.
Key Results
- Training Set Reduction: Pyramid allows models to train effectively on less than 1% of the raw data, converging much faster than models trained on full datasets.
- Accuracy Impact: Protecting individual past observations with differential privacy incurs only a ~1% penalty on accuracy, keeping models within 5% of state-of-the-art baselines.
- Performance Overhead: Pyramid adds a minimal performance overhead, with prediction latency increasing by only about 5% when caching is used.
- Model Compatibility: Pyramid supports various ML models, including linear models, neural networks, and boosted trees, and can even enable complex nonlinear models that would be infeasible with raw, high-dimensional data.
- Weighted Noise Infusion Effectiveness: This technique is critical for maintaining performance, especially on datasets like MovieLens, reducing accuracy loss from 15% to under 5%.
Limitations and Future Work
- Threat Model Scope: Pyramid's threat model assumes that the historical raw data store is protected independently and that attacks are well-defined with a start and end time. It does not protect against multiple intrusions within the same time window or against adversaries who monitor system states before a full break-in.
- Hot Data Exposure: While minimized, the hot data store is still exposed, and the paper acknowledges that even this small amount of data might reveal sensitive information.
- Workload Evolution Limitations: While supporting many workload changes, Pyramid does not fully support adding new features or changing the predicted label unless the new label is a subset of an existing one.
- Future Work: The authors suggest exploring other training set minimization techniques like vector quantization and sampling for broader applicability. They also propose extending the private weighted noise infusion to use multiple windows for more accurate weight estimates and developing a differentially private count selection mechanism. Additionally, they suggest a "Pyramid as-a-Featurization-Service" deployment model.
Summary updated