TL;DR
This paper introduces a novel method for forensic investigators to determine the data domain of black-box machine learning models. By leveraging a hierarchical corpus like ImageNet, the approach identifies a "meaningful distribution" of data that closely resembles the model's original training data, significantly improving the effectiveness of subsequent forensic investigations like model cloning.
What Problem Does This Paper Address?
Imagine a forensic investigator seizes a device containing a mysterious neural network model. This "black-box" model's inner workings are hidden, and its purpose is unknown. While existing techniques like membership inference or model inversion can reveal some information, they typically require a crucial piece of prior knowledge: the model's data domain (e.g., "it was trained on faces" or "it classifies traffic signs"). Without this, these methods are often ineffective. This paper tackles the challenge of identifying this unknown data domain for black-box models, aiming to find a representative dataset that can unlock further forensic insights.
Methods
The core of the proposed method lies in an objective function designed to select a subset of data from a comprehensive, hierarchical corpus (like ImageNet) that best represents the target model's original training distribution. This objective function has two key components:
- Functional Requirement: This measures how well the selected data elicits consistent and predictable responses from the black-box model. It assesses if the model correctly classifies samples from the selected distribution and if it exhibits robustness to minor transformations (like noise or rotation), indicating a well-trained model.
- Semantic Requirement: This leverages the metadata of the hierarchical corpus (e.g., class labels and their relationships) to ensure the selected data represents a semantically meaningful concept. It aims to filter out arbitrary collections of data that might satisfy the functional requirement but lack real-world coherence, promoting "common sense" in the chosen distribution.
To efficiently search the vast space of possible data subsets within the corpus, the authors propose a heuristic algorithm. This algorithm involves computing functional scores for individual data classes, assessing pairwise semantic closeness between potential candidate datasets, and then using clustering and filtering techniques to arrive at the final selected distribution.
Datasets
- Corpus: ImageNet is used as the primary hierarchical corpus.
- Training/Target Models: CIFAR-10 and CIFAR-100 are used to train target classification models for investigation.
- Evaluation Datasets: Caltech101 and Oxford Flowers are used as alternative datasets for comparison in model cloning tasks.
- Field Experiment Datasets: Two undocumented models from the Hugging Face Model Hub are investigated.
Key Insights and Ideas
- Data Domain is Foundational: The paper strongly emphasizes that identifying the data domain of a machine learning model is a critical prerequisite for any meaningful forensic investigation.
- Hierarchical Corpora are Powerful: The structured nature of datasets like ImageNet, with their rich metadata and hierarchical relationships, is instrumental in guiding the search for semantically meaningful data distributions.
- Bridging Functional and Semantic Gaps: The dual objective function effectively balances the model's observed behavior (functional) with the inherent meaning of the data (semantic), leading to more robust and relevant findings.
- Beyond Exact Matches: The goal isn't to find the exact original training set, but a meaningful distribution that enables effective follow-up investigations. This pragmatic approach is key to its applicability.
Key Results
- Improved Model Cloning: When using datasets selected by the proposed method for model cloning (a technique to replicate a target model's functionality), accuracy improved significantly. For a CIFAR-10 trained model, cloning accuracy rose from 45.5% (using Caltech101) to 72.0% (using the proposed method's selected dataset).
- Effective Domain Identification: In controlled experiments, the method accurately identified datasets that were either the exact same class or very similar to the target model's training classes.
- Field Application Success: The method successfully investigated two undocumented models from Hugging Face, suspecting one was trained on ImageNet and the other on plant/leaf data, demonstrating its utility on real-world, unknown models.
Limitations and Future Work
- Hierarchical Corpus Dependency: The method's effectiveness relies on the presence of similar distributions within the provided hierarchical corpus. If no such distribution exists, the method might return a "not found" result.
- Corpus Construction: While ImageNet is a good starting point, the authors suggest that constructing more comprehensive hierarchical corpora, potentially incorporating domain-specific datasets or leveraging generative models for data augmentation, is a promising avenue.
- Generative Model Integration: The paper explores generating new data samples (e.g., X-ray images, specific leaf types) to expand corpora, acknowledging that while this can be powerful, it might require additional constraints to ensure semantic meaningfulness.
- Broader Data Types: The authors propose extending the approach to other data types like text and graphs, as the core objective function and search method are data-agnostic.
- Attack Potential and Countermeasures: The paper acknowledges that the method could be misused for attacks and suggests exploring countermeasures, such as planting "backdoors" to confuse investigators.
Summary updated