Introduction
As deep learning models grow in scale and value, they become critical intellectual property and potential targets for theft. Ensuring that no single entity can unilaterally deploy or modify a high-value model is the goal of shared model governance. While cryptographic solutions like Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC) offer strong security, their extreme computational overhead (often 24x to 1000x) makes them impractical for training modern large-scale networks. This paper explores model splitting as a lightweight, scalable alternative.
The Model Splitting Framework
Model splitting involves partitioning a neural network's parameters across multiple parties. For instance, Alice might hold the initial layers, while Bob holds the remainder. Executing the model requires both parties to collaborate on forward and backward passes.
To evaluate the security of this approach, the authors define the Model Completion problem: how much effort is required for an adversary with access to the training data and a subset of parameters to recover the original model's performance? This is quantified by MC-hardness($\alpha$), the ratio of the cost to retrain missing components (to reach performance level $\alpha$) relative to the original training cost.
Key Insights: Architecture and Domain
The study reveals several counter-intuitive properties of model security:
- Architecture Vulnerabilities: Residual networks (like ResNet50) are surprisingly "leaky." Skip connections allow information to bypass layers, making it easier for the network to compensate for missing parameters compared to sequential architectures like AlexNet.
- The Difficulty of RL: Reinforcement learning (RL) models are significantly harder to complete than supervised learning (SL) models. In RL, the agent's ability to generate useful training data (trajectories) depends on the model's current state. Resetting a layer often breaks the agent's exploration strategy, creating a "chicken-and-egg" problem for recovery.
- Location over Size: The number of missing parameters is less important than their position. Early convolutional layers, which extract foundational features, are often much harder to recover than large fully connected layers at the end of the pipeline.
Quantitative Results
The authors establish a "40% Rule": to recover 100% of the original performance ($\alpha=1.0$), an attacker typically incurs about 40% of the original training cost. For models costing millions to train, this represents a substantial barrier. However, "good enough" performance is significantly cheaper; reaching 80% performance ($\alpha=0.8$) often costs less than 10% of the original training budget.
Conclusion and Future Directions
While model splitting does not provide the mathematical guarantees of cryptography, it offers a practical deterrent for high-value AI assets. The "hardness" measured here serves as an upper bound, as more sophisticated retraining algorithms might lower the cost of completion. Future research may focus on "governance-aware" architecture search—designing models that are inherently difficult to complete if split—and exploring horizontal splitting techniques within individual layers.
Summary updated