Abstract
The "Slice-and-Pack" (S&P) framework enables the creation of customized deep learning models by extracting specific functional fragments from multiple pre-trained models and reassembling them. This approach addresses the inefficiencies of the "learnware" paradigm, where users often face redundancy and high computational costs when trying to meet specific needs with off-the-shelf models. S&P significantly reduces hardware requirements and expands model variety in sharing markets, maintaining high accuracy even with minimal training data.
Introduction
In the emerging "learnware" ecosystem, developers contribute pre-trained models to a central market for user reuse. However, a significant challenge persists: real-world requirements are rarely satisfied by a single, monolithic model. Consider a community clinic requiring a diagnostic tool for common ailments. The market might offer high-end specialist models for specific organ systems (e.g., skin, stomach, infections), but not a single integrated model for "common diseases."
Currently, a user must download and execute multiple massive specialist models simultaneously. This leads to three primary issues:
- Redundancy: Users compute features for rare diseases they do not need.
- Resource Intensity: Running multiple large models necessitates expensive, high-end hardware.
- Privacy and Data Constraints: Markets often lack access to original training data, hindering standard transfer learning or model merging techniques.
The Slice-and-Pack Framework
The authors propose a two-stage framework designed to operate under data-limited constraints, requiring only a few samples per class.
1. Slicing: Extracting Abilities
The first stage transforms monolithic models into "fragments," each representing a specific capability, such as identifying a particular class.
- Ability Pooling: By applying $\ell_{2,1}$-norm regularization to layer activations, the framework identifies the specific neurons or filters most critical for a given class. These essential units are then "sliced" out.
- Ability Adaptation: Slicing inherently creates structural gaps and potential "ability collapse" (the loss of general, low-level features). To mitigate this, the authors introduce "repair layers" to manage size mismatches and fine-tune fragments to ensure they retain shared patterns across classes.
2. Packing: Assembling Custom Models
Once a library of fragments is established, they must be integrated to meet user specifications.
- Encoder-Decoder Mechanism: S&P employs a lightweight fusion layer (Encoder) to combine features from disparate fragments.
- Reconstruction Loss: To prevent information loss during fusion, Decoders verify that original fragment outputs can be reconstructed from the combined representation.
- Plug-and-Play Architecture: Fragments remain frozen during the packing process; only the small fusion and classifier layers are trained. This modularity allows for the rapid addition or removal of capabilities as user needs evolve.
Experimental Evaluation
The framework was rigorously tested on image (CIFAR-10, CIFAR-100) and text (TREC, SST-5) classification tasks using VGG16, ResNet34, CNN, and RNN architectures. Seven distinct tasks ($T_1$ to $T_7$) were designed to simulate real-world scenarios, such as combining models from similar or different superclasses.
Key Results and Insights
The "Slice-and-Pack" approach yields several significant advantages over traditional methods:
- Superior Accuracy in Low-Data Regimes: S&P consistently outperformed standard fine-tuning and state-of-the-art knowledge distillation methods (e.g., CA-MKD, NetGraft), particularly when only 5 or 10 samples per class were available.
- Model Compactness: For VGG16-based tasks, S&P produced models with approximately 50% fewer parameters than a standard ensemble, without sacrificing accuracy.
- Market Amplification: The paper introduces the Equivalent Expansion Rate. For a market with 5 models of 5 classes each, S&P can effectively offer between 200,000 and 600,000 custom model combinations, vastly increasing the utility of the shared model pool.
Conclusion and Future Work
By treating deep learning models as modular "Legos" rather than black boxes, Slice-and-Pack offers a scalable solution for customized model deployment. While currently focused on classification, future work aims to extend this slicing methodology to more complex tasks like object detection and to explore cross-architecture fragment integration. S&P represents a significant step toward a more efficient and flexible model-sharing economy.
Summary updated