Abstract
Modern neural networks often contain significantly more parameters than the size of their training data. We show that this excess capacity provides an opportunity for embedding secret machine learning models within a trained neural network. Our novel framework hides the existence of a secret neural network with arbitrary desired functionality within a carrier network. We prove theoretically that the secret network's detection is computationally infeasible and demonstrate empirically that the carrier network does not compromise the secret network's disguise. Our paper introduces a previously unknown steganographic technique that can be exploited by adversaries if left unchecked.
Introduction
Modern deep learning models are massive, often containing millions or billions of parameters—far more than is strictly necessary to represent the data they are trained on. This "excess capacity" is usually seen as a buffer for generalization, but it also presents a significant security vulnerability. This research introduces TrojanNet, a framework for model-to-model steganography. The goal is to embed a secret, high-performing neural network inside another one such that an inspector cannot find it without a secret key, even if they have full "white-box" access to the model's weights.
The TrojanNet Framework
The core mechanism of TrojanNet relies on parameter permutation. A sender and receiver share a secret key $k$, which acts as a seed for a pseudo-random permutation generator. This generator produces a specific shuffling $(\pi)$ of the network's weight tensors.
The network is trained on two tasks simultaneously using a joint loss function:
- The Public Task uses the weights in their standard order.
- The Secret Task uses the weights in the shuffled order dictated by the key.
To use the secret model, the receiver applies the secret permutation to the weights of the public model. Without the key, the weights appear as a random, non-functional mess for the secret task. To ensure the method is robust, the authors use cryptographic hashes (like SHA-256) to ensure that even similar keys produce wildly different permutations. They also address architectural "tells" like Batch Normalization by using Group Normalization, which doesn't require storing task-specific statistics that might tip off an inspector.
Theoretical Security
The authors prove mathematically that the "EXISTS-PERM" problem—determining if there exists any permutation of a model's weights that performs a specific task—is NP-complete. This means that even if an inspector knows exactly what secret task to look for, they cannot computationally prove a hidden model exists without the key. Surprisingly, training a model on two tasks via permutations doesn't result in "feature sharing" like typical multi-task learning; because the weights are shuffled, the two tasks are effectively de-correlated.
Empirical Evaluation
The researchers evaluated TrojanNet using four standard image classification benchmarks: CIFAR-10, CIFAR-100, SVHN, and GTSRB. They tested various combinations, such as hiding a GTSRB classifier inside a CIFAR-10 carrier, and even successfully hid a regression task inside a classification model.
As shown in the experimental results, TrojanNet achieved virtually the same accuracy as two separate models trained independently. For example, a CIFAR-10 carrier maintained ~94.4% accuracy while the hidden SVHN task reached ~97.9%. Furthermore, the larger the carrier model (e.g., ResNet-50 vs. ResNet-18), the more secret tasks it can hold. The authors successfully embedded up to 10 different secret tasks into a single ResNet-50 with only a ~1.5% drop in accuracy.
Conclusion and Future Work
TrojanNet demonstrates that a secret model's "footprint" can be reduced to just the few hundred bits of a key, with zero performance trade-offs in pairwise tests. However, limitations remain, such as output size constraints and the need to avoid standard Batch Normalization. Since detection is proven to be NP-complete, future work must focus on proactive defenses—perhaps training techniques that make it impossible to embed these permutations in the first place.
Summary updated