Federated Learning (FL) is often touted for its privacy-preserving nature, allowing multiple parties to train a shared global model without ever exchanging raw data. However, a new research paper titled "Model Hijacking Attack in Federated Learning" reveals a sophisticated vulnerability: an adversary can "hijack" a global model to perform an entirely different task—like recognizing handwritten digits instead of traffic signs—without the server or other clients ever noticing.
TL;DR
Researchers have introduced HijackFL, the first model hijacking attack tailored for Federated Learning. Unlike traditional attacks that poison model parameters, HijackFL uses pixel-level "cloaks" to trick a global model into executing a hidden task, achieving over 90% success rates while maintaining the model's original performance and stealth.
What Problem Does This Paper Address?
In a Model Hijacking Attack, an adversary wants to repurpose a machine learning model for a "hijacking task" different from its intended "original task." This poses two major risks:
- Accountability: A model owner could be unknowingly providing illegal or unethical services (e.g., a face recognition model hijacked for unauthorized surveillance).
- Parasitic Computing: The adversary uses the model owner's computational resources and maintenance for free.
Previous hijacking research focused on centralized learning, where an attacker could simply poison the training data. In Federated Learning, this is much harder. Because the central server averages updates from many clients, a single malicious client's poisoned update is usually "washed out" or easily detected by its negative impact on the model's accuracy. The authors set out to answer: Can you hijack a collaboratively trained model without modifying its parameters at all?
Methods
The authors propose HijackFL, which shifts the attack from the training phase to the input phase. Instead of changing the model, they change the data.
The attack follows a four-stage pipeline:
- Greedy-based Class Mapping: The adversary determines which classes in the hijacking task (e.g., MNIST digits 0-9) should map to which classes in the original task (e.g., CIFAR-10 categories like "bird" or "car"). They use a greedy algorithm to find the most "natural" fit based on the local model's existing feature perceptions.
- Anchor Features Computing: The attacker generates "anchor features"—idealized feature vectors that represent the original classes with near 100% confidence.
- Cloaks Computing: This is the core innovation. The attacker optimizes pixel-level perturbations, called "cloaks," for each hijacking class. When a cloak is added to a hijacking sample, it "aligns" that sample with an original class in the model's feature space.
- Execution: When the adversary wants to use the hijacked model, they apply the optimized cloak to their input. The global model, seeing features it recognizes, classifies the "cloaked" image into an original category, which the adversary then decodes back into the hijacking task's result.
Crucially, because the adversary submits benign updates during training, the global model remains "clean," making the attack invisible to standard defense mechanisms.
Datasets
The researchers evaluated HijackFL using four benchmark computer vision datasets in various "Original Task $\rightarrow$ Hijacking Task" combinations:
- CIFAR-10 $\rightarrow$ MNIST (10 classes $\rightarrow$ 9 classes)
- CIFAR-10 $\rightarrow$ GTSRB (Traffic signs)
- TinyImageNet-100 $\rightarrow$ MNIST
- TinyImageNet-100 $\rightarrow$ GTSRB
They tested these across three popular architectures: ResNet-18, MobileNet-V2, and ShuffleNet-V2.
Key Insights and Ideas
- Feature Space Alignment: The "aha moment" is realizing that you don't need to change the decision boundaries if you can move the data points. By finding the right "cloak," any image can be made to look like a "bird" or a "car" to the model's internal layers.
- Transferability is the Key: The attack works because the adversary's local model (trained on the global parameters) is similar enough to the final global model. The cloaks optimized locally "transfer" effectively to the global model.
- The "Negative Anchor" Trick: To make the attack more robust, the authors use a "negative anchor feature." They map all hijacking samples without the correct cloak to a specific "negative" class. This ensures that only the intended cloak produces a high-confidence prediction, reducing noise and decoding errors.
- Stealth through Compliance: Because HijackFL doesn't require malicious model updates, it bypasses the "utility fluctuation" that usually triggers alarms in FL servers.
Key Results
- Superior Performance: On ResNet-18 (CIFAR-10 to MNIST), HijackFL achieved an Attack Success Rate (ASR) of 92.75%. In contrast, traditional data poisoning baselines struggled to reach even 10% in the FL environment.
- Zero Utility Impact: Unlike poisoning attacks, which often degrade the model's performance on its original task, HijackFL had 0% impact on the global model's original utility.
- Resilience: The attack remains effective even as the global model undergoes more rounds of federated training, whereas poisoning effects tend to diminish over time.
Limitations and Future Work
- Class Count Constraint: Currently, the hijacking task cannot have more classes than the original task (minus one for the negative anchor). For example, you can't hijack a 10-class model to perform a 100-class task.
- Visual Artifacts: The "cloaks" can sometimes introduce noticeable visual differences to the images, which might be detectable by human inspection or specialized input filters.
- Defense Sensitivity: While HijackFL bypasses training-time defenses, the authors found that Feature Squeezing (reducing the color bit depth of inputs) can mitigate the attack, though often at the cost of the model's original accuracy.
The authors suggest that future work should explore more effective relabeling strategies to overcome class constraints and look into making cloaks even more visually subtle.
Summary updated