How Potent are Evasion Attacks for Poisoning Federated Learning-Based Signal Classifiers?

Su Wang, Rajeev Sahay, Christopher G. BrintonICC 2023 · OpenReview

TL;DR

This paper demonstrates that federated learning (FL) systems for radio signal classification are vulnerable to potent model poisoning attacks. By subtly perturbing local training data with adversarial evasion techniques, even a single compromised device can significantly degrade the global model's performance, highlighting a critical security blind spot in FL for wireless applications.

What Problem Does This Paper Address?

Federated learning (FL) has emerged as a promising approach for training machine learning models, particularly in scenarios like radio signal classification where data privacy is paramount. In FL, devices train models locally on their own data and only share model updates (parameters) with a central server, which then aggregates them into a global model. This design avoids transmitting raw data, offering a privacy advantage over traditional centralized training. However, FL systems are not immune to attacks. This paper focuses on a specific threat: model poisoning in the context of FL-based signal classifiers. The challenge is that while FL prevents direct data exfiltration, it can still be undermined by malicious participants who manipulate their local training process to corrupt the final global model. The authors aim to quantify the potency of such attacks, specifically using adversarial evasion techniques, and understand how they impact signal classification performance.

Methods

The core of the paper introduces a novel attack framework for model poisoning in FL-based signal classifiers. Instead of directly manipulating model weights after training (which can be detected), this framework leverages adversarial evasion attacks during the local training phase.

Here's a breakdown of the key techniques:

  • Adversarial Evasion Perturbation: Adversarial devices craft imperceptible perturbations to their local training data samples. These perturbations are designed to fool the local model into learning a shifted distribution, aiming to degrade the global model's accuracy. The perturbation is constrained by a power budget to remain visually similar to the original signal.
  • Fast Gradient Sign Method (FGSM): The paper uses FGSM, a common technique for generating adversarial examples, to create these perturbations. This method is chosen for its effectiveness in causing misclassifications with minimal changes to the input.
  • Model Poisoning Framework: The attack is integrated into the standard FL training loop. Adversarial devices apply the crafted perturbations to their local datasets before local model training. They then train their models on this poisoned data. The resulting poisoned model parameters are sent to the central server for aggregation.
  • Local Data Perturbation Generation: The adversarial devices aim to satisfy specific conditions: minimize the perturbation's magnitude (to be imperceptible), change the model's prediction on the perturbed sample, and adhere to a power budget.
  • Scaling of Adversarial Weights: To amplify the impact of poisoned updates, adversarial devices can optionally scale their trained model weights before sending them to the server.

Datasets

  • RadioML2016.10a dataset (RML): This is a standard benchmark dataset for automatic modulation classification (AMC). It contains signals from ten different modulation constellations (e.g., 8PSK, BPSK, CPFSK, GFSK) at various Signal-to-Noise Ratios (SNRs). The dataset was split into training (45K samples) and testing (15K samples) sets, with participating clients receiving portions of the training data.

Key Insights and Ideas

  • Potency of Evasion Attacks in FL: The most significant insight is that adversarial evasion attacks, when applied during the local training phase of FL, are surprisingly potent for signal classification. They can cause substantial degradation in the global model's accuracy, even with low-powered perturbations.
  • Imperceptibility is Key: The use of adversarial evasion attacks makes the poisoning difficult to detect. Unlike simple label flipping or injecting noise, these perturbations are designed to be visually indistinguishable from legitimate signals, making them harder to flag by standard FL security mechanisms.
  • Single Adversary Impact: The paper demonstrates that even a single compromised device can significantly impact the global model's performance, challenging the assumption that global averaging in FL inherently mitigates such attacks.
  • Scalability of Degradation: The performance degradation scales with the number of adversarial devices. As more clients participate in poisoning, the global model's accuracy suffers proportionally.
  • Non-IID Data Resilience: While non-independent and identically distributed (non-IID) data distributions across devices can offer some resilience to attacks (as adversaries might not have data for all labels), the proposed evasion attack framework still maintains significant potency.

Key Results

  • Significant Accuracy Degradation: The proposed attack framework, even with a single adversarial device and low-power perturbations, can reduce classification accuracy by over 31% in i.i.d. scenarios and over 17% in non-i.i.d. scenarios at the end of training, outperforming baseline attacks like AWGN and label flipping.
  • Linear Scaling with Adversaries: The accuracy degradation grows linearly with the proportion of adversarial devices. For instance, increasing the proportion of adversaries by 3x leads to a roughly 3x increase in performance degradation.
  • Network Size Impact: Larger FL networks require more adversarial devices to achieve the same level of accuracy penalty. This suggests that while network size can offer some robustness, it doesn't eliminate the threat.
  • Outperforming Baselines: The proposed method consistently outperforms AWGN and label flipping attacks in terms of potency across various scenarios (i.i.d./non-i.i.d., high/low power).

Limitations and Future Work

  • Limitations:
    • The paper focuses on a specific type of adversarial attack (evasion-based model poisoning). Other attack vectors might exist.
  • The study primarily uses a VT-CNN2 architecture. The findings might vary with different model architectures.
  • The paper acknowledges that non-IID scenarios offer some resilience, but the proposed attack still shows significant impact. Further investigation into the interplay between non-IID data and attack strategies could be beneficial.
  • Future Work:
    • Characterizing Large-Scale FL Networks: The authors plan to further investigate the characteristics of large-scale FL networks in the presence of adversarial influence.
  • Developing Defenses: A key future direction is to develop efficient and effective defense mechanisms specifically for FL against these types of adversarial attacks.

Summary updated

Abstract

There has been recent interest in leveraging federated learning (FL) for radio signal classification tasks. In FL, model parameters are periodically communicated from participating devices, training on their own local datasets, to a central server which aggregates them into a global model. While FL has privacy/security advantages due to raw data not leaving the devices, it is still susceptible to several adversarial attacks. In this work, we reveal the susceptibility of FL-based signal classifiers to model poisoning attacks, which compromise the training process despite not observing data transmissions. In this capacity, we develop an attack framework in which compromised FL devices perturb their local datasets using adversarial evasion attacks. As a result, the training process of the global model significantly degrades on in-distribution signals (i.e., signals received over channels with identical distributions at each edge device). We compare our work to previously proposed FL attacks and reveal that as few as one adversarial device operating with a low-powered perturbation under our attack framework can induce the potent model poisoning attack to the global classifier. Moreover, we find that more devices partaking in adversarial poisoning will proportionally degrade the classification performance.
BibTeX
@inproceedings{wang2023how,
  author = {Su Wang and Rajeev Sahay and Christopher G. Brinton},
  title = {{How Potent are Evasion Attacks for Poisoning Federated Learning-Based Signal Classifiers?}},
  booktitle = {ICC},
  year = {2023},
  url = {https://openreview.net/forum?id=csCKwzHGcM},
}
Figure