TL;DR
This paper introduces a novel framework for secure split learning in multi-hop wireless networks, where edge devices collaboratively train a model while fending off eavesdroppers. The core innovation is a deep reinforcement learning approach that intelligently assigns devices to training or deceptive signal transmission tasks, optimizes model partitioning, and manages transmit power to minimize leaked information while respecting energy and latency constraints.
What Problem Does This Paper Address?
Collaborative machine learning, particularly Federated Learning (FL) and Split Learning (SL), offers privacy benefits and distributed training capabilities. However, FL's requirement to send entire local models to a central server can be impractical for resource-constrained edge devices. SL addresses this by partitioning models, but this introduces new challenges: efficiently splitting models across heterogeneous devices, managing communication overhead, and, crucially, protecting against eavesdroppers who can intercept model updates and infer sensitive information. Existing SL solutions often assume fixed model splits or don't account for the security risks posed by eavesdroppers actively trying to steal model data. This paper tackles the problem of optimizing model splitting, device task assignment (training vs. deceptive signal transmission), and transmit power in a multi-hop SL setting to minimize information leakage to eavesdroppers under strict energy and latency constraints, all without prior knowledge of eavesdropper locations or capabilities.
Methods
The paper proposes a Soft Actor-Critic (SAC) deep reinforcement learning framework with an Intrinsic Curiosity Module (ICM) and Cross-Attention (CA), dubbed ICM-CA. This framework allows a centralized agent (the server) to make complex decisions:
- Device Selection: Which devices will participate in model training and which will transmit deceptive signals.
- Model Splitting & Assignment: How to partition the global model into sub-models and assign them to selected devices.
- Transmit Power Allocation: Determining the optimal power for both model training transmissions and deceptive signal transmissions.
The ICM module encourages exploration of novel states and actions, helping the agent discover better strategies for reducing information leakage. The CA module enhances learning efficiency by identifying and prioritizing important historical state-action pairs. The system operates in a multi-hop fashion, meaning data and gradients are passed sequentially through devices. The problem is formulated as an optimization problem to minimize leaked information subject to delay and energy constraints.
Datasets
The paper uses the ImageNet-1k dataset to train a ResNet-101 model. No new benchmarks are introduced; rather, the performance is evaluated against existing RL algorithms using this standard dataset.
Key Insights and Ideas
- Deceptive Signals as a Proactive Defense: Instead of just passively protecting data, the framework actively uses a subset of devices to generate misleading signals, confusing eavesdroppers.
- Joint Optimization is Crucial: The effectiveness of model splitting, device task assignment, and transmit power allocation are deeply intertwined. Optimizing them jointly is key to minimizing leakage while meeting performance constraints.
- RL for Dynamic, Unknown Environments: The use of SAC with ICM-CA is well-suited for this problem because it can adapt to dynamic network conditions and the unknown behavior of eavesdroppers without explicit prior knowledge.
- ICM for Exploration, CA for Focus: The ICM drives the agent to explore more states and actions, leading to better policies, while CA helps it focus on the most relevant past experiences for faster and more efficient learning.
- Transmit Power Trade-offs: Analytical results show that reducing model transmission power and increasing deceptive signal power can both help reduce leaked information, highlighting a critical trade-off to manage.
Key Results
- The proposed ICM-CA framework improves the convergence rate by up to 3x compared to a traditional SAC algorithm.
- It reduces the amount of information leaked to eavesdroppers by up to 13% compared to the traditional SAC algorithm.
- The ICM module alone improves accumulated reward by up to 9%, while the combined ICM-CA approach shows significant gains in convergence and accumulated reward.
- The method demonstrates superior performance in scenarios with increasing monitoring probabilities and a higher number of eavesdroppers, outperforming SAC and PPO baselines.
- The framework can implicitly estimate eavesdropper locations and adapt its strategy, showing similar convergence but a slight decrease in accumulated reward when eavesdropper locations are unknown.
Limitations and Future Work
- Action Masking: The paper acknowledges that the large action space can lead to invalid actions, and an action mask algorithm is used to address this.
- Interference Ignored in Corollaries: Some analytical results (Corollaries 1 and 2) make simplifying assumptions, such as ignoring interference from multiple deceptive devices.
- Future Work: The paper suggests that future work could explore more complex network topologies and a wider range of attack models. The analytical results on optimal transmit powers could be extended to more general scenarios.
Summary updated