Introduction
Deep learning deployment in smart buildings often faces significant resource constraints. While most Federated Learning (FL) research focuses on horizontal scenarios, smart buildings typically encounter Vertical Federated Learning (VFL) challenges. In these environments, diverse sensors—such as cameras, temperature sensors, and CO2 monitors—collect different features for the same set of entities.
In a typical VFL setup, each IoT device trains a local "feature model" to transform raw data into a compact "embedding," which is then transmitted to a central server for final processing. However, the limited power and bandwidth of IoT devices necessitate new strategies to reduce both computational and communication overhead.
The LVFL Framework
To address these challenges, this paper introduces the Lightweight Vertical Federated Learning (LVFL) framework. LVFL employs a "dual lightweighting" strategy to optimize on-device performance and network usage:
- Feature Model Lightweighting ($\alpha$): This component uses structured pruning to reduce on-device CPU and battery consumption. By identifying and removing entire filters or neurons in the local neural network based on $l_1$ norm importance, the framework creates a physically smaller, faster model.
- Feature Embedding Lightweighting ($\beta$): To conserve bandwidth, the framework applies unstructured pruning to the feature embeddings before transmission. By zeroing out the least important values, high compression ratios can be achieved without significantly impacting the global model's learning capability.
Experimental Evaluation
The framework was evaluated using a quadrant-split version of the CIFAR-10 dataset to simulate a multi-sensor environment. Each $32 \times 32$ image was divided into four quadrants, with each quadrant assigned to a different client. The local feature models utilized the VGG16 architecture, while the central server employed a 3-layer Fully Connected Network (FCN) as the "head model."
Results and Analysis
The experimental results demonstrate that LVFL can maintain test accuracy levels comparable to non-lightweighted baselines while significantly reducing resource consumption.
Pruning Sensitivity and Recovery
A critical insight from the study is the differing impact of model versus embedding pruning. As shown in the figures below, pruning the local model ($\alpha$) causes a sharp, immediate drop in accuracy. However, the model enters a "recovery phase" where the remaining neurons adapt, eventually converging to a high accuracy level. In contrast, pruning the feature embeddings ($\beta$) has a negligible impact on the final accuracy, suggesting that these embeddings contain significant redundancy.
Heterogeneous Efficiency
Unlike horizontal FL, where a single global model is shared, VFL allows for heterogeneous pruning. This means that lightweighting ratios can be tailored to the specific hardware capabilities and battery levels of individual sensors, providing a flexible approach to system-wide efficiency.
Conclusion
LVFL offers a viable path for deploying sophisticated deep learning models in resource-constrained IoT ecosystems. By balancing computational pruning with communication compression, the framework ensures robust learning performance. Future research will focus on dynamic optimization of pruning ratios to account for real-time fluctuations in network congestion and device power levels.
Summary updated