Abstract
Federated Transfer Learning (FTL) provides a privacy-preserving framework for collaborative machine learning across isolated data silos with differing feature spaces. However, the privacy guarantees of FTL introduce significant performance overheads. This study quantifies these costs using the FATE framework, revealing that FTL can be 17x to 24x slower than traditional distributed machine learning. The primary bottlenecks are identified as inefficient inter-process communication (IPC) and the computational burden of homomorphic encryption.
Introduction
Data silos created by strict privacy regulations like GDPR prevent organizations from pooling data for model training. While standard Federated Learning (FL) addresses this for parties sharing the same feature space, Federated Transfer Learning (FTL) extends collaboration to parties with heterogeneous data. Despite its theoretical appeal, the practical deployment of FTL is hindered by a "performance tax" that has remained largely unquantified until now.
Methodology
The researchers evaluated FTL performance using the Federated AI Technology Enabler (FATE) framework deployed on Google Cloud. They compared FTL against a baseline of standard distributed machine learning (TensorFlow) using a Breast Cancer dataset. The analysis decomposed "Task Completion Time" into CPU execution time (training vs. overhead) and data transfer time, testing both homogeneous and heterogeneous logistic regression scenarios.
Performance Bottlenecks
The study identifies three critical areas where performance degrades:
1. The Efficiency Gap
Only approximately 18% of CPU time is dedicated to actual model training. The remaining 82% is consumed by "extra execution" tasks required to manage the federated environment.
2. Inter-Process Communication (IPC)
The FATE framework's architecture, which mixes Python for machine learning logic and Java for infrastructure, creates a massive IPC bottleneck. Moving data between these environments and managing memory copies accounts for up to 77% of total execution time.
3. Encryption Overhead
FTL utilizes Paillier homomorphic encryption to ensure privacy. This process not only increases CPU load but also significantly inflates data size. A single numerical value can expand to 1024 bits, resulting in 18x more data transfer compared to unencrypted distributed learning.
Results
The empirical results highlight a stark contrast between FTL and traditional methods:
- Total Slowdown: FTL was 24x slower for homogeneous models and 17x slower for heterogeneous models compared to standard distributed learning.
- Network Sensitivity: In cross-region deployments, network latency accounts for up to 34% of total completion time.
- Encryption Impact: Data encryption alone doubles the training time compared to unencrypted FTL.
Conclusion and Future Work
Current FTL implementations face severe scalability challenges. Increasing hardware resources (vCPU and RAM) yielded diminishing returns, suggesting that software architecture is the primary constraint. Future improvements should focus on optimizing IPC through JVM native memory heaps and UNIX domain sockets, as well as exploring hardware acceleration (FPGAs/ASICs) for homomorphic encryption to make FTL viable for production-scale applications.
Summary updated