FLaaS: Federated Learning as a Service

Nicolas Kourtellis, Kleomenis Katevas, Diego Perino

TL;DR

This paper introduces FLaaS, a novel "Federated Learning as a Service" platform that enables multiple third-party applications to collaboratively build machine learning models without sharing raw user data. FLaaS addresses key challenges in permission management, privacy, and usability, demonstrating its feasibility on mobile devices for image object detection tasks.

What Problem Does This Paper Address?

Traditional Machine Learning as a Service (MLaaS) relies on centralized data collection, raising privacy concerns. Federated Learning (FL) offers a decentralized approach, keeping data on user devices. However, existing FL solutions often lack the ability for independent third-party applications to collaborate on building shared models, and they don't typically offer a user-friendly "as a service" model for easy deployment. This paper tackles the gap of enabling cross-application collaborative modeling in a privacy-preserving and accessible manner, addressing challenges like permission management, privacy guarantees, and efficient model training across diverse devices and network topologies.

Methods

FLaaS is designed as a service with high-level APIs and an SDK to facilitate its use. Its architecture comprises three main components:

  • Front-End: The user interface for customers (app developers) to configure and manage FL services.
  • Controller: Manages service configuration, monitors health, and orchestrates the FL process.
  • Central Server & Clients: The core of the FL execution. The Central Server handles federated aggregation of models, while Clients (e.g., mobile devices, edge nodes) run FLaaS Local modules for on-device training.

Key methodological contributions include:

  • Cross-Application Collaborative Modeling: FLaaS allows multiple independent applications to contribute to a single FL model, either for existing ML problems or to tackle new ones.
  • Permission and Privacy Management: The system incorporates APIs to define granular permissions for data and model sharing between applications, with an emphasis on privacy-preserving mechanisms (though specific DP implementation details are beyond the scope of this paper).
  • Hierarchical Model Construction: FLaaS supports building models across different network layers (device, edge, cloud).
  • "As a Service" Model: Providing high-level APIs and SDKs to simplify FL deployment for developers.
  • Transfer Learning (TL): FLaaS leverages TL to improve model performance and speed up training on resource-constrained mobile devices.

Datasets

The paper uses the CIFAR-10 dataset for training and evaluation. This dataset is split and distributed across experimental devices to simulate real-world scenarios of decentralized data. No new benchmarks were introduced; the focus was on evaluating FLaaS's performance on an existing, well-known dataset.

Key Insights and Ideas

  • Collaborative ML is Achievable and Beneficial: The paper demonstrates that enabling collaboration between applications for ML model building can lead to improved accuracy compared to individual models trained on smaller, siloed datasets.
  • On-Device Costs are Manageable: FLaaS can build FL models on mobile phones within hours, across many devices, with reasonable CPU utilization, memory footprint, and battery consumption. This suggests that FL is becoming practical for mobile environments.
  • Data Sharing for Joint Models is Efficient: The overhead for sharing data between applications on the same device for collaborative model building is minimal, making this a viable approach.
  • Device Heterogeneity Matters: Older devices may struggle with FL training due to longer execution times and higher power consumption, highlighting the importance of considering device capabilities.

Key Results

  • Accuracy Improvement: When two applications share data for joint FL modeling (S=250+250), the resulting model achieved 10% better accuracy than individual models trained on half the data (S=250).
  • On-Device Costs (per FL round):
    • CPU Utilization: 10-20% for S=250, and 27-34% for S=500.
  • Memory Footprint: Tens of MBs on average.
  • Battery Consumption: 3-5% per FL round.
  • Execution Time & Memory:
    • Newer devices (D1, D2) were significantly faster than older devices (D3).
  • Doubling the sample size (S=500 vs. S=250) practically doubled the execution cost for model training but had a similar cost per image loaded.
  • Memory consumption for model training also roughly doubled with doubled sample sizes.
  • Data Sharing Costs: Sharing 250/500 samples between apps took 54.7/65.9 ms and consumed 1.1/1.6 MB of memory on average.

Limitations and Future Work

  • Differential Privacy Implementation: While FLaaS plans to leverage Differential Privacy (DP), the paper acknowledges that finding an optimal way to add DP noise in FL to balance privacy and model utility is an open research problem and beyond the scope of this work.
  • Hierarchical FL Design: The design of optimal hierarchical FL methods for FLaaS is also noted as an open research problem.
  • Training Convergence and Performance: The impact of cross-application models on FL convergence and performance is a relevant research problem that FLaaS will need to address.
  • Proof of Concept Limitations: The current FLaaS version does not include implemented Controller and Front-End components.
  • Future Work: The authors aim to finalize FLaaS design and deployment for large-scale evaluation, arguing that many of the challenges identified represent fundamental open research problems in the Federated Learning space.

Summary updated

Abstract

Federated Learning (FL) is emerging as a promising technology to build machine learning models in a decentralized, privacy-preserving fashion. Indeed, FL enables local training on user devices, avoiding user data to be transferred to centralized servers, and can be enhanced with differential privacy mechanisms. Although FL has been recently deployed in real systems, the possibility of collaborative modeling across different 3rd-party applications has not yet been explored. In this paper, we tackle this problem and present Federated Learning as a Service (FLaaS), a system enabling different scenarios of 3rd-party application collaborative model building and addressing the consequent challenges of permission and privacy management, usability, and hierarchical model training. FLaaS can be deployed in different operational environments. As a proof of concept, we implement it on a mobile phone setting and discuss practical implications of results on simulated and real devices with respect to on-device training CPU cost, memory footprint and power consumed per FL model round. Therefore, we demonstrate FLaaS's feasibility in building unique or joint FL models across applications for image object detection in a few hours, across 100 devices.
BibTeX
@inproceedings{kourtellis2020flaas,
  author = {Nicolas Kourtellis and Kleomenis Katevas and Diego Perino},
  title = {{FLaaS: Federated Learning as a Service}},
  booktitle = {DistributedML@CoNEXT},
  year = {2020},
  url = {https://openreview.net/forum?id=rBSwxA6dCx9},
}
Figure