Introduction
Federated Machine Learning (FL) has emerged as a transformative paradigm for training machine learning models on decentralized data. By allowing clients to train models locally and sharing only model updates with a central aggregator, FL addresses critical concerns regarding data privacy and bandwidth. However, while the mathematical foundations of FL—such as gradient optimization and loss functions—have seen significant advancement, the practical challenges of building and maintaining FL as a distributed software system remain under-explored.
This paper presents a systematic literature review (SLR) of 231 primary studies, uniquely analyzing the field through the lens of Software Engineering (SE). By mapping current research to the Software Development Life Cycle (SDLC), we identify the state-of-the-art and highlight the gaps between theoretical research and production-ready systems.
Methodology
To ensure a comprehensive and unbiased review, we followed the established Kitchenham guidelines for systematic reviews. Our process involved an initial search across major digital libraries, including ACM, IEEE, ScienceDirect, and arXiv, covering the period from 2016 to early 2020.
As shown in the figure above, the methodology included rigorous paper screening based on inclusion and exclusion criteria, followed by a bi-directional snowballing process to ensure no significant studies were omitted. This resulted in a final set of 231 primary studies for data synthesis.
The Software Engineering Perspective
The core contribution of this review is the categorization of FL research into the standard phases of the software engineering lifecycle. We formulated specific research questions (RQs) to investigate how the community addresses background understanding, requirement analysis, architecture design, and implementation/evaluation.
1. Background and Requirements
Our analysis reveals that privacy is the primary driver for FL adoption, cited by 62% of the studies. From a requirement analysis standpoint, there is a heavy focus on non-functional requirements like communication efficiency (23%). However, other critical SE requirements, such as system auditability and user incentives ("client motivatability"), are significantly under-represented.
2. Architecture and Design
FL systems are fundamentally distributed. While the goal is decentralization, we observed a "centralization paradox": most current architectures still rely on a central server for orchestration, creating potential single points of failure. We identified a taxonomy of components, distinguishing between mandatory elements (like local model trainers) and optional but critical enhancements (like anomaly detectors and model compressors).
3. Implementation and Evaluation
A significant "lab-to-production" gap exists. Approximately 79% of the literature is classified as "validation research" based on simulations, while only 6% represents "evaluation research" involving real-world case studies. Furthermore, nearly 70% of research focuses exclusively on the training phase, neglecting deployment, monitoring, and maintenance.
Data Landscape and Benchmarks
The review catalogs the types of data used in FL research. Image data is the most dominant (49%), with the MNIST dataset serving as the most frequent benchmark for proof-of-concept. Structured data (21%) and text data (14%) follow. Notably, there is a lack of research utilizing IoT time-series data or complex graph data, despite the frequent marketing of FL for edge computing and IoT scenarios.
Key Insights
- Non-IID Challenges: Handling Independent and Identically Distributed (Non-IID) data remains a top-tier challenge. When every user's data distribution differs, model performance often degrades, and standardized SE solutions for this are still immature.
- The Training Bias: The disproportionate focus on model training means that the "unsexy" parts of the lifecycle—such as data cleaning at the edge and versioning—are largely ignored.
- Emerging Trends: There is a nascent but growing trend toward using blockchains for decentralized data governance to resolve the centralization paradox.
Future Directions
To advance FL from a research topic to a robust engineering discipline, future work must address the full lifecycle: moving beyond training to include edge-side data preparation, model deployment, and long-term performance monitoring. Additionally, researchers should explore semi-supervised FL for unlabeled client data and investigate how FL systems can comply with regulatory mandates like the "Right to be Forgotten" under GDPR.
By bridging the gap between ML algorithms and software engineering practices, we can build more reliable, scalable, and privacy-preserving federated systems.
Summary updated