In real-world federated learning (FL), data is rarely distributed evenly. A global model might be trained across thousands of edge devices, but each individual user has unique habits, environments, and preferences. For instance, in indoor localization, different users move through entirely different geographic areas (Wu et al. (2021)), while in medical imaging, different hospitals encounter vastly different patient demographics and disease prevalences (Qi et al. (2024)). This mismatch is known as the non-IID (not independent and identically distributed) data problem, or statistical heterogeneity.
To develop and stress-test federated learning algorithms in a lab setting, researchers cannot simply rely on real-world decentralized datasets, which are scarce and difficult to share due to privacy constraints. Instead, they must take standard, centralized benchmark datasets (like CIFAR-10 or standard text corpora) and artificially split them across simulated clients.
The challenge is doing this in a way that accurately mimics the messy, unbalanced nature of the real world. The standard solution that has emerged across the field is Dirichlet-based data partitioning.
The Intuition: Controlling the Dial of Heterogeneity
Imagine you have a centralized dataset containing thousands of images split across 10 distinct classes, and you want to distribute these images among 100 simulated clients.
If you randomly assign images to each client, every client ends up with a nearly identical, perfectly balanced miniature version of the original dataset. This represents an IID scenario. In reality, however, one user might only take photos of cats, while another might only take photos of trucks.
To simulate this mathematically, researchers use the Dirichlet distribution, a probability distribution over a simplex (a set of probabilities that add up to 1). For each class $c$ in the dataset, a allocation vector $p_c$ is drawn from a Dirichlet distribution:
$$p_c \sim \operatorname{Dir}(\alpha)$$
Here, $p_c$ determines the proportion of data from class $c$ that will be allocated to each client. The behavior of this allocation is entirely governed by a single hyperparameter: the concentration parameter $\alpha$.
The concentration parameter acts as a continuous dial for data heterogeneity:
- As $\alpha \to \infty$ (Uniform Allocation): The distribution flattens. Every client receives an almost identical, balanced mix of all classes. The environment becomes completely homogeneous (IID).
- As $\alpha \to 0$ (Extreme Skew): The distribution becomes highly sparse and concentrated. Most of the probability mass collapses onto a few clients. Consequently, each client receives data from only a single or very few classes, and many classes become entirely absent from local nodes (Cai et al. (2022)).
By adjusting this single parameter $\alpha$, researchers can smoothly transition from a perfectly uniform data distribution to extreme imbalances among clients (Archetti et al. (2023); Morafah (2023)). This provides a standardized, mathematically rigorous way to control the degree of label skew during evaluation (Matsuda et al. (2024); Hu et al. (2023)).
Beyond Simple Label Skew
While the most common application of Dirichlet partitioning is skewing class labels for image classification tasks like SVHN or CIFAR-10 (Li et al. (2024)), real-world heterogeneity is often multi-dimensional. The Dirichlet framework has been adapted to capture these complex variations:
Quantity Imbalance and Dual-Dirichlet Strategies
Real-world clients do not just have different types of data; they also have different amounts of data. To simulate this, some frameworks implement a dual-Dirichlet strategy (Li et al. (2024)). One Dirichlet distribution determines the class proportions for each client, while a second Dirichlet distribution determines the total volume of data samples assigned to each client.
Semantic and Feature Shifts
In Natural Language Processing (NLP), data heterogeneity is not just about label frequencies, but also about vocabulary and topic variations. The authors of FedNLP (Lin et al. (2022)) address this by combining Dirichlet distributions with semantic clustering. They use text embeddings to group data by topic, and then apply a Dirichlet distribution to allocate these semantic clusters to clients, replicating the distinct linguistic patterns found in real-world data silos. A similar approach is used for multi-semantic datasets like scene graphs, where Dirichlet allocations control the concentration of semantic themes across nodes (Ha et al. (2024)).
Fleet Fingerprinting
Instead of choosing an arbitrary $\alpha$, researchers can learn the statistical distribution of a real-world device fleet and use a Mixture-of-Dirichlet-Multinomials to generate proxy client partitions (Scott et al. (2024)). This ensures that the simulated client nodes accurately match the observed label and feature distributions of actual hardware deployments.
Why It Works as a Research Standard
Dirichlet-based data partitioning has become a foundational component of modern federated learning benchmarking platforms. Low-code simulation platforms like EasyFL (Zhuang et al. (2021)) and modular architectures like ModularFed (Arafeh et al. (2022)) embed these distribution strategies directly into their core data control centers.
By providing a reproducible, parameterized way to scale non-IID conditions, Dirichlet partitioning allows the community to systematically evaluate personalized federated learning benchmarks (Chen et al. (2022)) and domain generalization algorithms (Bai et al. (2024)). It enables researchers to pinpoint exactly where standard global aggregation algorithms begin to drift, how economic incentives for collaboration change as data diversity grows (Sun et al. (2023)), and how network resource allocation must adapt to handle highly skewed client updates (Liu et al. (2024)).
Key Foundational Work
The Impact of Data Distribution on Fairness and Robustness in Federated Learning (2021) — contributed 1 concept elements to this direction.
Personalized Federated Learning over non-IID Data for Indoor Localization (2021) — contributed 1 concept elements to this direction.
EasyFL: A Low-code Federated Learning Platform For Dummies (2021) — contributed 1 concept elements to this direction.