A robust federated biased learning algorithm for time series forecasting

Mingli Song, Xinyu Zhao, Witold Pedrycz

TL;DR

The authors introduce "Federated Biased Learning" (FedBiased), a novel federated learning algorithm that aggregates local models by weighting them according to their individual accuracy, rather than simply averaging them. Tested on multi-station air quality forecasting, FedBiased demonstrates superior robustness against noisy sensor data, proving that federated learning can outperform centralized learning in real-world, error-prone environments—even when data privacy is not the primary concern.

What Problem Does This Paper Address?

Federated Averaging (FedAvg) is the benchmark algorithm for aggregating local models in a federated learning (FL) setup. However, FedAvg assumes all local updates are equally valuable and simply averages their parameters. In real-world Internet of Things (IoT) or multi-sensor environments, this is rarely true; some sensors may be aging, faulty, or subject to local environmental anomalies, resulting in biased or noisy data.

Furthermore, there is an ongoing debate in the machine learning community: if data privacy and security are not a concern, is there any reason to use federated learning over traditional centralized learning? The authors tackle both the algorithmic limitation of FedAvg's "equal weight" approach and the broader question of whether FL offers inherent robustness advantages over centralized models when dealing with noisy time-series data.

Methods

The authors propose Federated Biased Learning (FedBiased). The core innovation is a dynamic, performance-based weighting mechanism during the global aggregation step.

  • The Algorithm: Instead of calculating a simple mean of all local model weights (like FedAvg), the central server in FedBiased calculates a normalized weight for each local model based on its Mean Squared Error (MSE). Models with lower errors contribute more heavily to the global model update.
  • Base Architecture: For the local models, the authors utilized a shallow Back Propagation Neural Network (BPNN) with a sliding window approach (24 hours of input to predict the 25th hour). They explicitly chose this over deeper networks like LSTMs because empirical tests showed the shallow network trained faster and achieved lower MSE for this specific univariate task.
  • Robustness Testing: To simulate sensor degradation, the authors injected Gaussian white noise into the time series. They tested three scenarios: noise across the entire timeline, noise in the first half, and noise in the second half, varying the Signal-to-Noise Ratio (SNR) to control intensity.
  • Spatial Grouping: Local clients (stations) were grouped for federated learning based on geographic proximity, calculated using Euclidean distance on latitude, longitude, and elevation.

Datasets

  • Beijing AQI Dataset: The primary dataset consists of hourly Air Quality Index (AQI) time series data collected from 35 monitoring stations in Beijing, China, during the first quarter of 2022.
  • Synthetic Noisy Data: The authors generated noisy variants of the AQI dataset using Gaussian white noise at various Signal-to-Noise Ratios (SNR = 10, 20, 30, and 40) to evaluate algorithmic robustness.

Key Insights and Ideas

  • Accuracy-weighted aggregation prevents model degradation: By penalizing poorly performing local models, the global model avoids being dragged down by a single faulty sensor. This makes the system highly resilient to localized data corruption.
  • FL is a powerful tool for robustness, not just privacy: A major takeaway is that federated learning acts as a structural regularization mechanism. Even when all data could be centralized, keeping it distributed and using FL helps mitigate the impact of noise. In the presence of data uncertainties, FL actually outperformed centralized learning.
  • Shallow networks can beat deep networks on univariate time series: The authors found that a simple BPNN outperformed an LSTM. This is a great reminder that researchers should establish baselines with simpler architectures before defaulting to computationally expensive deep learning models.
  • More clients do not automatically equal a better model: The study noted that simply adding more stations to the federated group doesn't guarantee better performance. Finding the optimal combination of geographically and environmentally relevant stations is more important than sheer scale.

Key Results

  • Superior Accuracy: Across various groupings of monitoring stations, FedBiased consistently achieved lower Mean Squared Error (MSE) and Mean Absolute Error (MAE) compared to standard FedAvg.
  • Statistical Significance: Two-sample t-tests confirmed that the performance improvements of FedBiased over FedAvg were statistically significant (p-values well below the 0.05 threshold).
  • High Robustness to Noise: When severe noise (SNR = 40) was injected into one station's data, FedBiased maintained high accuracy (MSE = 0.000136), successfully outperforming both FedAvg (0.000139) and Centralized learning (0.000137).
  • Noise Timing Matters: The experiments revealed that noise occurring in the first half of the time series had a slightly greater negative impact on the final training results than noise in the second half.

Limitations and Future Work

  • Underutilized Spatial Data: While geographic proximity was used to group stations, the spatial relationships were not explicitly encoded into the neural network architecture (e.g., via Graph Convolutional Networks). The authors note this as a primary vector for future improvement.
  • Scope of Evaluation: The framework was tested exclusively on environmental time series (AQI). The authors acknowledge the need to validate FedBiased's effectiveness and generalizability on diverse, non-environmental datasets.
  • Algorithm Comparisons: Future work will involve benchmarking FedBiased against a wider array of modern, state-of-the-art federated learning algorithms beyond just FedAvg.

Summary updated

Abstract

<title>Abstract</title> The federated averaging algorithm (FedAvg) is extensively used for multi-sensor data modeling but often overlooks the unique characteristics of local models when privacy and data security are not considered. This study introduces a novel federated learning algorithm built upon the FedAvg framework, which emphasizes the specificity of each local model to optimize global knowledge aggregation. The algorithm's effectiveness is demonstrated through an air quality index prediction problem, showcasing superior prediction performance and robustness in noisy data scenarios. Additionally, the study delves into the reliability and robustness of the proposed approach, addressing the prevalent notion that centralized learning methods often surpass federated learning when data security is not a concern. Our experiments affirm the necessity and superiority of federated learning methods, even in the absence of privacy considerations, by effectively managing real-world noisy data.
BibTeX
@article{song2025robust,
  author = {Mingli Song and Xinyu Zhao and Witold Pedrycz},
  title = {{A robust federated biased learning algorithm for time series forecasting}},
  journal = {Clust. Comput.},
  year = {2025},
  doi = {10.1007/s10586-024-05061-7},
}
Figure