Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach

Qingxiang Liu, Sheng Sun, Yuxuan Liang, Min Liu, Jingjing Xue

TL;DR

FUELS (Federated dUal sEmantic aLignment-based contraStive learning) is a new framework for spatio-temporal forecasting that tackles the "one-size-fits-all" limitation of traditional Federated Learning. By using a dual-contrastive approach to align temporal and spatial data patterns, it achieves state-of-the-art prediction accuracy while slashing communication costs by a staggering 94%.

What Problem Does This Paper Address?

Spatio-temporal forecasting—predicting things like traffic flow or cellular network demand—is notoriously difficult because data is often siloed across different organizations (e.g., different telecom operators). Federated Learning (FL) allows these organizations to train a shared model without exchanging raw data, but it has a major flaw: it struggles with spatio-temporal heterogeneity.

In plain English, traffic patterns in a quiet suburb are fundamentally different from those in a bustling city center (spatial heterogeneity), and morning rush hour looks nothing like midnight (temporal heterogeneity). Standard FL methods often try to force a single global model on everyone, which washes out these unique local "personalities." Existing personalized FL (PFL) methods try to help, but they often fail to capture the complex, coupled nature of time and space in these datasets.

Methods

The authors propose FUELS, which introduces two specialized contrastive learning tasks to inject "heterogeneity awareness" into the model's latent representations:

  1. Intra-client Contrastive Task (Temporal): Within a single client (like one base station), the model needs to distinguish between different times. The authors designed a hard negative filtering module. This module dynamically identifies "hard negatives"—time stamps that might look similar on the surface but represent different underlying patterns—and ensures the model learns to tell them apart.
  2. Inter-client Contrastive Task (Spatial): To handle differences between locations, the server uses periodicity-aware prototypes. Instead of sending massive model weights back and forth (which is slow and expensive), clients send compact "prototypes" that represent their local data distributions.
  3. JSD-based Aggregation: The server uses Jensen-Shannon Divergence (JSD) to measure how similar these prototypes are. It then clusters similar clients together to create "customized global prototypes." This allows a suburban base station to learn from other suburban stations while staying distinct from downtown ones.

Datasets

The researchers evaluated FUELS on four real-world datasets:

  • Telecom Italia Big Data Challenge (Trentino): This includes three types of data—Short Message Services (SMS), Voice Calls, and Internet Services (Net)—collected from 6,575 cells over two months.
  • METR-LA: A standard benchmark containing traffic speed records from 207 loop detectors in Los Angeles.

Key Insights and Ideas

  • Prototypes as Communication Carriers: One of the biggest "aha" moments is moving away from transmitting full model parameters. By using compact prototypes, FUELS preserves the "semantic essence" of a location's data while being incredibly lightweight.
  • Periodicity is the Key: Traffic data is cyclical. By designing "periodicity-aware" prototypes, the authors filter out random noise and focus on the stable, recurring patterns that actually matter for forecasting.
  • Smart Similarity with JSD: Using JSD allows the server to quantify distribution differences without ever seeing the raw data. This strikes a perfect balance between personalization and privacy.
  • Hard Negative Filtering: In contrastive learning, choosing "negative pairs" (things that are different) is hard. Simply picking random time stamps doesn't work well. The dynamic filtering module ensures the model is challenged by the most confusingly similar-looking data points, leading to much more robust representations.

Key Results

FUELS consistently outperformed six state-of-the-art baselines (including FedAvg, FedProx, and pFedMe) across all datasets.

  • Accuracy: It achieved the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE). For example, on the "Net" dataset, it improved MSE significantly over the next best competitor.
  • Efficiency: The most impressive result is the 94% reduction in communication parameters. Because it transmits prototypes rather than full model weights, it is far more practical for real-world deployment where bandwidth is limited.
  • Robustness: The authors demonstrated that FUELS maintains its performance even when privacy-preserving noise (like Differential Privacy) is added to the prototypes.

Limitations and Future Work

While FUELS is highly effective, the authors note a few areas for growth:

  • Computational Overhead: While communication is reduced, the local training involves encoding augmented data for contrastive tasks, which adds some local compute cost.
  • Dependency on Periodicity: The current prototype design relies heavily on the periodic nature of the data. While common in traffic and telecom, it might need adjustment for less cyclical spatio-temporal tasks.
  • Scaling JSD: Calculating JSD for thousands of clients can be computationally expensive for the server, though the authors proposed a random sampling strategy to mitigate this.

The authors suggest that because the communication carrier is independent of the model architecture, future work could involve building FUELS on top of Large Language Models (LLMs) or other heterogeneous local models.

Summary updated

Abstract

The existing federated learning (FL) methods for spatio-temporal forecasting fail to capture the inherent spatio-temporal heterogeneity, which calls for personalized FL (PFL) methods to model the spatio-temporally variant representations. While contrastive learning is promising in tackling spatio-temporal heterogeneity, the existing methods are noneffective in distinguishing positive and negative pairs and can hardly apply to PFL paradigm. To tackle this limitation, we propose a novel PFL method, named Federated dUal sEmantic aLignment-based contraStive learning (FUELS), which can adaptively align positive and negative pairs based on semantic similarity, thereby injecting precise spatio-temporal heterogeneity into the latent representation space by auxiliary contrastive tasks. From temporal perspective, a hard negative filtering module is introduced to dynamically align heterogeneous temporal representations for the supplemented intra-client contrastive task. From spatial perspective, we design lightweight-but-efficient prototypes as client-level semantic representations, based on which the server evaluates spatial similarity and yields client-customized global prototypes for the supplemented inter-client contrastive task. Extensive experiments demonstrate that FUELS outperforms state-of-the-art methods, with impressive communication cost reduction.
BibTeX
@inproceedings{liu2025personalized,
  author = {Qingxiang Liu and Sheng Sun and Yuxuan Liang and Min Liu and Jingjing Xue},
  title = {{Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i11.33328},
}
Figure