Cali3F: Calibrated Fast Fair Federated Recommendation System

Zhitao Zhu, Shijing Si, Jianzong Wang, Jing Xiao

Abstract

Federated Learning (FL) has emerged as a vital paradigm for building recommendation systems that respect user privacy by keeping data on-device. However, traditional FL approaches often struggle with the "tug-of-war" between global model consistency and local personalization, leading to issues with data heterogeneity, training efficiency, and algorithmic fairness. We present Cali3F (Calibrated Fast Fair Federated), a novel framework that addresses these challenges through three integrated components: clustering-based sampling, calibrated local updates, and fast intra-cluster aggregation. Experimental results on standard benchmarks demonstrate that Cali3F not only improves recommendation accuracy and fairness but also achieves peak performance up to 10x faster than existing personalized FL methods.

Introduction

Standard recommendation systems typically require the centralization of sensitive user data, such as search histories and ratings, posing significant privacy risks. Federated Learning (FL) mitigates these risks by training models locally on user devices and only sharing encrypted updates with a central server. Despite its privacy benefits, FL in the context of recommendations faces three primary hurdles:

  1. Data Heterogeneity: Users possess diverse tastes. A single global model often fails to capture individual niches, resulting in poor performance for many participants.
  2. Fairness: Models optimized for the "average" user often provide substandard recommendations to "outlier" users with unique preferences. Maintaining uniform performance across a diverse population is a significant challenge.
  3. Efficiency: Training personalized models across thousands of devices is computationally expensive and slow to converge, leading to high communication costs and excessive battery drain on mobile devices.

Cali3F is designed to tackle these issues simultaneously, providing a balanced approach to personalization, speed, and fairness.

The Cali3F Framework

Cali3F builds upon Neural Collaborative Filtering (NCF) and introduces three core technical innovations to optimize the federated training process:

1. ClusSamp (Clustering-based Sampling)

Rather than selecting users at random for each training round, Cali3F employs ClusSamp. This method groups users into clusters based on similarities—utilizing privacy-neutral metadata or embeddings. Within each cluster, the system identifies "delegate" clients to represent the group, while others are designated as "subordinates." This targeted sampling ensures that the gradients sent to the server are representative of the entire user base without requiring every device to participate in every round.

2. CaliUp (Calibrated Update)

To address the fairness and personalization gap, CaliUp utilizes a multi-task learning approach. Local models are trained to be highly personalized but are simultaneously "calibrated" (regularized) against the global model using an adaptive $l_2$ norm. This tethering prevents local models from diverging into isolated echo chambers while ensuring they retain the collaborative knowledge inherent in the global community. This balance is the "secret sauce" that allows for specialization without losing the benefits of the broader dataset.

3. FastAgg (Fast Aggregation)

To accelerate convergence, FastAgg facilitates intra-group sharing. The updates learned by "delegates" are broadcast to their respective "subordinates" within the same cluster. This allows the entire system to benefit from localized improvements much faster than traditional methods that rely solely on global server-client communication cycles.

Experimental Setup

The authors evaluated Cali3F against standard industry benchmarks, including:

  • MovieLens 1M & 100K: Standard datasets for movie recommendation performance.
  • Pinterest: A highly sparse dataset containing over 1.5 million interactions, used to evaluate performance on implicit feedback (e.g., clicks and pins).

The model architecture partitions weights into user embeddings, item embeddings, and non-embedding (MLP) layers, treating each differently during the update process to further optimize for both personalization and global knowledge transfer.

Key Results and Discussion

Superior Accuracy and Fairness

Cali3F consistently outperformed the standard FedAvg baseline. On the MovieLens 100K dataset, Cali3F achieved an NDCG@10 of 0.40, a notable improvement over FedAvg’s 0.36. Furthermore, the framework significantly improved fairness. The standard deviation of performance across users—a key metric for unfairness—dropped from 0.38 (FedAvg) to 0.33 (Cali3F), indicating a narrower gap between the best-served and worst-served users.

Drastic Speedup and Efficiency

One of the most striking results is the efficiency of the Cali3F framework. The model reached within 5% of its peak performance in just 10 communication rounds. In terms of wall-clock time on the MovieLens 1M dataset, Cali3F reached this threshold in 3.4 minutes, whereas the competing Ditto framework required 34.5 minutes—a 10x speedup. The authors also noted that Cali3F can omit up to 98.6% of training rounds compared to traditional FL while still maintaining satisfactory results.

Personalization as a Catalyst

Interestingly, the research suggests that allowing for local personalization actually aids global convergence. By refining gradients at the cluster level before they are sent back to the server, the global model receives higher-quality updates, leading to a more stable and faster training process.

Limitations and Future Work

While Cali3F represents a significant step forward, some limitations remain. The ClusSamp method relies on the availability of privacy-neutral metadata for initial clustering; in its absence, the system must rely on evolving embeddings, which may be less stable in early rounds. Additionally, the current work focuses on performance and fairness rather than "Byzantine" robustness—the ability to withstand malicious users attempting to poison the model. Future research will focus on these security aspects and provide a more rigorous theoretical analysis of the framework's convergence properties.

Summary updated

BibTeX
@inproceedings{zhu2022cali,
  author = {Zhitao Zhu and Shijing Si and Jianzong Wang and Jing Xiao},
  title = {{Cali3F: Calibrated Fast Fair Federated Recommendation System}},
  booktitle = {IJCNN},
  year = {2022},
  url = {https://openreview.net/forum?id=3WTJytU2dF},
}
Figure