Towards Artificial General or Personalized Intelligence? A Survey on Foundation Models for Personalized Federated Intelligence

Yu Qiao, Huy Q. Le, Avi Deb Raha, Phuong-Nam Tran, Apurba Adhikary, Mengchun Zhang, Loc X. Nguyen, Eui-Nam Huh, Dusit Niyato, Choong Seon Hong

Introduction

The rapid evolution of Artificial Intelligence has brought us to a pivotal juncture. While the industry has largely focused on the pursuit of Artificial General Intelligence (AGI)—a universal, centralized "brain"—this survey proposes a paradigm shift toward Artificial Personalized Intelligence (API). This is realized through Personalized Federated Intelligence (PFI), a framework that marries the immense power of Foundation Models (FMs) with the privacy-preserving, decentralized architecture of Federated Learning (FL).

Survey Structure Overview

The Core Challenges: Privacy, Resources, and Personalization

Despite the success of Large Language Models (LLMs), three significant barriers hinder their widespread real-world deployment:

  1. Privacy Concerns: Training centralized FMs requires massive datasets, often involving sensitive user information that individuals and organizations are reluctant to share.
  2. Resource Constraints: The computational cost of training models with billions of parameters is astronomical. For instance, training GPT-3 cost approximately $4.6 million. Such requirements are impossible for edge devices like smartphones or IoT sensors.
  3. The Personalization Gap: Centralized models often act as "jacks of all trades," failing to capture the specific nuances of an individual's professional workflow, local dialect, or medical history.

The PFI Technical Landscape

To address these challenges, the survey categorizes the PFI landscape into three primary pillars of opportunity:

1. RAG-based PFI

Retrieval-Augmented Generation (RAG) allows a model to query a local, private document index. By decoupling knowledge from generation, AI can utilize private data as context without that data ever leaving the user's device.

2. Efficient Adaptation (PEFT)

Parameter-Efficient Fine-Tuning (PEFT) techniques, such as LoRA (Low-Rank Adaptation) and Prompt Tuning, allow for updating a tiny fraction (often less than 1%) of a model's parameters. This makes federated updates over limited bandwidth feasible and efficient.

3. Trustworthy PFI

The survey emphasizes the need to mitigate the "dark side" of FMs, including reducing bias, preventing hallucinations through Reinforcement Learning from Human/AI Feedback (RLHF/RLAIF), and defending against adversarial attacks like model poisoning.

Taxonomy of Federated Learning: Horizontal, Vertical, and Transfer Learning

Key Insights and Paradigms

The paper identifies several critical shifts in the AI landscape:

  • The "ChatGPT Moment" for Vision: With models like Meta’s SAM (Segment Anything Model), computer vision has reached foundational scale, enabling multimodal PFI.
  • The Heterogeneity Triad: PFI must navigate mismatches in Domain (data distribution), Model (hardware capabilities), and System (network/battery constraints).
  • Meta-PFI: A forward-looking vision where autonomous AI agents learn from real-time behavioral feedback in a "user-agent-world" ecosystem.

Results and Future Directions

Quantitative benchmarks synthesized in the survey highlight the necessity of PFI. For example, LoRA-based retriever adaptation has been shown to boost recall by 2x to 4x on niche datasets. Furthermore, PFI techniques like quantization can reduce the communication overhead of a 7B parameter model from 28GB per round to manageable levels for 5G/6G networks.

Looking ahead, the authors identify "Green PFI" (sustainable, energy-aware training) and "Quantum-Enabled PFI" as the next frontiers. The ultimate goal is to move away from "one-size-fits-all" AI toward specialized "assistants" that truly understand the individual user.

Summary updated

Abstract

The rise of large language models (LLMs), such as ChatGPT, Gemini, and Grok, has reshaped the AI landscape. As prominent instances of foundational models (FMs), they exhibit remarkable capabilities in generating human-like content, pushing the boundaries towards artificial general intelligence (AGI). However, their large-scale nature, privacy sensitivity, and substantial computational demands pose significant challenges for personalized customization for end users. To bridge this gap, we present the vision of artificial personalized intelligence (API), which focuses on adapting FMs to individual users while ensuring privacy. As a central enabler of API, we propose personalized federated intelligence (PFI), a new paradigm that not only integrates the privacy benefits of federated learning (FL) with the generalization capabilities of FMs but also places personalization at its core. To this end, we first survey recent advances in FL and FMs that lay the foundation for PFI. We then explore core stages of the PFI pipeline: efficient personalization at the edge, trustworthy adaptation, and adaptive refinement via retrieval-augmented generation. Finally, we highlight future directions for enabling PFI. Overall, this survey aims to lay a foundation for the development of API as a complementary direction to AGI, with PFI as a key enabling paradigm.
BibTeX
@misc{qiao2025towards,
  author = {Yu Qiao and Huy Q. Le and Avi Deb Raha and Phuong-Nam Tran and Apurba Adhikary and Mengchun Zhang and Loc X. Nguyen and Eui-Nam Huh and Dusit Niyato and Choong Seon Hong},
  title = {{Towards Artificial General or Personalized Intelligence? A Survey on Foundation Models for Personalized Federated Intelligence}},
  year = {2025},
  eprint = {2505.06907},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.06907},
}
Figure