Abstract
Data, as an observable form of knowledge, has become one of the most important factors of production for the development of Artificial Intelligence (AI). Meanwhile, increasing legislation and regulations on private and proprietary information results in scattered data sources also known as the "data islands". Although some collaborative learning paradigms such as Federated Learning (FL) can enable privacy-preserving training over decentralized data, they have inherent deficiencies in fairness, costs and reproducibility because of being learning-centric, which greatly limits the way how participants cooperate with each other. In light of this, we present a knowledge-centric paradigm termed Knowledge Augmentation in Federation (KAF), with focus on how to enhance local knowledge through collaborative effort. We provide the suggested system architecture, formulate the prototypical optimization objective, and review emerging studies that employ methodologies suitable for KAF. On our roadmap, with a three-way categorization we describe the methods for knowledge expansion, knowledge filtering, and label and feature space correction in the federation. Further, we highlight several challenges and open questions that deserve more attention from the community. With our investigation, we intend to offer new insights for what collaborative learning can bring back to decentralized data.
Introduction
As data privacy regulations like GDPR and CCPA tighten, data remains trapped in isolated "islands." While Federated Learning (FL) allows for privacy-preserving collaboration, it is fundamentally learning-centric. This focus leads to three major deficiencies: intractable fairness in balancing costs and payoffs, prohibitive bandwidth and compute costs for heavy model updates, and poor reproducibility due to participant volatility.
To address these, we introduce Task-Agnostic Federation (TAF), or Knowledge Augmentation in Federation (KAF). This paradigm shift moves away from training a single, task-specific model toward enabling data islands to exchange generic knowledge to enhance, clean, and expand their local datasets.
The TAF Architecture and Knowledge Couriers
In the TAF framework, collaboration does not require a common end goal. Participants exchange "Knowledge Couriers"—specific forms of information such as statistical metadata, latent vectors, attention scores, or synthetic samples. These couriers are often significantly lighter than full model gradients, reducing communication overhead. The ultimate goal is to return "enhanced private data" to the participants, providing a sustainable gain that remains with the local island even after the federation ends.
Methodological Roadmap
We categorize the techniques serving as building blocks for TAF into a three-way roadmap:
- Collaborative Data Expansion: Techniques to fill in missing information (Sample Extrapolation) or diversify local data (Federated Data Augmentation) using shared generators (GANs/VAEs) rather than raw data.
- Collaborative Data Refinement: Methods to exclude misinformation or "poisoned" data through dataset-level and sample-level filtering, as well as handling class imbalances across the federation.
- Collective Harmonization of Data: Aligning data across sources via Federated Data Correction (fixing mislabeled samples) and Federated Feature Engineering (creating a unified, expressive feature space).
Key Insights and Results
The core advantage of TAF is that the "gain" is baked into the local data. Once a dataset is enhanced or cleaned, that value is permanent. Quantitative reviews highlight the efficiency of this approach; for example, one study on autonomous driving achieved a 99% compression rate for network traffic using federated feature selection with negligible performance impact. Furthermore, empirical evidence suggests that "low-quality" data can be just as detrimental to federation performance as skewed (non-IID) data, justifying the critical need for the Data Refinement branch of TAF.
Future Work
Several open frontiers remain for TAF, including moving toward decentralized P2P knowledge trading, exploring Federated Retrieval-Augmented Generation (RAG), and developing protocols for "Knowledge Erasure" to support the right to be forgotten. By focusing on what learning can bring back to the data, TAF offers a more flexible and robust path for decentralized AI development.
Summary updated