Introduction
Standard Federated Learning (FL) has emerged as a powerful paradigm for training machine learning models across decentralized data silos while preserving privacy. However, traditional FL frameworks typically treat clients as passive "data containers." In many high-stakes industrial and medical applications, participants like factories or hospitals possess significant domain knowledge—human expertise, physical laws, or specialized craftsmanship—that is not fully captured in their local datasets.
This paper introduces Knowledge-Injected Federated Learning (FLwKM), a framework that allows decentralized clients to enhance a global model with their own local expert knowledge. By integrating "point-prediction" and "range-constraint" knowledge, FLwKM addresses the dual challenges of data scarcity and unreasonable model predictions that often plague purely data-driven approaches.
The FLwKM Framework
The core challenge in knowledge-injected FL is incorporating heterogeneous, proprietary knowledge without compromising privacy or requiring a complete overhaul of existing FL optimization algorithms. The proposed framework achieves this by treating local knowledge as a transformation of the global model rather than a modification of the training objective.
Knowledge-Based Models (KMs)
FLwKM defines two primary types of local knowledge:
- Prediction-type (P-KM): Local expert models that provide specific point estimates based on domain expertise.
- Range-type (R-KM): A set of hard constraints or "guardrails" that define the set of physically or logically possible outcomes (e.g., a chemical process result that must fall within a specific safety range).
Function-Transformation Mapping
The innovation lies in the Function-Transformation Mapping ($\mathcal{T}$). Instead of altering the global server model directly, each client $m$ applies a transformation to the global model $f(\theta; \cdot)$ to create a personalized local model $f^m$. This mapping allows the global model to evolve in a function space that respects local constraints while benefiting from the aggregate data of all participants.
Methodology and Implementation
The transformation formula used by each client is defined as: $$f^m = (1 - \lambda) \cdot \text{softmax}(f(x) + \infty \cdot (g_r(x) - \mathbf{1}^k)) + \lambda \cdot g_p(x)$$
In this equation, $\lambda$ (Trust Level) is a hyperparameter balancing the global data-driven model against local expert predictions. The $\infty$ term acts as a hard mask; if the global model predicts a class that the local R-KM deems impossible, that prediction is effectively zeroed out before the final softmax operation. Crucially, because this transformation occurs locally and only gradients are shared, the specific proprietary formulas of the local KMs remain private.
Experimental Evaluation
The authors evaluated FLwKM across three distinct scenarios:
- Industrial Coking Process: A real-world dataset from coal-mixing industries in China where data collection is expensive and domain expertise is vital.
- Forest Cover Type (Covtype): A classic UCI dataset for predicting forest cover from cartographic variables.
- Fashion MNIST (FMNIST): A standard benchmark for image classification.
Key Results
- Constraint Satisfaction: In the industrial case study, FLwKM reduced the Percentage of Violation (POV)—the frequency with which the model breaks expert rules—from as high as 12% in standard FL to 0%.
- Performance Gains: On the Covtype dataset, accuracy for specific clients jumped from 65% to 71%. The gains were most pronounced in "data-scarce" environments (e.g., using only 1% of available training data), demonstrating that expert knowledge can effectively compensate for missing data.
- The "U-Shape" Effect: The impact of the trust level $\lambda$ follows a U-shape; optimal performance requires a careful balance between global data and local expertise. Too much reliance on either can lead to sub-optimal results.
Conclusion and Future Work
FLwKM provides a robust, algorithm-agnostic way to inject domain expertise into federated learning. It ensures that models remain physically and logically consistent while improving accuracy in data-poor environments. Future research directions include automating the tuning of the trust level $\lambda$ through bilevel optimization and extending the framework to handle potentially faulty or malicious local knowledge.
Summary updated