Abstract
Federated Deep Learning frameworks can be used strategically to monitor Land Use locally and infer environmental impacts globally. Distributed data from across the world would be needed to build a global model for Land Use classification. The need for a Federated approach in this application domain would be to avoid transfer of data from distributed locations and save network bandwidth to reduce communication cost. We use a Federated UNet model for Semantic Segmentation of satellite and street view images. The novelty of the proposed architecture is the integration of Knowledge Distillation to reduce communication cost and response time. The accuracy obtained was above 95% and we also brought in a significant model compression to over 17 times and 62 times for street View and satellite images respectively. Our proposed framework has the potential to be a game-changer in real-time tracking of climate change across the planet.
Introduction
Monitoring Land Use and Land Cover (LULC) is critical for tracking climate change impacts and ensuring international environmental compliance. However, global-scale monitoring faces two primary challenges: the bandwidth bottleneck of centralizing high-resolution imagery and the inherent heterogeneity of data (non-IID) across different geographic regions. While Federated Learning (FL) addresses privacy and data movement concerns, standard deep learning models like UNet are often too large for frequent synchronization across distributed networks.
Methodology: The FedUKD Framework
The researchers propose FedUKD (Federated UNet with Knowledge Distillation), which integrates a Teacher-Student architecture into the FL loop to optimize communication efficiency.
Teacher-Student Architecture
The core innovation lies in how local updates are handled within each client node:
- The Teacher: A large, high-performance UNet model (approx. 17 million parameters) remains local to the client, serving as a regional expert trained on local data.
- The Student: A compact version of the network (as few as 118,000 parameters) is used for global synchronization.
- Knowledge Distillation (KD): Instead of sharing the Teacher's heavy weights, the client uses the Teacher to "distill" knowledge into the Student. The Student learns to mimic the Teacher’s pixel-wise predictions locally, effectively compressing the regional expertise.
Federated Averaging
Only the lightweight Student model weights are transmitted to the central server. The server aggregates these updates and redistributes the improved global Student model to all clients. This ensures the "heavy lifting" remains local while only a compact, efficient representation of learning is shared across the network.
Datasets and Evaluation
The framework was evaluated on two distinct types of imagery to simulate global diversity:
- Chennai Land Use (CLU) Dataset: A novel dataset pairing official government masterplan maps with Google Earth satellite imagery across 70 regions.
- CityScape Street View (CSP) Dataset: A standard benchmark for urban scene understanding across 50 cities.
To simulate real-world challenges, the datasets were intentionally skewed to test the model's robustness against non-IID data distributions, where different clients see vastly different types of land use.
Key Results and Insights
FedUKD demonstrates that high performance can be maintained alongside extreme efficiency:
- Accuracy: The model achieved over 95% accuracy on satellite data and 71-78% on complex street-view imagery.
- Model Compression: The Student model achieved a 17x reduction for street views and a staggering 62x reduction (from 17M to 270k parameters) for satellite images.
- Communication Efficiency: Data transfer requirements for satellite models dropped from 69 MB to just 1 MB per round—a 69x reduction in memory footprint.
Beyond efficiency, the authors noted that Federated Averaging acts as a regularizer, helping local models avoid overfitting by aligning them with global trends derived from the collective knowledge of all clients.
Conclusion
FedUKD presents a scalable solution for global environmental monitoring. By repurposing Knowledge Distillation as a communication compression tool, the framework enables high-fidelity semantic segmentation with minimal bandwidth requirements. Future work will focus on scaling the system to thousands of nodes and integrating it into real-time climate tracking systems.
Summary updated