Abstract
Machine learning models often exhibit significant performance disparities across geographic locations, a phenomenon known as locational bias. This bias can lead to inequitable outcomes in critical applications such as agricultural subsidy distribution or traffic safety management. We introduce Referee-Meta-Learning (Meta-Ref), a framework that addresses this challenge by treating fairness as a dynamic optimization problem. Meta-Ref employs a "referee" network to assign location-specific learning rates, enabling models to adapt rapidly to new regions while minimizing performance gaps between local patches. Our experiments on satellite crop classification and traffic risk estimation demonstrate that Meta-Ref consistently outperforms existing meta-learning and domain adaptation methods in both accuracy and locational fairness.
Introduction: The Challenge of Locational Fairness
While fairness in machine learning is often discussed in the context of demographic attributes like race or gender, locational fairness is equally critical for equitable resource distribution. A model used for satellite-based crop monitoring might perform well in one county but fail in another due to differences in soil, climate, or farming practices.
The primary technical challenge is the "non-stationary" nature of geographic data. Unlike static categories, locational groups shift; a model trained on a specific set of cities must be able to generalize to entirely new regions. Standard fairness techniques often struggle with this spatial shift. The goal is to achieve fast adaptation (few-shot learning) in a new region while ensuring that prediction quality remains consistent across all local areas within that region.
Methods: The Meta-Ref Framework
Meta-Ref builds upon Model-Agnostic Meta-Learning (MAML) by introducing a specialized "Referee" network. This referee acts as a balancer, learning how to guide the predictor model toward fair outcomes.
Three-Phase Training Process
The framework operates in three distinct phases:
- Performance Estimation: The model calculates local prediction losses and performance metrics across a cluster of locations (a "spatial task").
- Fairness-Aware Learning Rate Estimation: The Meta-Ref network analyzes local performance relative to the global average, alongside data features. It then assigns a specific learning rate to each location. Locations where the model struggles receive higher learning rates, forcing the model to prioritize them.
- Dual Meta-Updates: The system performs a "gradient of gradients" update. It simultaneously updates the prediction model for accuracy and the Meta-Ref network to ensure its learning rate assignments effectively improve fairness in subsequent rounds.
Key Innovations
- Relative Performance Invariance: By focusing on the difference between local error and global average, the referee remains effective even as the overall model improves, allowing for better transferability to new regions.
- Sigmoid Bounding: To stabilize early training, a sigmoid-shaped curve gradually expands the range of possible learning rates as the model matures.
Experimental Evaluation
The researchers evaluated Meta-Ref on two high-impact real-world scenarios:
- Satellite-based Crop Classification: Mapping walnut plantations in California’s Central Valley using Sentinel-2 multispectral data.
- Traffic Accident Risk Estimation: Predicting daily accident counts in Iowa based on weather, road conditions, and historical traffic records.
To measure success, the authors utilized the Adjusted Locational Fairness (ALF) metric, which penalizes models that achieve low variance by simply being equally inaccurate across all locations.
Results and Discussion
Meta-Ref consistently outperformed standard Deep Neural Networks (DNN), LSTMs, and standard MAML in both accuracy and fairness:
- Crop Monitoring: Meta-Ref achieved the highest F1-score (~0.71) and the lowest Locational Fairness (LF) score (~0.107), indicating the most consistent performance across counties.
- Traffic Safety: In the regression task, Meta-Ref achieved the lowest RMSE (0.131) and the best fairness scores (0.102 LF), demonstrating superior adaptation to complex spatial shifts.
Conclusion
Meta-Ref provides a robust solution for the problem of locational bias in spatial data. By framing fairness as a meta-learning task guided by a "referee," the model achieves both high accuracy and equitable performance across diverse geographic regions. Future work will focus on incorporating policy-defined fairness criteria and reducing the reliance on labeled data for few-shot adaptation.
Summary updated