Standard AI models are typically trained to maximize average accuracy, minimizing global prediction errors like Mean Squared Error (MSE). However, this "one-size-fits-all" approach often results in performance disparities: some users receive highly accurate predictions while others suffer disproportionately. This paper introduces a novel Equitable Objective and a policy gradient-based training framework that prioritizes performance uniformity, ensuring public models serve a diverse range of downstream tasks fairly.
The Problem with "Average" Accuracy
Public AI models—such as those used for weather forecasting, traffic prediction, or carbon intensity tracking—act as a service for various "agents" (users or organizations). These agents rely on the model's predictions to make specific decisions, like scheduling electric vehicle (EV) charging or allocating healthcare resources.
The critical gap in standard training is its exclusive focus on global prediction error. Because different agents have distinct goals, constraints, and data complexities, a model might be highly accurate on average but perform poorly for specific, disadvantaged groups. This creates a form of "social inequity" where the benefits of AI are unevenly distributed.
The EQUITABLE PM Framework
To address this, the researchers propose EQUITABLE PM (Public Model), a framework designed to integrate the diverse objectives of downstream agents directly into the optimization process.
Core innovations of this approach include:
- The Equitable Objective ($L^q_{EQ}$): Inspired by $\alpha$-fairness in resource allocation, this objective introduces a hyperparameter $q$ to reweight the training loss. As $q$ increases, the optimization places more emphasis on the "worst-off" agents (those facing the highest costs or errors), forcing the model to improve outcomes for the most disadvantaged tasks.
- Decision-Focused Regret: Instead of merely evaluating prediction error, the model minimizes "regret"—the extra cost an agent incurs by making a decision based on a prediction rather than the ground truth.
- Policy Gradient (PG) Optimization: Because many real-world decision processes (like combinatorial optimization) are non-differentiable, standard backpropagation fails. The authors adapt Policy Gradient methods from reinforcement learning to optimize the public model even when the downstream decision process is a "black box."
- Fairness-Accuracy Balancing: A combined objective uses a parameter $\beta$ to allow developers to tune the trade-off between global accuracy and performance equity.
Real-World Applications and Results
The authors evaluated EQUITABLE PM using two complex scenarios:
- Carbon Efficiency in Data Centers: Using U.S. Energy Information Administration fuel mix data and Azure cloud workloads, the goal was to help data centers minimize carbon emissions and processing latency.
- Scheduled EV Charging: Combining ACN-Data, CAISO electricity prices, and National Household Travel Survey logs to minimize financial costs, carbon footprints, and water consumption for a diverse fleet of EV owners.
The empirical results demonstrate a striking improvement in equity:
- Reduced Disparity: In the data center application, EQUITABLE PM reduced the variance of cost regret by orders of magnitude (e.g., from 0.0085 in the baseline to 0.0002).
- Improved Worst-Case Outcomes: The discrepancy between the 95th and 5th percentiles ($C_{95} - C_5$) of cost regret dropped significantly. This indicates a much more uniform user experience across the simulated agents.
- The "Fairness Knob" in Action: The hyperparameter $q$ effectively acts as a tunable knob for equity. Theoretical proofs and empirical data show that increasing $q$ mathematically reduces the variance of performance across different agents.
- Regret over Error: While the baseline model achieved slightly better global MSE, EQUITABLE PM resulted in much lower actual costs for the agents. This proves that "more accurate" predictions in a vacuum don't always lead to better real-world decisions.
Limitations and Future Work
While promising, the framework has practical constraints. Training requires some knowledge of the agents' cost functions, which users might hesitate to share due to privacy or competitive concerns. Additionally, scaling this policy gradient approach from modest architectures (like LSTMs and Transformers) to massive foundation models remains an open challenge. Finally, future work must address adversarial robustness to prevent agents from "gaming the system" by maliciously reporting artificially high costs to receive preferential treatment.
Summary updated