TL;DR
Current algorithmic fairness frameworks are often too narrow, relying on cooperative service providers and ignoring the broader "externalities" caused by system-wide optimization. This paper proposes Protective Optimization Technologies (POTs): external, non-cooperative tools that allow affected communities to fight back by strategically manipulating a system's inputs to mitigate environmental and social harms.
What Problem Does This Paper Address?
The authors argue that the burgeoning field of "algorithmic fairness" has a blind spot. Most current approaches focus strictly on the algorithm's internal logic (inputs and outputs) and assume that the service provider is a willing partner in fixing bias.
However, digital systems are more than just isolated algorithms; they are complex optimization engines designed to extract value. This optimization often creates negative externalities—harms that fall on non-users or the environment and aren't captured by the provider's internal metrics. For example, a traffic app like Waze might perfectly optimize a driver's route, but in doing so, it creates massive congestion and safety issues for residents of a previously quiet neighborhood.
The authors identify three main limitations in current fairness research:
- Narrow Scope: Focusing only on algorithmic outputs misses harms arising from the system's interaction with the physical world.
- Bias-Centric: Focusing only on discrimination ignores broader social and environmental impacts.
- Provider-Dependent: Assuming providers will implement "fairness" solutions ignores cases where their incentives are fundamentally misaligned with social good.
Methods
The authors ground their work in Michael Jackson’s theory of requirements engineering, distinguishing between the Machine (the software/algorithm) and the World (the application domain). They argue that harms often occur because the machine’s specification ($S$) fails to meet the world’s actual requirements ($R$).
To formalize this, they use economic utility functions. A provider optimizes their own utility ($U$) and their perceived social benefit ($\hat{B}$). Externalities ($\Delta B$) arise when the provider's model ($\hat{B}$) diverges from the actual social benefit ($B$).
POTs are designed as a bi-level multi-objective optimization problem. The POT designer seeks to find the least expensive modification to the system's inputs ($x \to x'$) that maximizes a protective social utility ($B_{pot}$), while accounting for how the target system will re-optimize its own parameters ($\theta$) in response to those modified inputs.
Datasets
The paper validates the POT concept through two distinct case studies:
- Traffic Routing: Used map data from the Open Street Maps API for three towns (Leonia, NJ; Lieusaint, France; and Fremont, CA). Travel times were estimated based on road type and validated against Google Maps data.
- Credit Scoring: Used the UCI German Credit Risk dataset, which contains 1,000 loan applications with features like job type, bank account status, and repayment history.
Key Insights and Ideas
- Optimization is the Source of Harm: Many social harms aren't just "bugs" or "biases" in an algorithm; they are the direct, intended result of optimizing for value extraction (e.g., maximizing user engagement or minimizing provider risk).
- The "God's View" Gap: Providers rarely have a complete view of social utility. Their models ($\hat{B}$) almost always omit impacts on non-users and the environment.
- Agency through Subversion: POTs represent a shift in power. Instead of asking a provider for fairness, affected parties use the system's own optimization logic against it to protect themselves.
- Shifting Harms: A crucial insight is that POTs might not eliminate harm entirely but can "shift" it. A town blocking Waze traffic effectively moves that congestion to another area.
Key Results
The authors demonstrated the effectiveness of POTs in their two case studies:
-
Traffic Thwarting: By strategically increasing travel times on just a subset of local roads (e.g., via speed bumps or traffic light changes), towns can discourage routing apps from using them as shortcuts. For Leonia, NJ, increasing segment times by 75% could prevent passthrough traffic until highway speeds dropped as low as 19 mph.
Figure 3: Solutions for Leonia, NJ (left) and Lieusaint, France (right), when the time of road segments is allowed to be in- creased by 75%. Town streets are marked in blue, highways in orange, and surroundings in grey. The red dots signify a and b, the closest points to the highway in the town. The roads marked by thick, black lines are the optimal set of seg- ments in which the time should be increased. -
Credit Scoring Poisoning: A collective of users can inject "poisoned" data (successfully repaid loans with specific attributes) into a bank's training set. In their simulation, injecting just 10 poisoned applications (1.25% of the training set) reduced the number of false negatives (wrongly denied loans) for a disadvantaged target group by nearly 65%.
Limitations and Future Work
The authors are candid about the risks associated with POTs:
- Arms Races: Service providers might deploy counter-measures, leading to a technical arms race that favors those with the most resources.
- Predatory Inclusion: In the credit scoring case, while the POT increased loan access, the authors note this could lead to "predatory inclusion," where marginalized groups are granted access to credit on unfavorable terms that lead to long-term debt.
- Complexity of Tracking: As optimization systems move toward continuous, multi-channel tracking, building effective POTs becomes significantly harder, potentially requiring the creation of "optimized doubles" of entities in the environment.
- Shifting vs. Solving: POTs often shift harms rather than eliminating them, raising difficult political questions about which communities have the resources to protect themselves at the expense of others.
Summary updated