Abstract
Multivariate ranking systems in high-stakes domains, such as education and hiring, often face the challenge of maintaining long-term fairness without sacrificing system stability. Traditional debiasing methods frequently focus on static snapshots, leading to "drastic" policy swings that can undermine social trust. In this work, we introduce FAiRDAS (Fairness-Aware Ranking as Dynamic Abstract System), a framework that treats fairness as a controllable dynamic system. By modeling fairness metrics over time and employing a feedback mechanism, FAiRDAS ensures that ethical interventions remain stable and smooth. Our experiments on real-world educational data demonstrate that FAiRDAS achieves parity in fairness and accuracy while significantly improving the stability of mitigation actions compared to traditional baselines.
Introduction
In Artificial Intelligence for Education (AIEd), algorithms are increasingly used to rank students for critical interventions, such as identifying dropout risks or allocating limited scholarships. While ensuring fairness across demographic groups is a known requirement, most existing methods treat fairness as a one-off constraint. This approach ignores Fairness Stability: the need for a system to evolve predictably over time.
If an algorithm detects a demographic imbalance and applies a "drastic" correction, it may effectively penalize a specific group so heavily that they disappear from top rankings in subsequent iterations. Such volatility makes the system ethically questionable and socially untrustworthy. The challenge lies in achieving long-term equity without making the mitigation actions themselves highly unstable or unpredictable.
The FAiRDAS Framework
FAiRDAS formalizes the evolution of fairness and quality metrics as an abstract dynamic system. Instead of simply setting a static fairness threshold, the framework defines a linear discrete dynamic system to govern the trajectory of the metrics:
$$y_{t+1} = \lambda \odot (y_t - \mu) + y_t$$
In this model, $\mu$ is the target fairness threshold, and $\lambda$ (the eigenvalue) acts as a "smoothness knob." A low $\lambda$ ensures the system moves toward the fairness goal gradually, while a high $\lambda$ forces faster, potentially more volatile convergence.
To implement these goals, the system calculates an action vector ($\theta$), representing penalty or reward factors applied to student scores based on sensitive attributes. The framework solves an optimization problem using Sequential Least Squares Programming (SLSQP) to minimize the distance between actual metrics and the ideal state dictated by the dynamic system.
Experimental Setup
We evaluated FAiRDAS using a case study in AIEd based on real-world data from the Canarian Agency for Quality Assessment and Accreditation (ACCUEE). The dataset spans four academic years (2015–2019) and includes student proficiency in Math, Spanish, and English.
The protected variable used was ESCS (Economic, Social, and Cultural Status), serving as a proxy for socioeconomic background. To simulate a sequential ranking process, we created 100 batches of 32 students, allowing us to observe the long-term behavior of the mitigation policies.
Key Results and Discussion
FAiRDAS was compared against a baseline that minimizes unfairness at each step without considering temporal state.
- Metric Parity: Both FAiRDAS and the baseline achieved similar levels of fairness, measured by the Disparate Impact Discrimination Index (DIDI), and accuracy, measured by the Sum of Absolute Errors (SAE).
- Superior Stability: The defining advantage of FAiRDAS is Action Smoothness. Under strict fairness requirements, the baseline's actions were significantly more unstable, frequently jumping between extreme penalties. In contrast, FAiRDAS produced a smooth, gradual gradient of adjustments.
- The Stability-Fairness Trade-off: Our results highlight a measurable trade-off between the speed of reaching a fairness goal and the stability of the policy. We found that an eigenvalue of $\lambda = 0.2$ provided an optimal balance for this specific dataset.
Conclusion
By treating fairness as a trajectory rather than a static target, FAiRDAS provides a mathematically grounded way to ensure ethical interventions are both effective and stable. This approach prevents the "erasure" of groups that can occur with over-corrective baseline methods. Future work will explore applying FAiRDAS to more complex action spaces, such as deep neural network weights, and testing its generalization in other high-stakes domains like hiring and lending.
Summary updated