TL;DR
This paper tackles the "filtering effect" in New York City's elite public high school admissions, where socioeconomic disparities lead to unequal opportunities. The authors propose and analyze targeted interventions, like vouchers for tutoring, to mitigate this bias and improve fairness, demonstrating their effectiveness through theoretical models and real-world data.
What Problem Does This Paper Address?
The paper addresses a significant issue in public school admissions: the "filtering effect." In New York City, specialized high schools (SHSs) rely heavily on the Specialized High School Admissions Test (SHSAT). However, SHSAT scores are influenced by socioeconomic status and access to test preparation, creating a disadvantage for students from lower-income backgrounds. This leads to a situation where a disproportionately small number of middle schools produce the majority of SHS admits, exacerbating segregation and limiting opportunities. While previous attempts to address this involved policy changes like quotas, they faced legal challenges and mixed results. This paper seeks to find a more effective and fair approach by focusing on pre-admission interventions.
Methods
The authors employ an "operations research" approach, deviating from traditional policy-focused methods. They develop a mathematical model of a continuous matching market with a "bias" factor. This bias represents the difference between a student's true potential and their perceived potential (measured by SHSAT scores), with disadvantaged students assumed to have their potential underestimated. They then analyze the impact of this bias on student assignments and introduce "vouchers" (representing resources like tutoring or scholarships) as interventions to "debias" students. The core of their methodology involves:
- Modeling Bias: A multiplicative bias factor ($\beta$) is introduced, where a disadvantaged student's perceived potential is $\beta$ times their true potential.
- Matching Algorithm: They use a stable matching framework, adapted for continuous markets, to model how students are assigned to schools based on perceived potential.
- Intervention Analysis: They analyze the impact of deterministic and randomized voucher allocation policies on measures of "mistreatment" (how much worse a student is assigned due to bias).
- Fairness Metrics: They focus on minimizing maximum mistreatment and the total mistreatment (Positive Area Under the Curve - PAUC).
- Empirical Validation: They use real SHSAT score data from NYC to estimate model parameters and evaluate the proposed interventions.
Datasets
The primary dataset used is SHSAT admissions data from New York City for the 2016-17 academic year. This dataset includes SHSAT scores, student preference lists for SHSs, and information on whether students are classified as "disadvantaged" by the NYC Department of Education.
Key Insights and Ideas
- Asymmetric Impact of Bias: The bias disproportionately harms disadvantaged students, with only a slight benefit to advantaged students.
- Targeting Average Performers: Interventions (vouchers) are most effective when targeted at average-performing disadvantaged students, not necessarily the top performers. This is because a small improvement for an average student can lead to a larger jump in school assignment compared to a top performer.
- Limitations of Deterministic Interventions: Deterministic voucher allocation can create incentives for students to underperform to qualify for the intervention.
- Randomization for Fairness: Randomized voucher allocation policies can be incentive-compatible and individually fair, while still effectively reducing mistreatment.
- Robustness to Model Misspecification: The core findings of the paper remain largely consistent even when the assumed bias model is slightly misspecified, suggesting the practical applicability of the proposed interventions.
Key Results
- Optimal Intervention Target: Both theoretical analysis and empirical validation suggest that vouchers should be targeted at average-performing disadvantaged students to maximize fairness gains.
- PAUC Reduction: The proposed interventions can significantly reduce aggregate mistreatment (PAUC) for disadvantaged students. For example, with a low budget (ĉ=0.1), the theoretical intervention achieved a PAUC reduction of up to 17.28%, with minimal difference compared to empirically optimal intervals.
- Robustness: Even with various forms of bias (additive, uniform noise, Gaussian noise), the theoretical model's insights on targeting average performers hold, with differences in PAUC reduction generally being small, especially for lower budgets.
- Incentive Compatibility: Deterministic policies struggle with incentive compatibility, but randomized policies (like Proportional-to-Mistreatment) can achieve this property.
Limitations and Future Work
- Ex-Post Fairness Perception: Randomized interventions might still lead to perceptions of unfairness ex-post. The authors suggest future work could explore strategies like implementing voucher programs in carefully selected schools with low economic indices.
- Definition of Disadvantaged Students: The analysis relies on the NYC DOE's definition of disadvantaged students. Changes to this definition would require recalibrating the model.
- Model Assumptions: The paper assumes a multiplicative bias and that vouchers fully debias students. Relaxing these assumptions (e.g., stochastic-effect vouchers, more general bias models) is explored but leaves room for further theoretical analysis.
- Heterogeneous Preferences: While the empirical validation uses heterogeneous preferences, the theoretical model assumes homogeneous preferences for tractability. Extending the theoretical framework to fully account for heterogeneous preferences is a direction for future work.
- Potential Distribution: The Pareto distribution is used for student potentials. Investigating the impact of other distributions on the findings is also suggested.
Summary updated