TL;DR
This paper introduces the "Sorted Effects Method" (SPE) to better understand heterogeneous treatment effects in nonlinear models. Instead of just reporting average effects, SPEs provide a full distribution of effects, revealing nuanced patterns and enabling classification of individuals into most/least affected groups. This method is applied to the gender wage gap, uncovering significant heterogeneity beyond simple averages.
What Problem Does This Paper Address?
In many real-world scenarios, the impact of a treatment or policy isn't uniform across individuals. For example, the effect of a job training program might be substantial for some but negligible for others. Traditional econometric methods often summarize these effects by reporting an "average partial effect" (APE). However, this averaging can mask crucial variations in how different individuals experience the effect. This is particularly true in nonlinear models where effects can vary dramatically with individual characteristics. The challenge is to move beyond these averages and fully characterize and visualize this heterogeneity, allowing for a deeper understanding of who benefits most and least, and why.
Methods
The core innovation is the Sorted Effects Method (SPE). Instead of just calculating an average effect, SPEs estimate and report the entire collection of partial effects, sorted in increasing order and indexed by percentiles. This creates a "sorted effect curve" that visually represents the full spectrum of heterogeneous effects.
Key methodological contributions include:
- Hadamard Differentiability: The authors develop new mathematical results on the Hadamard differentiability of multivariate sorting and classification operators. This is a crucial technical step for deriving the statistical properties of their estimators.
- Functional Delta Method: This is used to translate the properties of the estimators of the partial effects and the covariate distribution into the properties of the estimated SPEs.
- Functional Central Limit Theorems (FCLTs): They establish FCLTs for the empirical SPE-process, which are essential for constructing confidence bands and performing inference.
- Bootstrap Procedures: Practical algorithms are provided for estimating the SPEs and their confidence bands, including finite-sample bias corrections and uniform confidence bands.
- Classification Analysis (CA): The SPEs are used to classify observational units into "most affected" and "least affected" groups, allowing for a comparison of their characteristics.
Datasets
The paper uses the following datasets:
- U.S. March Supplement of the Current Population Survey (CPS) in 2015: This is the primary dataset used for the gender wage gap analysis.
- Boston mortgage applications data from 1990 (HMDA): Used for an empirical application on the effect of race on mortgage denials.
The paper also utilizes simulated data for numerical illustrations to evaluate the accuracy of their methods.
Key Insights and Ideas
- Beyond Averages: The SPE method fundamentally shifts the focus from a single average effect to the entire distribution of effects. This is crucial for understanding nuanced phenomena like the gender wage gap, where averages can be misleading.
- Visualizing Heterogeneity: The sorted effect curves provide an intuitive and powerful way to visualize the range and distribution of effects, making complex patterns accessible.
- Identifying "Who is Affected": The classification analysis allows researchers to move from understanding how effects vary to identifying who experiences them most and least, and what characteristics define these groups. This is invaluable for targeted policy interventions.
- Surprising Gender Wage Gap Patterns: The application to the gender wage gap reveals significant heterogeneity, with the gap varying dramatically for married vs. never-married women, and being particularly negative for married women without college degrees.
Key Results
- Gender Wage Gap Heterogeneity: The analysis of the gender wage gap using CPS data shows substantial heterogeneity. For married women, the gap ranges from -60% to 0% across percentiles, while for never-married women, it ranges from -40% to +20%.
- Characteristics of Most Affected Women: The most adversely affected women in the gender wage gap analysis tend to be married, lack college degrees, work in sales, and have high potential experience.
- Mortgage Denial Disparities: In the mortgage denial application, while black applicants are, on average, more likely to be denied, the SPE analysis reveals that this effect is not uniform. There exist subgroups of applicants who are significantly more likely to be denied if black, while others are unaffected by this racial gap.
- Methodological Validation: Numerical simulations demonstrate that the proposed SPE method and its associated confidence intervals perform well, providing accurate coverage even in cases with critical points in the PE function.
Limitations and Future Work
- Regularity Conditions: The theoretical results rely on certain regularity conditions for the partial effect function (e.g., being smooth and not "locally flat"). The authors acknowledge that their current inference methods do not apply when these conditions are violated and suggest future work to develop more robust inference procedures.
- Discrete Covariates: While the paper extends the analysis to include discrete covariates, the theoretical treatment is more complex and might require further development.
- Computational Complexity: While bootstrap is used to make inference practical, the underlying mathematical machinery involving differential geometry and manifold integration can be computationally intensive.
- Causal Interpretation: The paper focuses on "predictive effects," and while causal interpretations are possible under certain assumptions, the primary focus is on characterizing observed heterogeneity.
Future work could explore inference methods that are more robust to violations of regularity conditions and further extend the theoretical framework to more complex data structures.
Summary updated