Estimation of a score-explained non-randomized treatment effect in fixed and high dimensions

Debarghya Mukherjee, Moulinath Banerjee, Ya’acov Ritov

Abstract

In fields ranging from political science to psychometry, treatment is often assigned based on specific individual characteristics rather than random allocation—such as scholarships awarded for merit or medical interventions based on physiological thresholds. Traditional methods for analyzing these non-randomized treatments, like Regression Discontinuity Design (RDD), frequently suffer from slow convergence rates (slower than $\sqrt{n}$) and are limited to estimating local effects near the assignment threshold. This paper introduces SCENTS (Score Explained Non-Randomized Treatment Systems), a novel framework that achieves the optimal $\sqrt{n}$ convergence rate and provides global treatment effect estimates. By leveraging background covariates to explain the assignment score, SCENTS offers a more precise and scalable approach for both fixed and high-dimensional data settings.

Introduction

In a randomized controlled trial (RCT), treatment assignment is independent of a subject's characteristics. However, in observational studies and social policy, treatment is often endogenous: the reason an individual receives a treatment (e.g., high academic ability) is frequently correlated with the outcome (e.g., future earnings).

To address this, researchers have traditionally relied on RDD. While useful, RDD has two primary drawbacks. First, it is "local," meaning it only estimates the treatment effect for individuals right at the cutoff point. Second, because it relies on a narrow bandwidth of data, the estimator is statistically "noisy" and converges slowly as sample sizes increase. The SCENTS framework overcomes these hurdles by modeling the assignment mechanism across the entire population.

The SCENTS Methodology

The core innovation of SCENTS is the utilization of background data to explain the "score" variable $Q$ that determines treatment assignment. The system is modeled as a partial linear model where the outcome $Y$ depends on the treatment, linear covariates, and a non-linear "nuisance" function representing the noise in the score assignment.

Key Components of the Estimator

  1. B-Spline Approximation: To handle the non-linear relationship between the score's noise and the outcome without imposing rigid parametric assumptions, the framework employs B-splines to approximate the nuisance function.
  2. Three-Part Data Splitting: To ensure the estimator remains unbiased, the algorithm partitions the dataset into three distinct segments:
  3. Part 1: Estimation of the relationship between background covariates ($Z$) and the assignment score ($Q$).
  4. Part 2: Estimation of the non-linear nuisance function.
  5. Part 3: Calculation of the final treatment effect ($\alpha_0$).
  6. High-Dimensional Scaling: In modern contexts where the number of variables ($p$) may exceed the number of observations ($n$), SCENTS incorporates a de-biasing procedure using LASSO regression. This allows the model to maintain $\sqrt{n}$ consistency even in high-dimensional covariate spaces.

Empirical Applications

The authors validated the SCENTS framework using two distinct real-world datasets, comparing the results against traditional RDD approaches.

Turkey Municipality Voting Data

This study investigated whether the election of Islamic versus secular mayors in 1994 influenced women’s high school education rates by the year 2000 ($n = 2629$). Previous RDD-based studies suggested that Islamic party rule had a significantly positive effect on women's education. However, the SCENTS analysis found no significant effect. This discrepancy suggests that the RDD results may have been driven by specific conditions in "cliff-hanger" elections, whereas SCENTS provides a more robust "global" view of the social impact across all municipalities.

Canadian University GPA Data

This analysis focused on the effect of academic probation (triggered by a GPA cutoff) on subsequent student performance. SCENTS confirmed a positive and highly significant treatment effect, supporting the "wake-up call" hypothesis—that the threat of expulsion motivates students to improve their academic standing. Unlike RDD, SCENTS was able to provide a more stable estimate by utilizing the full range of student data.

Key Insights and Advantages

  • Global vs. Local Inference: By modeling the score variable across its entire range, SCENTS moves beyond the boundary to provide a treatment effect applicable to the broader population.
  • Statistical Efficiency: The authors demonstrate that the SCENTS estimator is semi-parametrically efficient under normal errors. This means it achieves the highest possible precision for this class of statistical problems.
  • Mathematical Convergence: The framework rigorously achieves the $\sqrt{n}$ convergence rate, making it significantly more stable than non-parametric RDD methods, which typically converge at a slower rate (e.g., $n^{-2/5}$ or $n^{-4/9}$).

Limitations and Future Directions

While powerful, the SCENTS framework relies on the availability of "good" covariates—background variables that can effectively explain the assignment score. If the score is determined by unobserved factors or is purely random, the benefits of the model are reduced. Additionally, the current primary model assumes a constant treatment effect across the population. Future work may focus on expanding the framework to better accommodate heterogeneous treatment effects where the impact varies based on individual "innate ability" or other latent characteristics. Finally, because calculating asymptotic variance is complex, the method currently utilizes bootstrapping for confidence intervals, which can be computationally intensive for extremely large datasets.

Summary updated

Abstract

Non-randomized treatment effect models are widely used for the assessment of treatment effects in various fields and in particular social science disciplines like political science, psychometry, psychology. More specifically, these are situations where treatment is assigned to an individual based on some of their characteristics (e.g. scholarship is allocated based on merit or antihypertensive treatments are allocated based on blood pressure level) instead of being allocated randomly, as is the case, for example, in randomized clinical trials. Popular methods that have been largely employed till date for estimation of such treatment effects suffer from slow rates of convergence (i.e. slower than $\sqrt{n}$). In this paper, we present a new model coined SCENTS: Score Explained Non-Randomized Treatment Systems, and a corresponding method that allows estimation of the treatment effect at $\sqrt{n}$ rate in the presence of fairly general forms of confoundedness, when the `score' variable on whose basis treatment is assigned can be explained via certain feature measurements of the individuals under study. We show that our estimator is asymptotically normal in general and semi-parametrically efficient under normal errors. We further extend our analysis to high dimensional covariates and propose a $\sqrt n$ consistent and asymptotically normal estimator based on a de-biasing procedure. Our analysis for the high dimensional incarnation can be readily extended to analyze partial linear models in the presence of noisy variables corresponding to the non-linear part of the model, where the noise can be correlated with the variables corresponding to the linear part. We analyze two real datasets via our method and compare our results with those obtained by using previous approaches. We conclude this paper with a discussion on some possible extensions of our approach.
BibTeX
@misc{mukherjee2021estimation,
  author = {Debarghya Mukherjee and Moulinath Banerjee and Ya’acov Ritov},
  title = {{Estimation of a score-explained non-randomized treatment effect in fixed and high dimensions}},
  year = {2021},
  eprint = {2102.11229},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.11229},
}
Figure