Generalized Maximum Likelihood Estimators and their applications to stratified sampling and post-stratification with many unobserved strata

Eitan Greenshtein, Ya’acov RitovarXiv 2019 · DOI · arXiv

Authors: Eitan Greenshtein and Ya’acov Ritov

Abstract

Consider the problem of estimating a weighted average of the means of $n$ strata, based on a random sample with realized $K_i$ observations from stratum $i, \; i=1,...,n$. This task is non-trivial in cases where for a significant portion of the strata the corresponding $K_i=0$. Such a situation may happen in post-stratification, when it is desired to have a very fine stratification. A fine stratification could be desired in order that assumptions, or, approximations, like Missing At Random conditional on strata, will be appealing. A fine stratification could also be desired in observational studies, when it is desired to estimate average treatment effect, by averaging the effects in small and homogenous strata. Our approach is based on applying Generalized Maximum Likelihood Estimators (GMLE), and ideas that are related to Non-Parametric Empirical Bayes, in order to estimate the means of strata $i$ with corresponding $K_i=0$. There are no assumptions about a relation between the means of the unobserved strata (i.e., with $K_i=0$) and those of the observed strata. The performance of our approach is demonstrated both in simulations and on a real data set. Some consistency and asymptotic results are also presented. In addition, related basic results about GMLE estimation of the mean of mixtures of exponential families are provided.

Introduction

In survey statistics, researchers often face a dilemma when using stratification to improve accuracy. To satisfy the "Missing At Random" (MAR) assumption, it is often necessary to employ "fine stratification"—breaking the population into very small, homogenous subgroups. However, the finer the stratification, the more likely it is that some strata will end up with zero observations (unobserved strata).

Traditional "naive" estimators typically ignore these empty cells. This introduces significant bias if the reason a stratum is empty (e.g., low response rates) is correlated with the variable being measured. While ad-hoc fixes like "collapsing" strata exist, they are often arbitrary. This paper proposes a formal framework using Generalized Maximum Likelihood Estimators (GMLE) to account for these unobserved strata without making rigid assumptions about their relationship to observed ones.

Methodology

The authors leverage Non-Parametric Empirical Bayes (NPEB) ideas to estimate population means. Instead of assuming a specific distribution for stratum parameters, they treat the parameters of each stratum (such as the mean and response probability) as latent variables drawn from an unknown distribution $G$.

The GMLE is used to estimate $G$ non-parametrically. The study focuses on two primary scenarios:

  1. Stratified sampling with non-response: Modeled using a Binomial distribution for the number of observations.
  2. Post-stratification: Modeled using a Poisson distribution.

Computationally, the authors implement the EM-algorithm on a 2D grid (40x40) to find the GMLE. This allows for the estimation of means in unobserved strata based on the overall distribution of parameters learned from the observed data.

Key Results and Simulation Performance

The performance of the GMLE approach was evaluated against naive estimators using both synthetic and real-world data. In simulations involving 1,000 strata, the GMLE demonstrated remarkable robustness.

A critical finding is that the GMLE significantly reduces bias in scenarios where response rates are lower in strata with specific means. For instance, in one simulation where the true value was 0.5, the naive estimator produced a biased result of 0.385, while the GMLE successfully pulled the estimate back to 0.505.

Comparison of Naive and GMLE estimators across different strata parameter configurations.

In real-world applications using the Israeli "Social Survey" (2015–2017) to estimate home ownership in Tel-Aviv, the GMLE remained stable even when high rates of missing data were simulated. Furthermore, the authors proved a theoretical "Naive Equivalence": if all strata have at least one observation, the GMLE reduces to the naive estimator, ensuring the sophisticated machinery only activates when unobserved strata are present.

Insights and Limitations

While the GMLE performs well in practice, the paper notes some theoretical nuances. In the Poisson scenario (post-stratification), the GMLE is consistent. However, in the Binomial scenario (non-response), it is not strictly consistent due to identifiability issues—meaning multiple underlying distributions could explain the same finite outcomes when sample sizes per stratum are very small.

Despite these theoretical limitations, the empirical results suggest that GMLE is a powerful tool for handling fine stratification. Future work may involve developing confidence intervals for the population average by solving convex optimization problems to address the consistency challenges in the Binomial case.

Summary updated

Abstract

Consider the problem of estimating a weighted average of the means of $n$ strata, based on a random sample with realized $K_i$ observations from stratum $i, \; i=1,...,n$. This task is non-trivial in cases where for a significant portion of the strata the corresponding $K_i=0$. Such a situation may happen in post-stratification, when it is desired to have a very fine sftratification. A fine stratification could be desired in order that assumptions, or, approximations, like Missing At Random conditional on strata, will be appealing. A fine stratification could also be desired in observational studies, when it is desired to estimate average treatment effect, by averaging the effects in small and homogenous strata. Our approach is based on applying Generalized Maximum Likelihood Estimators (GMLE), and ideas that are related to Non-Parametric Empirical Bayes, in order to estimate the means of strata $i$ with corresponding $K_i=0$. There are no assumptions about a relation between the means of the unobserved strata (i.e., with $K_i=0$) and those of the observed strata. The performance of our approach is demonstrated both in simulations and on a real data set. Some consistency and asymptotic results are also presented. In addition, related basic results about GMLE estimation of the mean of mixtures of exponential families are provided.
BibTeX
@misc{greenshtein2019generalized,
  author = {Eitan Greenshtein and Ya’acov Ritov},
  title = {{Generalized Maximum Likelihood Estimators and their applications to stratified sampling and post-stratification with many unobserved strata}},
  year = {2019},
  eprint = {1909.09336},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.09336},
}
Figure