Abstract
Let $f(y|\theta), \; \theta\in \Omega$ be a parametric family, $\eta(\theta)$ a given function, and $G$ an unknown mixing distribution. This paper explores the estimation of the population mean $E_G (\eta(\theta)) \equiv \eta_G$ based on independent observations $Y_1,...,Y_n$, where $Y_i \sim f(y|\theta_i)$ and $\theta_i \sim G$ are i.i.d. We investigate the Generalized Maximum Likelihood Estimator (GMLE) for this problem, demonstrating its properties, its relationship to non-parametric empirical Bayes estimation under squared loss, and its application to sampling problems. The performance of the GMLE is validated through simulations and a real-world analysis of home ownership in Tel-Aviv.
Introduction
In statistical inference, we frequently encounter "mixture models" where a population is composed of heterogeneous subgroups. For example, in a large-scale survey across multiple neighborhoods, each neighborhood may possess a unique characteristic $\theta_i$ (such as an unemployment rate), but we only observe a limited number of samples $Y_i$ from each.
A significant challenge in these settings is the presence of "empty strata"—subgroups from which no data is collected. Standard "naive" estimators often ignore these strata, which can introduce substantial bias if the probability of a subgroup being sampled is correlated with its underlying parameter. This paper proposes the use of the Generalized Maximum Likelihood Estimator (GMLE) to robustly estimate the global mean of a function of these parameters, even when the underlying distribution $G$ is unknown and data is sparse.
The GMLE Framework
The Generalized Maximum Likelihood Estimator, also referred to as the Nonparametric Maximum Likelihood Estimator (NPMLE), does not assume a specific functional form (e.g., a Normal distribution) for the mixing distribution $G$. Instead, it searches the space of all possible probability distributions to identify the one that maximizes the likelihood of the observed data.
The Empirical Bayes Connection
A key theoretical contribution of this work is the link between GMLE and Empirical Bayes (EB) methods. We demonstrate that the GMLE for the population mean $\eta_G$ is mathematically equivalent to the average of individual "plug-in" Empirical Bayes estimates. This result bridges the gap between global population-level inference and local inference regarding specific individuals or subgroups.
Theoretical Foundations
We extend the weak convergence results established by Kiefer and Wolfowitz (1956) to a "compound decision" framework. This extension shows that the GMLE remains effective even in scenarios where the parameters $\theta_1, ..., \theta_n$ are treated as fixed constants rather than random variables sampled from $G$. Computationally, the GMLE is implemented using an Expectation-Maximization (EM) algorithm over a fine grid of potential parameter values, leveraging modern convex optimization techniques.
Applications in Sampling and Non-Response
The utility of the GMLE is most apparent in sampling scenarios involving non-response. If response rates are lower in certain strata (e.g., high-unemployment areas), a simple sample mean will be biased. The GMLE addresses this by learning the distribution of parameters from the responding groups and using that information to "fill in" the missing data for the non-responding groups.
In simple exponential families, such as estimating the mean of Normal distributions, the GMLE naturally reduces to the sample mean, satisfying a "do no harm" principle. However, in more complex cases like Poisson or Binomial mixtures with varying sample sizes, the GMLE provides a significant correction over naive methods.
Empirical Evaluation
Simulation Studies
The researchers conducted simulations using Poisson and Binomial distributions to compare the GMLE against naive estimators (which ignore empty groups) and "extreme collapse" estimators (which treat the entire population as a single homogeneous group). The results indicated that when response rates were correlated with the underlying parameters, the naive estimator exhibited significant bias, whereas the GMLE remained consistently close to the true population mean.
Tel-Aviv Case Study
The method was applied to the Israeli Social Survey (2015–2017) to estimate home ownership rates across 156 statistical areas in Tel-Aviv. Because homeowners are generally more likely to respond to surveys than renters, a standard aggregate estimate (0.488) likely overestimates ownership. The GMLE produced an estimate of 0.434, which aligned closely with stratified benchmarks, demonstrating its ability to correct for response bias under the "missing at random" (MAR) assumption.
Discussion and Limitations
While the GMLE is a powerful tool for mixture models, it is not without challenges:
- Computational Complexity: The EM algorithm on high-dimensional grids can be computationally intensive and requires careful selection of grid boundaries.
- Identifiability: In certain models, such as the Binomial case with fixed sample sizes, the distribution $G$ may not be fully identifiable. While the GMLE still performs well for estimating the mean, other functionals may be more difficult to recover.
- Inference: Constructing confidence intervals for these estimates remains a challenging task. While the authors suggest a method using convex optimization, its practical implementation is a subject for future research.
In conclusion, the GMLE provides a robust, non-parametric framework for estimating population means in the presence of heterogeneity and missing data, offering a significant improvement over traditional estimators in complex sampling designs.
Summary updated