Abstract
In many practical research settings, missing data is an unavoidable challenge that can compromise the validity of statistical inferences. Deciding whether data is Missing at Random (MAR) or Missing Not at Random (MNAR) is a critical step in choosing an appropriate analytical model. This paper introduces a new semiparametric hypothesis testing approach to distinguish between these two mechanisms. By utilizing instrumental variables and a discrepancy measure between a MAR-specific estimator and a robust MNAR-capable estimator, the proposed test provides a data-driven method for verifying the integrity of missing data assumptions. The efficacy of this approach is demonstrated through theoretical analysis, simulation studies, and an application to a real-world weight-loss clinical trial.
Introduction
Missing data is a ubiquitous problem in fields ranging from clinical trials to social sciences. Statisticians generally classify missingness into three categories: Missing Completely at Random (MCAR), where missingness is independent of any data; Missing at Random (MAR), where missingness depends on observed data; and Missing Not at Random (MNAR), where the probability of data being missing depends on the unobserved values themselves.
While MAR is the standard assumption for most modern statistical software and analyses, it is often an assumption of convenience rather than a verified fact. If data is actually MNAR, applying MAR-based methods leads to biased estimates and incorrect conclusions. However, testing for MNAR is notoriously difficult because the information required to prove the mechanism is, by definition, missing. This research addresses this gap by providing an objective, semiparametric test to determine if the MAR assumption holds or if a more complex MNAR model is required.
Methodology
The Discrepancy Measure
The core of the proposed test is a "discrepancy measure" strategy, conceptually similar to the classic Hausman test. The researchers identify two distinct estimators for the model parameters ($\beta$):
- Estimator A ($\hat{\beta}$): An Inverse Probability Weighting (IPW) estimator. This estimator is efficient and valid under the MAR assumption but becomes biased ("breaks") if the data is MNAR.
- Estimator B ($\tilde{\beta}$): A semiparametric estimator (following Zhao and Shao, 2015) that remains consistent under both MAR and MNAR conditions.
If the MAR assumption is true, both estimators should converge to the same value. If the data is MNAR, the two estimators will diverge significantly. The difference between these two estimators forms the basis of the test statistic.
Instrumental Variables (IV)
To identify the model under MNAR conditions, the approach utilizes an Instrumental Variable ($Z$). This is a variable that is related to the outcome of interest but is independent of the missingness mechanism once other covariates are accounted for. The IV acts as a bridge, allowing the researchers to bypass the "unobserved" problem inherent in MNAR data.
Semiparametric Flexibility
Unlike previous testing attempts that required strict parametric assumptions about the missingness mechanism, this method is semiparametric. It uses kernel regression to handle the "nuisance" of the missingness mechanism nonparametrically. This ensures that the test results are not artifacts of a poorly specified mathematical model for why data is missing.
Empirical Results
Simulation Studies
The authors conducted extensive simulations using synthetic multivariate normal distributions. The results showed that the test effectively controlled the Type I error rate (maintaining it at approximately 5%) when the data was truly MAR. Furthermore, the "power" of the test—its ability to correctly detect MNAR—increased significantly as the dependence of missingness on the unobserved values grew stronger.
Real-World Application: The "Keep It Off" Study
The test was applied to the "Keep It Off" study, a randomized controlled trial involving 191 participants in a weight-loss program. In this study, participants reported weights via wireless scales, but many daily entries were missing. Because the study also included "gold standard" in-person weights at the six-month mark, it provided a unique opportunity to validate the test.
The proposed hypothesis test yielded a p-value of 0.02, successfully rejecting the MAR assumption. This confirmed the researchers' suspicion: participants who were gaining weight were less likely to step on the scale. While a standard MAR-based analysis overestimated the success of the weight-loss program, the MNAR-robust model (validated by the in-person weights) provided a much more accurate and sober assessment of the trial's outcomes.
Conclusion and Future Work
This semiparametric testing approach provides a vital tool for researchers to move beyond guesswork when handling missing data. By providing a formal p-value for the MAR assumption, it allows for more rigorous and transparent data analysis.
However, the method does have limitations. Its effectiveness depends on the availability of a strong Instrumental Variable; if the IV is weakly related to the outcome, the test's power diminishes. Future research may extend this framework to handle unknown dispersion parameters in generalized linear models and scenarios where covariates, rather than just the response variable, are missing.
Summary updated