Increasing dimension asymptotics for two-way crossed mixed effect models

Ziyang Lyu, Scott A. Sisson, A. H. WelshThe Annals of Statistics 2024 · DOI

TL;DR

This paper develops the asymptotic theory for estimating parameters in two-way crossed mixed-effects models, even when the number of rows, columns, and observations per cell all grow infinitely large. This work provides a robust framework for analyzing complex hierarchical data structures where traditional asymptotic assumptions don't hold, leading to reliable statistical inferences.

What Problem Does This Paper Address?

Mixed-effects models are powerful tools for analyzing clustered or longitudinal data. However, standard asymptotic theory for these models often assumes either a fixed number of clusters with increasing cluster size, or a fixed cluster size with increasing number of clusters. Real-world scenarios, like analyzing educational data where students are grouped by teachers and classes, often involve "crossed" random effects (e.g., teachers teaching multiple classes, and classes having multiple teachers). In such cases, both the number of "clusters" (rows and columns) and the number of observations within these clusters (cells) can grow simultaneously. Existing asymptotic results for mixed models with crossed random effects are limited, particularly for maximum likelihood (ML) and restricted maximum likelihood (REML) estimators, and often require restrictive assumptions. This paper aims to fill this gap by providing asymptotic results for these estimators under more general conditions, allowing for the simultaneous, unrestricted growth of all dimensions of the data.

Methods

The authors tackle this problem by first defining a two-way crossed random-effects model with interaction. They then derive the maximum likelihood (ML) and restricted maximum likelihood (REML) estimating equations. The core of their contribution lies in establishing asymptotic normality for these estimators as the number of rows ($g$), columns ($h$), and observations per cell ($m$) all tend to infinity, without imposing restrictions on their relative growth rates. They achieve this by:

  • Developing a structured covariance matrix: They show that the asymptotic covariance matrix of the estimators has a specific, interpretable block-diagonal structure.
  • Using mild conditions: Instead of assuming normality for random effects and errors, they rely on finite moment conditions (specifically, existence of finite "4+δ" moments).
  • Leveraging Kronecker products: The mathematical framework heavily utilizes Kronecker products for efficient matrix manipulation, particularly for the inverse of the dispersion matrix.
  • Establishing asymptotic equivalence: They demonstrate that the REML estimators are asymptotically equivalent to the ML estimators.
  • Providing asymptotic representations: They derive explicit representations for the estimators in terms of their influence functions, which are crucial for constructing confidence intervals and understanding their behavior.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and uses simulated data for validation.

Key Insights and Ideas

  • Simultaneous growth of dimensions: The paper's most significant contribution is handling the simultaneous, unrestricted growth of $g$, $h$, and $m$. This is crucial for many real-world applications where all aspects of the data structure are large.
  • Structured asymptotic covariance: The resulting asymptotic covariance matrix is not just theoretically derived but also has a clear structure, making it interpretable and useful for practical inference.
  • Robustness to normality: The reliance on moment conditions rather than strict normality assumptions makes the results more broadly applicable to a wider range of data distributions.
  • Orthogonality of estimators: The paper highlights the asymptotic orthogonality between different sets of estimators (row, column, interaction, and within-cell), simplifying inference, with specific dependencies noted when $g$ and $h$ grow at the same rate.

Key Results

The paper establishes that under mild conditions, the ML and REML estimators for the parameters in a two-way crossed mixed-effects model with interaction are asymptotically normal. This holds even when the number of rows, columns, and observations per cell all tend to infinity without restriction on their relative rates. The estimators possess a structured asymptotic covariance matrix. The paper also provides explicit asymptotic representations for these estimators, which are essential for constructing confidence intervals. Simulation results (Tables 1 and 2) demonstrate that the theoretical asymptotic coverage and length of confidence intervals generally align with empirical findings, especially as $g$, $h$, and $m$ increase.

Limitations and Future Work

  • Balanced Case Focus: The derivation of the exact inverse of the dispersion matrix relies on the balanced case (equal number of observations $m$ in each cell). While the authors believe their results are indicative of the unbalanced case, this remains a simplification.
  • Computational Complexity: While the theoretical framework is established, the computational implementation for very large datasets can still be challenging.
  • Prediction of Random Effects: The paper notes that an infinite increase in $m$ is necessary for consistent prediction of random effects, a topic they suggest for future work.
  • Extension to GLMMs: The paper mentions that deriving asymptotic behavior for generalized linear mixed effect models (GLMMs) with crossed random effects is particularly difficult, suggesting this as a potential area for future research.

Summary updated

Abstract

This paper presents asymptotic results for the maximum likelihood and restricted maximum likelihood (REML) estimators within a two-way crossed mixed effect model, when the number of rows, columns, and the number of observations per cell tend to infinity. The relative growth rate for the number of rows, columns, and cells is unrestricted, whether considered pairwise or collectively. Under very mild conditions (which include moment conditions instead of requiring normality for either the random effects or errors), the estimators are proven to be asymptotically normal, with a structured covariance matrix. We also discuss the case where the number of observations per cell is fixed at 1.
BibTeX
@article{lyu2024increasing,
  author = {Ziyang Lyu and Scott A. Sisson and A. H. Welsh},
  title = {{Increasing dimension asymptotics for two-way crossed mixed effect models}},
  journal = {The Annals of Statistics},
  year = {2024},
  doi = {10.1214/24-aos2469},
}
Figure