Correcting the estimator for the mean vectors in a multivariate errors-in-variables regression model

Johannes F. Lutzeyer, Edward A. K. CohenarXiv 2015 · DOI · arXiv

TL;DR

This paper identifies and fixes a mathematical error in a foundational 1981 study on multivariate "errors-in-variables" regression. The authors provide the corrected formulas for estimating mean vectors—the true, unobserved values behind noisy data—ensuring that researchers using these models get consistent and unbiased results.

What Problem Does This Paper Address?

In standard linear regression, we usually assume our independent variables ($X$) are measured perfectly, while only the dependent variables ($Y$) contain noise. However, in many real-world scenarios—like economics or biology—both sets of variables are subject to measurement error. This is known as an errors-in-variables (EIV) model.

Using traditional least squares on EIV models leads to biased and inconsistent results. In 1981, Leon Jay Gleser published a seminal paper providing the "correct" ways to estimate these parameters using Maximum Likelihood (MLE) and Ordinary Least Squares (OLSE). For over 30 years, Gleser’s paper has been the go-to reference. However, Lutzeyer and Cohen discovered that Gleser’s formulas for the "mean vectors" (the latent, error-free values of the variables) were incorrectly derived for models that include an intercept. This paper sets the record straight.

Methods

The paper is a rigorous mathematical derivation. The authors revisit the original optimization problem posed by Gleser: minimizing the "orthogonally invariant norm" of the residual matrix.

Key steps include:

  1. Defining the Model: They represent the observed data as a combination of true mean vectors ($U$) and random error ($E$).
  2. Eigen-decomposition: They utilize the eigen-decomposition of the data covariance matrix ($W = XC_nX'$) to partition the space into signal and noise components.
  3. Algebraic Expansion: By expanding the estimator for the mean vectors ($U_1$), they demonstrate exactly where the original 1981 derivation missed a term.
  4. Correction: They derive a new, simplified expression that accounts for the intercept term ($\alpha$), which was the source of the original error.

Datasets

Not discussed in the paper. This is a theoretical mathematical correction and does not use empirical datasets or simulations.

Key Insights and Ideas

  • The "No-Intercept" Exception: The authors confirm that Gleser’s original formula was actually correct for models without an intercept. The error only creeps in when a constant offset ($\alpha$) is introduced.
  • The Missing Term: The discrepancy comes down to a specific term: $n^{-1}X_1\mathbf{1}_n\mathbf{1}_n'$. This term essentially handles the centering of the data. Without it, the estimator for the mean vectors doesn't properly align with the estimated slope and intercept.
  • Unified Correction: Because the Maximum Likelihood Estimator (MLE) and the Ordinary Least Squares Estimator (OLSE) for these mean vectors are mathematically related in this context, fixing the OLSE formula automatically corrects the MLE as well.

Key Results

The primary result is the corrected estimator for the matrix of mean vectors, $U_1$: $$\hat{U}_1(\hat{\alpha}, \hat{B}) = n^{-1}X_1\mathbf{1}_n\mathbf{1}_n' + (G_{11}G_{11}' X_1 + G_{11}G_{21}' X_2) C_n$$

Compared to the original 1981 version, this formula includes an additional centering term (the first part of the equation) and correctly applies the centering matrix $C_n$ to the second part. This ensures the estimator is mathematically consistent with the rest of the regression model.

Limitations and Future Work

As a "short paper," the scope is strictly limited to correcting the mathematical derivation.

  • Numerical Impact: The authors do not perform simulations to show how much this error actually affects practical results. In many cases, the difference might be small, but in high-precision applications, it could be significant.
  • Historical Audit: The paper suggests that previous work relying on Gleser’s specific mean vector formulas may need to be re-evaluated, though it doesn't identify specific studies affected.
  • Future Directions: Future research could investigate if similar subtle errors exist in other variations of EIV models, such as those with non-Gaussian errors or heteroscedasticity.

Summary updated

Abstract

The multivariate errors-in-variables regression model is applicable when both dependent and independent variables in a multivariate regression are subject to measurement errors. In such a scenario it is long established that the traditional least squares approach to estimating the model parameters is biased and inconsistent. The generalized least squares, ordinary least squares and maximum likelihood estimators (under the assumption of Gaussian errors) were derived in the seminal paper of Gleser (1981). However, the ordinary least squares and maximum likelihood estimators for the mean vectors were incorrectly derived. In this short paper we amend this error, presenting the correct estimators of the mean vectors.
BibTeX
@misc{lutzeyer2015correcting,
  author = {Johannes F. Lutzeyer and Edward A. K. Cohen},
  title = {{Correcting the estimator for the mean vectors in a multivariate errors-in-variables regression model}},
  year = {2015},
  eprint = {1510.03600},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1510.03600},
}
Figure