Economists who estimate how a firm's or county's past shapes its present usually rely on System GMM (generalized method of moments). This method sidesteps feedback between cause and effect by using the unit's own older history as a stand-in for outside instruments. Its key assumption is that lagged values two or more periods back are uncorrelated with today's shocks. Nothing in the method says how far back to go. Analysts pick a lag depth, decide whether to "collapse" the instrument matrix, and check that standard diagnostic tests do not reject. They then report one coefficient.
We argue that this single choice hides a Rashomon effect: many instrument specifications pass every diagnostic yet give materially different causal estimates. We further argue that the spread grows with lag depth for a specific reason, which is that deep lags go stale. When a structural break falls between an old lag and the present, that lag can become correlated with today's shock, so a "valid" internal instrument quietly becomes invalid. We propose to measure this spread and to replace hand-picked lag depth with a data-driven stopping rule. The rule borrows the bias–variance window selection used for model assessment under temporal shift. It adds lags only while the new estimate stays statistically consistent with the estimates from shallower, fresher lags.
Core Claim
The claim has two parts.
The first is diagnostic. For realistic panels, define the admissible set as all System GMM specifications that pass the standard battery and forecast held-out periods within a tolerance $\epsilon$ of the best specification. The range of the persistence coefficient $\hat\gamma$ and the policy coefficient $\hat\beta$ across this set is wide enough to change a paper's substantive conclusion. That range grows with maximum lag depth when structural breaks are present, and shrinks with lag depth when they are absent. This contrast is what separates stale-lag invalidity from the familiar many-instruments bias.
The second is constructive. A Lepski-type rule over lag depth recovers most of the accuracy of an oracle that knows the correct depth, and yields confidence intervals with close to nominal coverage. Current practice and LASSO-based instrument selection do not.
The predictive multiplicity literature supplies the lens here. Predictive Multiplicity in Classification defined ambiguity and discrepancy over an $\epsilon$-level set of near-optimal classifiers and showed they are large on recidivism data. The Role of Hyperparameters in Predictive Multiplicity showed that innocuous pipeline choices act as arbitrary tie-breakers. Be Intentional About Fairness! showed that picking a random member of that set, rather than choosing intentionally, carries a large hidden cost. It also left open whether selections made on static data survive temporal shift.
All three papers work with classifiers on exchangeable data. We transfer the idea to specifications of a causal estimator, where "near-equivalent" means passing the same diagnostics. We also make time the axis along which multiplicity grows.
Relation to Prior Work
The closest work is Arellano-Bond LASSO. It attacks instrument proliferation in long panels by selecting a sparse instrument subset with LASSO, then cross-fitting with forward orthogonal deviations. On a 2,510-county, 32-week COVID panel, it replaced "implausibly large" standard estimates with stable ones.
That paper supplies our strongest baseline and part of our motivation. Its diagnosis concerns overfitting of the weighting matrix, which is an $O(T/N)$ bias that exists even under stationarity. It does not ask whether deep lags are invalid rather than merely numerous. It also returns a single selection even though collinear lags admit many near-equivalent subsets. Finally, it covers the differenced formulation only, not the stacked System GMM design. Our proposal differs on all three counts.
The applied System GMM papers serve as our test beds and as evidence of the practice we question. Each reports one specification over periods containing plausible breaks. Financial distress and earnings management covers 200 firms on the Ho Chi Minh Stock Exchange over 2011–2020. Bank performance and non-interest income diversification covers 159 Chinese banks over 2006–2020, spanning the 2008 crisis.
The temporal obsolescence literature supplies both the mechanism and the method. Wild-Time documents that degradation is monotone in temporal distance, with an average 20% drop from in-distribution to future years. That is the empirical basis for expecting older lags to be less valid. Wild-Time evaluates predictors, not instruments. Model Assessment and Selection under Temporal Distribution Shift adapts the Goldenshluger–Lepski method to choose how much history to use in evaluation. Its rule keeps enlarging the window while doing so does not introduce detectable bias. We reuse that logic, but the "window" becomes the set of lags admitted as instruments, and the quantity being stabilized is a causal coefficient rather than a loss.
Drift-Resilient TabPFN models drift by extrapolating a conditional predictor forward. That is a different goal from ours, since it neither estimates structural parameters nor corrects for feedback. We therefore do not use it as a competitor.
Research Program
The object of study is a family of System GMM specifications indexed by three choices: maximum lag depth $L$, whether instruments are collapsed, and the start of the estimation sample. For each specification $s$ we record five quantities. These are the two-step estimate $\hat\theta_s = (\hat\gamma_s, \hat\beta_s)$, the Hansen $J$ p-value, the difference-in-Hansen p-value for the levels moments, the second-order serial correlation test, and a held-out forecast loss $\ell_s$. The loss is the one-step-ahead squared error on the final periods, cross-fitted over entities so that no unit contributes to both fitting and scoring.
The admissible set is
$$\mathcal{A}(\epsilon) = \{ s : s \text{ passes all diagnostics},\ \ell_s \le \min_{s'} \ell_{s'} + \epsilon \},$$
and the headline multiplicity statistic is the estimate range, $\max_{s \in \mathcal{A}(\epsilon)} \hat\beta_s - \min_{s \in \mathcal{A}(\epsilon)} \hat\beta_s$. We report this range in units of the median standard error. We also report the fraction of admissible specifications whose 95% interval excludes zero with the opposite sign to the modal specification. That fraction is the analogue of discrepancy from Predictive Multiplicity in Classification.
The proposed estimator, which we call Lepski lag depth, orders the depths $L = 2, 3, \dots, L_{\max}$. Adding lags reduces variance but, under breaks, adds bias. The rule selects
$$\hat L = \max\{ L : |\hat\beta_L - \hat\beta_{L'}| \le \kappa\, \hat\sigma_{L'} \ \text{for all } L' < L \},$$
where $\hat\sigma_{L'}$ is the Windmeijer-corrected standard error at depth $L'$. The threshold $\kappa$ is calibrated by a block bootstrap over entities, so the false-stopping rate under no break is controlled at 5%.
The same rule can run over the sample start date, which trims obsolete periods rather than obsolete instruments. We evaluate that variant separately, because it is a different fix for the same problem. We will state conditions under which $\hat L$ attains oracle-like error. The expected form is the Lepski rate: error within a log factor of the best achievable bias-plus-standard-deviation over depths, following the argument used for adaptive windows by Han et al.
The main weakness is that the shallowest depths can suffer weak-instrument bias. That would violate the rule's premise that shallow estimates are approximately unbiased. System GMM's levels moments are the reason to hope this stays small, and the simulations test it directly.
Baselines
The naive reference point is the fixed-effects within estimator with cluster-robust errors. It is known to carry Nickell bias of order $1/T$. It anchors how bad doing nothing is, and it runs on identical panels.
Current practice is represented by two-step System GMM with collapsed instruments and lags capped at 2–3. This is what most applied papers in the file report. The same specification with all available lags represents the full-instrument extreme.
The strongest existing competitor is Arellano-Bond LASSO. We will run it with the authors' cross-fitting and forward orthogonal deviations on the same panels, with its penalty chosen by its own rule. We also run a System GMM variant in which LASSO selects among the pooled differenced and levels instruments. On the COVID county panel, the only reported results are substantive and computational: a 50% increase in next-week cases from K-12 openings, compounding to roughly a 300% long-run increase, with peak memory of 45 MB against 1.7 GB for standard Arellano-Bond. No reported result was found on Lacuna for AB-LASSO's simulation bias, RMSE, or coverage, so we will generate these ourselves under a common design.
Beating AB-LASSO means lower RMSE and closer-to-nominal coverage in break designs, without losing to it in no-break designs. On the real COVID panel, it means our estimate lies inside a narrower admissible range, with a forecast loss no worse than AB-LASSO's.
A difference-in-Hansen pruning baseline drops instrument blocks at the deepest lag while that test rejects. This is the textbook fix for suspected invalid instruments.
Three ablations isolate the key ingredient. First, a random admissible specification, drawn uniformly from $\mathcal{A}(\epsilon)$, mirrors the random-model comparison in Be Intentional About Fairness! and prices the cost of arbitrary choice. Second, an oracle-depth estimator, which uses the true break location in simulation, bounds achievable accuracy. Third, the Lepski rule applied over sample start date instead of lag depth tests whether trimming time or trimming instruments is the effective remedy.
All methods share one tuning budget: $L_{\max}$, the bootstrap draws, and the entity-level cross-fitting folds are identical across methods.
Why not just…?
Why not just cap lags at 2–3 and collapse, as everyone does? This is the most serious threat. If breaks are rare, a short cap already avoids stale lags. However, the cap is a fixed guess. In persistent series, shallow levels are weak instruments, which is exactly why System GMM exists. The simulations vary persistence $\gamma \in \{0.5, 0.8, 0.95\}$ and break timing. If the capped estimator matches Lepski lag depth across that grid, we will report that the adaptive rule is unnecessary. The admissible-set diagnostic would remain as the contribution.
Why not just use AB-LASSO? LASSO selects instruments by first-stage predictive strength. A stale deep lag can be highly predictive because of persistence while being invalid. Sparsity therefore gives no protection against the failure we target. The break designs test this directly: AB-LASSO should retain invalid deep lags when they are strong.
Why not just rely on Hansen and difference-in-Hansen tests? The file notes that with many instruments the $J$-test loses power and returns p-values implausibly near 1. Sequential pruning also has no calibrated stopping guarantee. The difference-in-Hansen baseline is run head-to-head. Our rule is essentially a calibrated version of that comparison performed on coefficients rather than on moments, so the experiment will show whether calibration matters.
Why not just report a specification curve over all choices? Reporting the curve is part of our proposal. Its limitation is that it describes multiplicity without resolving it. The random-admissible ablation measures how costly unresolved choice is.
Proposed Experiments
The simulation design uses $N \in \{100, 500, 2500\}$ and $T \in \{10, 20, 32\}$, a lagged outcome, and one endogenous regressor that responds to past shocks. The break design includes a common factor, with entity-specific loadings, that switches on at a time $\tau$. This makes $Y_{i,t-s}$ correlated with $\Delta\varepsilon_{it}$ for lags reaching before $\tau$.
The no-break control design keeps everything else fixed. The contrast between the two designs is the decisive mechanism test. The stale-lag hypothesis requires the admissible-set range to grow with $L_{\max}$ only in the break design. Growth in both designs would indicate ordinary many-instruments bias, and we would say so. Outcome metrics are bias, RMSE, 95% coverage, estimate range, and opposite-sign fraction.
The empirical studies use three panels. The first is the U.S. county COVID panel, reconstructed from the public sources used by AB-LASSO. Here the question is whether its 50% and 300% figures lie inside the admissible range and how wide that range is. The second is the Ho Chi Minh Stock Exchange firm panel from Financial distress and earnings management, where we check whether the reported Altman Z-score coefficient of 0.002, significant at 5%, survives across $\mathcal{A}(\epsilon)$. The third is the Chinese bank panel from Bank performance and non-interest income diversification, where the 2008 crisis provides a known break date for a placebo check. There, Lepski lag depth should stop admitting lags that reach before 2008 in post-2010 cross-sections, and should not stop when the sample is restricted to pre-2008 years.
Where the firm and bank panels cannot be obtained from the authors, we will rebuild them from public filings and state the reconstruction explicitly.
The project succeeds if three things hold. Estimate ranges on real panels exceed two standard errors. Range growth appears only in the break design. Lepski lag depth comes within 20% of oracle RMSE while beating AB-LASSO and difference-in-Hansen pruning on coverage. The project fails if the capped-lag practice matches the adaptive rule everywhere. In that case, the publishable finding reduces to the multiplicity audit alone.
References
- Predictive Multiplicity in Classification
- The Role of Hyperparameters in Predictive Multiplicity
- Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set
- Arellano-Bond LASSO Estimator for Dynamic Linear Panel Models
- Financial distress and earnings management: evidence from non-financial companies listed on the Ho Chi Minh Stock Exchange
- Bank performance and Non-interest income diversification: Evidence from Chinese commercial banks
- Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time
- Model Assessment and Selection under Temporal Distribution Shift
- Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data