Health systems audit disparities quarter by quarter. Which groups are under-screened, which carry more hypertension, which receive fewer electrocardiograms. These audits are run on records that are still filling in. Two kinds of information arrive late: the clinical event itself, and the patient's race or ethnicity field. The second delay is not random. In a five-million-patient primary-care cohort, 73% of patients had a delayed race entry and over half waited 60 days or more. The patients whose race arrived late were older, sicker, and more frequent visitors (Bias Delayed is Bias Denied?). An audit run at quarter's end therefore sees a healthier, unrepresentative slice of each group, and for electrocardiograms it reported the wrong direction of disparity 14% of the time.
We propose to treat this as a nowcasting problem. Nowcasting means estimating what a still-incomplete record will eventually say, using the history of how earlier records filled in. The key change from existing nowcasting is that the group label itself is one of the things being nowcast. Our core claim is that patients who are still unlabeled at a given lag are a distinct and drifting population. Their eventual group membership and their eventual outcomes are therefore best predicted from what earlier cohorts of "still-unlabeled-at-that-lag" patients turned out to be. Off-the-shelf demographic imputation does the opposite: it implicitly assumes these patients resemble everyone else. We further claim that this lag-conditional model can be made to transfer across states and to small groups by restricting it to predictors whose relationship to the eventual label is stable across environments. This is the same lesson cross-ancestry genetics learned about proxy markers that work in one population and fail in another.
Where the three directions meet
The surveillance-latency literature knows how to fill in a reporting triangle, the table of event counts indexed by when each event happened and how long it took to be reported. One approach models that table with a smooth latent Gaussian process under a Negative Binomial likelihood. It caught a COVID-19 mortality uptick in Brazil weeks early, and it reversed a false $R_t<1$ reading (Gaussian process nowcasting). A second approach deconvolves recovery and death curves with a fitted delay distribution (Reporting delays: A widely neglected impact factor in COVID-19 forecasts). A third bypasses registries with search data (ARGO). All of these model one undifferentiated population, so they supply our count-correction machinery but nothing for subgroups.
The administrative-missingness literature supplies the motivating evidence. Gosciak et al. document the structured race-label delay, but their analysis is retrospective and builds no corrective estimator. Their test of Bayesian Improved First Name Surname Geocoding (BIFSG), which guesses race from name and address, found that it overestimated minority prevalence and failed to improve disparity estimates for three of six outcomes. That is the failure we target. Two other papers fix the problem statically and constrain our design. Wu et al. pick which unobserved clinical features to collect under a budget (Collecting data when missingness is unknown). Luo et al. protect hidden subgroups using gradient-similarity graphs (Fairness without Demographics on Electronic Health Records). Neither treats arrival time as information. Work on adaptation when missingness patterns shift between domains (Domain Adaptation under Missingness Shift) is the closest methodological relative. Here the shift runs along the lag axis within one data source rather than between sources.
The ancestry-generalizability direction enters as a constraint and a design lens rather than as a data source. Mendelian randomization papers make the point that a marker strongly tied to a trait in European cohorts may be nearly uninformative elsewhere. Examples are the multi-SNP GSMR analyses with pleiotropy filtering (Causal associations between risk factors and common diseases) and parallel European/East Asian analyses (psychiatric disorders and glaucoma). The reason is that the marker was a correlate, not the mechanism. Rules calibrated on White patients similarly misfire in diverse cohorts (monogenic diabetes). BIFSG is exactly such a proxy. It is calibrated on the general population, and its link to true race need not hold in the late-labeled subpopulation, or for American Indian/Alaska Native and Native Hawaiian/Pacific Islander patients. For the method, we borrow Invariant Ancestry Search. That work finds predictor sets whose relationship to a target is unchanged across environments, and it showed that taking the union of minimal such sets recovers more true causes than intersecting them. We use it to choose which covariates the label model may rely on, with states and years as the environments.
Core Claim
For a patient $i$ in audit quarter $t$, let $a_i$ be the lag until the outcome is recorded and $r_i$ the lag until race is recorded. An audit at lag $d$ sees four kinds of patients: labeled with outcome, labeled without outcome, and unlabeled with or without outcome. The prevalence in group $g$ that the eventual back-filled audit will report is
$$ \hat p_{g,t}(d)=\frac{\sum_i \Pr(G_i=g\mid \mathcal{O}_i(d))\,\Pr(Y_i=1\mid G_i=g,\mathcal{O}_i(d))}{\sum_i \Pr(G_i=g\mid \mathcal{O}_i(d))}. $$
Here $G_i$ is the eventual race label, $Y_i$ the eventual outcome, and $\mathcal{O}_i(d)$ everything visible at lag $d$. That includes whether $r_i\le d$ and $a_i\le d$, visit count, age, BIFSG probabilities, and practice. The two factors are learned jointly from past quarters that have since filled in. The joint fit matters because the "healthy patient" effect makes label lateness and outcome status correlated. Treating them independently, or imputing $G_i$ without conditioning on lateness, should reproduce the biases Gosciak et al. observed.
The prediction we will test is concrete. Most of the gain over BIFSG comes from conditioning on "still unlabeled at lag $d$." The invariance-based covariate selection is what keeps those gains when the model is moved to a new state or a small group.
Research Program
The estimator has three pieces. The first is a per-group outcome nowcast. This is the additive Gaussian process over time and delay from Bastide et al., extended with a group index and partial pooling, so that small groups borrow strength from large ones without being forced to share delay shapes. The second is a lag-conditional label model, $\Pr(G_i=g\mid\mathcal{O}_i(d))$. It is a multinomial model trained on patient–lag pairs from completed historical quarters. It is refit on a rolling window, because reporting behavior drifts and the delay papers flag non-stationarity as unsolved. BIFSG enters as one input, not as the answer. The third is the covariate restriction. We run invariant ancestry search across state-by-year environments to find covariate sets whose relation to eventual race is stable. The label model is then fit only on their union.
Uncertainty from all pieces is propagated through posterior sampling. Each audit therefore reports an interval on the disparity and a probability that its sign is correct. That second quantity is what a reviewer of an audit actually needs.
The analysis side asks when early audits can be trusted at all. We will characterize, by outcome and group, the lag $d$ at which the nowcast's sign-correct probability exceeds a chosen threshold. This yields a principled answer to "how long should we wait?" in place of a fixed annual cadence.
Baselines
The naive reference is the complete-case audit at quarter end. It drops unlabeled patients and computes group prevalences on what is visible. This is current practice and the source of the 14% electrocardiogram sign-switching rate in Bias Delayed is Bias Denied?. Beating it means a lower sign-error rate and lower absolute disparity error against the final back-filled audit, on the same held-out quarters.
The strongest existing correction for missing labels is BIFSG imputation as evaluated by Gosciak et al. Its reported result is failure to significantly improve accuracy on three of six outcomes. Beyond that, the file gives no per-outcome error figures, so we will reproduce it on identical AFC splits. Beating it means improving disparity error on at least five of the six outcomes, with the improvement concentrated where BIFSG failed.
The strongest count-nowcasting competitor is a per-group Gaussian-process nowcast in the style of Bastide et al., run on labeled patients only, with unlabeled patients allocated to groups in proportion to the currently labeled mix. In its own paper this model "consistently outperformed or matched" NobBS on RMSE and CRPS. No reported numeric result was found on Lacuna for that comparison. It gets the same kernel family, the same MCMC budget, and the same rolling window as our model. This is the baseline that most directly tests whether modeling the label delay adds anything beyond modeling the count delay.
The decisive ablation is our full model with lag-conditioning removed from the label model, so that $\Pr(G_i=g\mid\mathcal{O}_i(d))$ ignores whether and for how long the label is missing. A second ablation removes the invariance restriction and fits on all covariates. Both ablations use identical tuning budgets, chosen on a validation block of quarters that precedes all test quarters. The final back-filled audit serves as the oracle.
Why not just…?
Why not just wait until the records fill in? For retrospective research this is fine. But audits that steer screening programs, and surveillance during outbreaks, cannot wait years. Gosciak et al. show that the Arkansas hypertension gap between Asian/Pacific Islander and Hispanic patients was invisible in the early audit, and a gap of over 5 points appeared only three years later. Our "when can we trust it" analysis turns the waiting time into a measured quantity rather than a guess. If waiting a single extra month removes most of the error, the experiments will show it and we will report that as the practical recommendation.
Why not just impute race with BIFSG or a richer proxy model? Proxy imputation assumes that the relationship between name, address, behavior and race is the same in the unlabeled patients as in the labeled ones. Gosciak et al. show that the unlabeled are systematically different, and BIFSG failed on half the outcomes. Richer behavioral imputers such as the phone-data models of Jahani et al. also decay when behavior shifts. Our lag-conditioning ablation tests this directly. If a static imputer given the same covariates matches our model, the core claim is false.
Why not just nowcast each labeled group separately and spread the unlabeled proportionally? This is the real threat, because it is simple and reuses mature tools. It fails in principle if late-labeled patients differ in outcome rates from early-labeled patients in the same group, which the "healthy patient" finding suggests. Whether that difference is large enough to matter is an empirical question. The head-to-head against this baseline, broken down by outcome, will settle it. A tie on chronic-disease outcomes with a win only on procedures would be a meaningful, narrower result.
Why not just train with a fairness-without-demographics method? Methods like Graph of Gradients reweight training to protect hidden subgroups. They do not produce an audit estimate of group-level prevalence. Luo et al. also report that conventional reweighting methods break under 10% label noise, and late-recorded outcomes are precisely structured label noise (Wu et al.). These methods address a different question.
Proposed Experiments
The main testbed is the American Family Cohort: 1,000 practices across 50 states, more than 100 million timestamped interactions from 2010 to 2024, and the six outcomes studied by Gosciak et al., including hypertension, diabetes, depression screening, and electrocardiograms. We replay history. For each test quarter, the models see only records timestamped by audit lag $d$ (0, 30, 60, 90, and 180 days). They are trained only on quarters that had completed by then, and they are scored against the final back-filled audit.
The metrics are absolute error in the between-group prevalence gap, sign-error rate, 90% interval coverage, and CRPS on group prevalences. For transport, we run leave-states-out evaluation. We also report results separately for American Indian/Alaska Native and Native Hawaiian/Pacific Islander patients, where delays differ in kind from those of larger groups.
A secondary check on Brazil's SIVEP-Gripe data verifies that our group-extended Gaussian-process component reproduces the aggregate count behavior of Bastide et al. before we trust it on subgroups. If race fields there turn out to carry their own delays, we will use the data as an external replication.
The decisive outcomes are three. The first is whether the full model brings the electrocardiogram sign-error rate well below the 14% complete-case figure while keeping interval coverage near nominal. The second is whether removing lag-conditioning erases most of that gain, which would confirm the mechanism. The third is whether the invariance restriction preserves the gain on held-out states and small groups where the unrestricted model degrades. That last result would show the model learned a transportable relationship rather than a local one.
References
- Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
- Gaussian process nowcasting: application to COVID-19 mortality reporting
- Reporting delays: A widely neglected impact factor in COVID-19 forecasts
- Accurate estimation of influenza epidemics using Google search data via ARGO
- Collecting data when missingness is unknown: a method for improving model performance given under-reporting in patient populations
- Fairness without Demographics on Electronic Health Records
- Domain Adaptation under Missingness Shift
- Causal associations between risk factors and common diseases inferred from GWAS summary data
- Mendelian randomization study shows no causal relationship between psychiatric disorders and glaucoma in European and East Asian populations
- Precision diagnosis for monogenic diabetes requires ethnicity specific criteria for genetic testing
- Invariant Ancestry Search
- Improving official statistics in emerging markets using machine learning and mobile phone data