Auditing Space-Time Fairness Against Latent Need, Not Recorded Events

Algorithms that allocate bikes, rides, patrols or public-health attention are usually judged fair or unfair against the events a system happened to record, such as pickups, clinic notifications or app requests. Those records are filtered twice. A neighborhood with no bikes produces no trips, and a population that rarely sees a doctor produces few flu reports. Both filters are strongest in exactly the poor, segregated, multiply marginalized places that fairness audits are meant to protect. This note proposes an audit that asks a different question. It does not ask whether predictions match recorded demand equally across groups. It asks whether service matches latent need equally across intersecting demographic groups, space and time. The audit estimates need with a hierarchical, weighted logistic model of the probability that a need gets recorded at all.

The idea borrows from two mature toolkits that have not met in the machine-learning fairness literature. Epidemiologists correct "passive surveillance" data, meaning data that exists only when a sick person chooses to seek care, by modeling the chance of being ascertained. Survey statisticians handle the many near-empty demographic cells created by crossing race, age, income and place through multilevel regression and post-stratification (MRP): fit a model that shares strength across related cells, then reweight to census totals. Combining the two gives an audit that stays defined for rare intersectional groups instead of dropping them. It also avoids treating zero observed trips as zero demand.

Core Claim

Current space-time fairness metrics are computed against an ascertainment-biased target. As a result, they can certify a model as fair while it reproduces the service gap. The fix is to estimate a per-cell recording probability $\pi(c)$, where a cell $c$ is an intersectional demographic group within a spatial zone and time window. The estimate is anchored by an auxiliary signal that is less access-dependent. It comes from a logistic model with partially pooled demographic, spatial and temporal effects. Fairness gaps are then reported against the corrected need, with posterior intervals, and only for cells whose intervals are narrow enough to resolve.

The claim becomes testable because it predicts that the corrected audit and the standard audit will disagree in a specific direction. The standard audit will understate gaps in low-access, high-poverty cells. We can check that prediction where a less-biased reference exists.

Why this gap is real

The motivating evidence comes from both sides. Fairness-Aware Demand Prediction for New Mobility and FairST define region- and individual-based fairness gaps (RFG, IFG) for bike-share and ride-hailing demand prediction. They train and evaluate on observed pickups and concede that absent supply suppresses recorded demand. They also treat race, age and education one at a time. Socioeconomic bias in influenza surveillance shows the surveillance analogue: outpatient flu surveillance in Dallas-Fort Worth predicts worst in the poorest zip codes (P < 0.0001), and adding emergency-department and search-query streams does not close the gap. Estimating incidence of sexually transmissible infections in Australia shows how badly a naive correction can fail. Dividing notifications by tests suggested chlamydia was falling, when modeling screening effort showed incidence among young men nearly doubled. The same work also reversed the apparent sex difference. Towards Algorithmic Fairness in Space-Time names non-random space-time missingness and feedback loops as sources of bias but offers no estimator.

Other work constrains the design. Doubly-Regressing Approach for Subgroup Fairness (DRAF) handles up to 18 sensitive attributes by auditing only subgroups above a size threshold. It argues that constraints on tiny groups fit noise, and we accept that warning. On the Epistemic Limits of Personalized Prediction proves that some subgroup claims cannot be verified at any realistic sample size. Size-adaptive Hypothesis Testing for Fairness shows that small-group disparities are often sampling noise. Together they rule out an audit that emits a point estimate for every cell. Ours must say which cells it can and cannot resolve.

Two sources supply the method. A calibrated BISG shows that raking spatially segregated estimates to known margins removes statewide bias, cutting mean absolute error from 10% to 4% in Florida and from 7% to 3% in North Carolina. Auditing Cross-Cultural Consistency of Human-Annotated Labels shows MRP working inside a machine-learning bias audit. Neither models an observation filter or space-time structure.

Research Program

The model treats the recorded count in cell $c$ as a thinned version of true need:

$$y_{\text{obs}}(c) \sim \text{Poisson}\big(N(c)\,\lambda(c)\,\pi(c)\big), \qquad \text{logit}\,\pi(c) = x(c)^\top\beta + \alpha_{g(c)} + u_{s(c)} + v_{t(c)}.$$

Here $N(c)$ is the census population of the cell and $\lambda(c)$ the per-capita latent need rate. The access covariates $x(c)$ are available supply, testing volume and price. The term $\alpha_{g}$ is a partially pooled effect for intersectional group $g$, built from main effects and shrunk interactions as in MRP. The spatial effect $u_s$ follows a neighborhood-smoothing prior, which addresses the independence assumption flagged in the survey-calibration literature. The temporal effect is $v_t$.

Need and access cannot be separated from recorded counts alone. We therefore anchor identification with an auxiliary signal. In the flu setting, the anchor is hospital discharge records, where ascertainment is far less discretionary than outpatient visits. In the mobility setting, it is periods when supply is saturated, so that recorded trips approach need. Once fitted, $1/\hat\pi(c)$ acts as a shrunken inverse-probability weight. The partial pooling is what prevents the weight explosion that plain inverse weighting suffers in near-empty cells. Cell estimates are then raked to census joint margins, following Greengard and Gelman.

The audited quantity for a prediction or allocation $\hat y$ is a need-relative gap. For each resolvable cell we compute service per unit of estimated need, $\hat y(c)/\hat\lambda(c)N(c)$, and report the spread across intersectional groups within each zone-hour block. A cell counts as resolvable when the posterior interval of this ratio is narrower than a preset width. Every other cell is reported explicitly as unresolvable. The unresolvable list replaces DRAF's silent threshold with a visible, need-weighted count of unaudited people.

We will build four things: the estimator, released code for the audit metric, a semi-synthetic benchmark with known suppression, and re-audits of published fairness-aware models.

Baselines

The naive reference is the standard audit on observed labels. It computes RFG and IFG exactly as defined in FairST, treating recorded trips or notifications as need, with no intersectional breakdown. Beating it means that on data with known suppression, our need-relative gap ranks cells by true unfairness better (higher rank correlation with the known gap) and certifies fewer truly unfair models as fair.

The strongest existing fairness method is FairST itself. It is a 3D CNN with fairness regularizers, and on Seattle bike-share its IF regularizer reduced the gap from 25.07 to 15.28, with reported reductions of over 99% (RFG) and 80% (IFG) for race. We will retrain FairST on the same splits and the same Seattle and RideAustin data, with the same hyperparameter search budget as any model we train. We then audit it with both the standard metric and ours. We count it as "beaten" if FairST's reported gap reduction largely disappears under the need-relative audit on semi-synthetic data, while a FairST variant trained on our corrected targets retains its reduction.

The strongest existing correction baseline in surveillance is the Cameron et al. probability-tree adjustment, which applies fixed recording probabilities per age-sex stratum. We run it with the same strata and priors available to us. Our model must track the hospitalization reference more closely than this adjustment in high-poverty zip codes to count as a win.

The sparsity baseline is DRAF-style truncation: audit only cells above a threshold $\gamma$, with $\gamma$ tuned on the same budget.

The key ablation is MRP without the recording layer: the same partial pooling, spatial smoothing and raking, but with $\pi(c)$ fixed to one. This isolates whether the ascertainment component, rather than the shrinkage, drives any improvement.

Why not just…?

Why not just enforce marginal fairness on each attribute? Marginal parity can coexist with severe intersectional unfairness, as DRAF demonstrates. Bounding and Approximating Intersectional Fairness through Marginal Fairness shows the marginal-to-intersectional link is tight only when attributes are independent. Race and location in segregated cities are strongly dependent, so the independence condition fails. We will report marginal gaps alongside intersectional ones to show where they diverge.

Why not just normalize by supply or testing volume? This is the relative-benchmark approach of HCRide and the naive ratio of Cameron et al. The Australian chlamydia result shows this ratio can reverse the sign of a trend, because who gets tested is itself demographic. Supply-ratio normalization is included as a baseline in the semi-synthetic experiment, where the true answer is known.

Why not just use plain MRP or raking and skip the recording model? This is the real threat. Pooling and raking may account for most of the gain, and the recording layer adds identification risk. The MRP-only ablation settles it. If MRP-only matches the full model on the hospitalization comparison and the synthetic benchmark, we will report that the ascertainment layer is unnecessary and publish the simpler audit. Our prediction is that MRP-only fixes sparsity but not direction: it will still understate gaps in low-access cells, because it pools over recorded counts that are already thinned.

Why not just add more data streams? The Dallas-Fort Worth study already tried emergency-department and search-trend streams, and the poverty gap persisted. More passive streams inherit the same access filter.

Proposed Experiments

The first experiment is a semi-synthetic suppression test on Seattle dockless bikeshare and RideAustin. We treat trips in the best-supplied hours as a need surface. We then inject known suppression that depends on block-group income and on synthetic intersectional composition drawn from census tables, which yields data with a known true gap. We score each audit by rank correlation with the true cell gaps, by the false-certification rate of fairness-aware models, and by interval coverage.

The second experiment is a real-data check on the Dallas-Fort Worth influenza data, 2007–2012. We fit the recording model using outpatient surveillance and use hospital discharges only for held-out validation, and we compare each method's burden estimates by poverty quartile against the discharge-based reference. The file reports only a significance level for the original gap, not an error magnitude, so no reported result was found on Lacuna for a numeric error baseline there. We will recompute it.

The third experiment uses the Australian notification and testing series. It checks whether our model reproduces the known reversal in the male 15–34 trend without hand-set triangular priors.

The decisive result for the proposal is directional. The need-relative audit must widen, not merely change, measured gaps in high-poverty and low-access cells relative to the standard audit, and it must track the known truth there. The full model must also beat the MRP-only ablation on that directional error.

We state the main limitation openly. If the anchor is itself access-limited, as hospital admission partly is, the correction is a lower bound on the true gap. We will report it as such.