Studies that turn street-view photos into measurements, such as how green a street is or how many utility wires hang over it, now routinely claim causal effects on crashes and chronic disease. Almost all of them rest on a single snapshot. The photos, the neighborhood covariates and the outcomes are collected in the same window, so affluence, municipal investment and residential sorting can create the association as easily as the streetscape can. This proposal treats that weakness as a data opportunity. The same places have also been photographed years earlier, from the ground and from orbit. Those older images record the neighborhood's persistent hidden character before the current exposure and outcome existed. With two such independent older views, we can use proximal causal inference, a method that corrects for an unmeasured confounder using two imperfect "shadows" of it instead of measuring it directly.
The contribution, stated for a colleague in two sentences: when only cross-sectional outcome data exist, use archival satellite imagery and archival street-level imagery from before the exposure window as the two proxies that proximal causal inference needs, because separating them in time and in modality is what makes their errors plausibly independent. Then carry the vision model's own error in estimating the exposure into the causal estimate, instead of treating segmentation output as ground truth.
Core Claim
Two recent street-view studies apply causal estimators to visual features. An Integrated Causal Inference Framework for Traffic Safety Modeling with Semantic Street-View Visual Features uses double machine learning, a method that predicts out observed confounders from both treatment and outcome, on 228,352 Miami images. It finds that only greenery survives adjustment, with an average treatment effect (ATE) of −6.38 at p = 0.005. Semantic4Safety weights by the generalized propensity score on Austin imagery and reports significant causal links for over 90% of feature–accident pairs. These papers motivate the proposal, and both have the same two gaps. First, they adjust only for observed covariates, such as the social vulnerability index or OpenStreetMap road type, and both acknowledge that time-invariant neighborhood traits remain uncontrolled. Second, they plug segmentation proportions in as if they were error-free.
We claim that both gaps can be addressed without longitudinal outcome data, and that this is the case when historical imagery exists but outcome panels do not. That situation is common for health records and for many cities' crash data before digitization. The claim is falsifiable. Where outcome panels do exist, the proximal estimate computed from one cross-section should agree with a panel estimate much more closely than the double machine learning estimate does.
Research Program
Identification. Let $A$ be the current exposure, for example the greenery proportion estimated from recent street view. Let $Y$ be the outcome in a later window, $X$ the observed covariates, and $U$ the unmeasured persistent neighborhood state (historical wealth, maintenance culture, traffic regime). Proximal identification, as developed in Semiparametric Proximal Causal Inference and explained in An Introduction to Proximal Causal Learning, needs two proxies of $U$ with specific properties. The treatment-side proxy $Z$ may relate to $A$ but must not affect $Y$ except through $U$ and $A$. The outcome-side proxy $W$ must not be affected by $A$. The two proxies must be independent given $U$, $A$ and $X$.
We assign $W$ to street-level imagery captured before the exposure window. We assign $Z$ to satellite imagery from a different, earlier era. We then solve for an outcome bridge function $h$ satisfying
$$ E[Y \mid A, Z, X] = E[h(W, A, X) \mid A, Z, X], $$
which yields the dose-response curve $E[h(W, a, X)]$ at each exposure level $a$.
This assignment addresses a problem that Proximal Causal Inference With Text Data exposed for language. That paper shows that drawing both proxies from the same document breaks identification, so distinct pre-treatment sources are required. In our setting the naive choice would take $W$ and $Z$ from, say, different compass headings of the same current panorama. That choice fails twice: both views contain the treatment itself, and both share the same camera, season and lighting. Separating the proxies in time handles the first problem. Separating them by modality (overhead versus ground) and by capture era handles the second. We borrow the odds-ratio violation check from the text paper and add a visual-specific test. It asks whether $W$ still predicts $Z$ after conditioning on a rich summary of the current scene, which would indicate leaked shared appearance rather than shared confounding.
Estimation. We use the kernel and neural bridge estimators from PCL: Proximal Causal Learning and the doubly robust form from the semiparametric theory. The proxies enter as frozen image embeddings reduced to a few dozen dimensions. Frozen embeddings keep the bridge problem well posed. We choose feature-matching encoders such as DINOv2 over pixel-reconstruction encoders, following the finding in Smoke and Mirrors in Causal Downstream Tasks that reconstruction-trained encoders over-attend to background.
Exposure measurement error. The same paper shows two things. A classifier above 95% accuracy can still produce 50% error in the ATE when its mistakes differ between treated and untreated units. Hard labels bias estimates while soft probabilities do not. We therefore keep $A$ as a continuous segmentation proportion. We hand-label a stratified validation sample of a few thousand panoramas and fit a calibration model of true versus estimated exposure conditional on $X$ and the proxies. That correction is propagated through the bridge estimator by resampling, so uncertainty in the vision model appears in the confidence interval.
Main risk. Historical street-level imagery exists only in some places. Where it is missing, $W$ falls back to an older-era satellite image at a different resolution. We report results separately by proxy configuration.
Baselines
The naive reference point regresses the outcome on the estimated exposure and the observed covariates. This is the log-Poisson design of Google Street View Images as Predictors of Patient Health Outcomes, 2017-2019. Every method is expected to beat it on semi-synthetic data, so it serves only to calibrate how much bias exists.
The strongest existing method is the double machine learning pipeline of the Miami traffic-safety paper, which reports a greenery ATE of −6.38 (p = 0.005). We re-run it on identical grid cells, outcomes and covariates, with the same nuisance learners and hyperparameter search budget as our bridge functions. Generalized propensity score weighting as in Semantic4Safety is a second backdoor baseline. The file gives no single reported ATE number for it, only the claim that over 90% of pairs were significant. Beating these baselines means lower absolute bias and nominal interval coverage on semi-synthetic data, and closer agreement with panel estimates on real data. Matching or slightly improving the Miami point estimate does not count.
A stronger backdoor competitor adds the archival embeddings directly as covariates in double machine learning ("adjust for old images"). This is the key ablation. It isolates whether the two-proxy proximal structure matters, or whether simply having pre-treatment imagery does the work. A second ablation replaces the temporally separated proxies with same-era multi-heading views, to test the independence argument directly. A third drops the measurement-error correction. For that component the reference competitor is validation on treatment effect bias, the model-selection rule from Smoke and Mirrors, which cut ATE error by 11–18% on ISTAnt.
Why not just…?
Why not just adjust for more census covariates with double machine learning? Such adjustment only removes confounding that the covariates actually record. In the Miami study, adjustment already collapsed four of five visual features, and the authors still flag unmeasured time-invariant neighborhood traits. The text proximal paper shows naive backdoor adjustment staying biased when confounders are unrecorded. The semi-synthetic experiments below measure this directly, because the hidden confounder there is known.
Why not just put the old images into double machine learning as covariates? This is the real threat, and it may be enough when the archival images almost fully reveal $U$. Proxy theory says backdoor adjustment on a noisy proxy leaves residual bias that shrinks only as the proxy becomes perfect, while the bridge estimator removes it under the independence conditions. The ablation settles which regime we are in by varying how informative the proxies are. If backdoor adjustment with archival covariates matches proximal estimation on the real-data panel checks, we will report that the simpler method suffices.
Why not just run a difference-in-differences or panel analysis? Where outcome panels and repeated imagery both exist, a panel analysis is better, and we use it as the reference answer rather than as a competitor. The proposal targets the common case where outcomes are one cross-section, such as the Utah records or the smartwatch pilot in Effect of urban environment on cardiovascular health, but the imagery archive is deep.
Why not just use a more accurate segmentation model? Smoke and Mirrors tested 6,480 models and found that accuracy does not predict downstream causal error. The measurement-error ablation reports ATE bias as a function of segmentation accuracy to show whether the same holds for street scenes.
Proposed Experiments
First, a semi-synthetic benchmark built on real imagery. We take Miami and Austin grid cells. We define $U$ from historical census variables that are withheld from all estimators. We then simulate exposure and crash outcomes with known greenery effects, including sorting-style dependence of exposure on $U$. Bias, root mean squared error and 95% interval coverage are compared across all methods and ablations. Severity of confounding and informativeness of the proxies are varied systematically. CausalMNIST and ISTAnt serve only as sanity checks for the measurement-error module, where ground-truth effects are exact.
Second, a real-data panel check. Florida crash data span multiple years, so we select sites where greenery changed visibly between street-view capture years. First-differenced crash counts give a within-site panel estimate. We then ask whether the proximal estimate computed from a single cross-section lands closer to this estimate than double machine learning does.
Third, falsification tests. These use negative-control outcomes that greenery should not affect, such as vehicle-breakdown incidents in the Semantic4Safety categories. A method passes only if it returns a null there.
Fourth, the Utah health associations are re-estimated under the proximal design. This is the reverse-causation and sorting case, and there we report how much of each association survives.
The decisive result is the panel check. If proximal estimates from a single snapshot track the panel estimates and backdoor methods do not, then archival imagery can resolve a meaningful share of cross-sectional ambiguity. If they do not track, the diagnostics will show which assumption failed: proxy independence, proxy informativeness, or exposure measurement error.
References
- An Integrated Causal Inference Framework for Traffic Safety Modeling with Semantic Street-View Visual Features
- Semantic4Safety: Causal Insights from Zero-shot Street View Imagery Segmentation for Urban Road Safety
- Semiparametric Proximal Causal Inference
- An Introduction to Proximal Causal Learning
- Proximal Causal Inference With Text Data
- PCL: Proximal Causal Learning
- Smoke and Mirrors in Causal Downstream Tasks
- Google Street View Images as Predictors of Patient Health Outcomes, 2017-2019
- Effect of urban environment on cardiovascular health: a feasibility pilot study using machine learning to predict heart rate variability in patients with heart failure