People judge their own standing by comparing themselves with what is around them, and the clearest version of "around them" is what they see when they walk out the front door. Psychology has strong evidence that perceived social rank, more than objective income, predicts depression. Computer vision can now score street photographs for how wealthy or well-kept a place looks. This note joins the two. We propose to measure, from street-level imagery, the visual gap between a person's own home frontage and the streetscape within walking distance. We then test whether that gap predicts depressive symptoms through lowered subjective social status, after controlling for the person's own income and the absolute quality of their surroundings.
The case for doing this now rests on one striking result and one obvious hole in it. Well-Being Depends on Social Comparison modeled Twitter-derived happiness for 5.8 million US users and found a "sign flip." On its own, county income goes with more happiness. Once a person's own income is controlled, richer surroundings go with less happiness. That reference group was a whole county, taken from census tables, and nobody compares themselves to a county. Visual proxy models make it possible to define the reference group as the actual houses, cars and storefronts a person passes every day. The outcome measures can then come from validated questionnaires rather than inferred mood.
Core Claim
Our claim is that a hyper-local, image-derived comparison signal predicts depressive symptoms beyond two other sources: the absolute visual quality of a neighborhood, and census-based relative income. This signal is the difference between the "perceived affluence" of one's own frontage and that of the surrounding walkable streetscape. We further claim that subjective social status (where a person places themselves on a 10-rung social ladder) carries much of the effect.
The sharp, falsifiable prediction has two parts. First, the visual affluence of the surrounding ring should look protective on its own. Second, it should turn harmful once own-frontage affluence and own income are held fixed. That is the Giorgi sign flip, reproduced at the scale of a street.
Prior work does not do this. Google Street View Images as Predictors of Patient Health Outcomes linked tract-averaged street-view features to clinical diagnoses in about 938,000 Utah patients. It used absolute features only, with no subjective measures and no within-tract comparison. Effects of urban living environments on mental health in adults related map-based morphometrics around UK Biobank homes to psychiatric symptoms. It used no ground-level imagery and did not measure appraisal. Characterizing the perception of urban spaces from visual analytics of street-level imagery showed that crowd-rated visual wealth tracks block income (r ≈ 0.41). It never touched resident health. Our proposal differs from all three in the same way: the exposure is a contrast between the self and nearby others, and the hypothesized pathway runs through measured subjective rank.
Research Program
Visual affluence scorer. The first component is a scorer for how affluent and well-maintained a scene looks. We train it on the wealth dimension of Place Pulse 2.0 pairwise comparisons, about 110,000 street images from 56 cities. Following Freitas et al., we convert each pair choice into a continuous score with a Bayesian ranking model. We fine-tune a Swin Transformer, the backbone that UPDExplainer found strongest for detecting physical disorder on the same benchmark. The ranking loss is ordinal, in the spirit of the ordinal contrastive loss in AutoLTS, because affluence is a graded quantity rather than a class.
We also train a second head for physical disorder. It lets us separate "looks poor" from "looks neglected." Neglect has its own hypothesized effect through fear and threat rather than through comparison.
Exposures. For each participant we compute two scores. The frontage score uses the panoramas closest to the home address. We crop them to the facade using the geometric single-image surveying in SIM, which locates objects with roughly 2.5 m horizontal error. The ring score is the mean affluence of panoramas within 100–400 m along the street network, excluding the participant's own block face.
We define the exposure of interest, visual relative deprivation, as ring minus frontage. Imagery is matched to the capture date nearest the survey. We drop or flag participants in large apartment blocks, where a "frontage" says little about the individual household.
Data. We use two populations with complementary strengths.
The first is UK Biobank. It offers scale, self-reported psychiatric symptoms, household income bands, and the morphometric baseline already used by the urban-environment study. Exposures are computed under an approved application for residential coordinates. We have not verified the precision of the coordinates UK Biobank releases. If that precision is too coarse to identify a frontage, UK Biobank is used only for ring-level analyses: absolute ring affluence, plus the ring-by-income interaction.
The second is a new targeted cohort of roughly 2,000 adults. It is sampled to over-represent mixed-income streets in two cities with dense street-view coverage. It measures the MacArthur subjective social status ladder, CESD-8 depression, the PSS-10 perceived stress scale, the WHO-5 well-being index, perceived neighborhood status, household income and tenure. Address-level imagery makes the frontage analysis possible here. Measuring subjective status makes the mediation test possible.
Why the subjective-perception literature shapes the design. The association between subjective socioeconomic status and depressive symptoms in women of reproductive age found that subjective status has a direct effect on CESD-8 (β = −0.1092) in 4,219 CFPS women, with most of the total effect running through well-being. That makes subjective status the natural mediator. Work on perceived isolation in youth (Social isolation, mental health, and use of digital interventions in youth) and on perceived stress in frontline staff (Three Mental Health Symptoms of Frontline Medical Staff) supports the same general pattern: objective conditions act on mental health mostly through appraisal.
These studies share one weakness, which constrains our design. Exposure and outcome were both self-reported and cross-sectional, so a depressed person may simply rate everything worse. The rural study The mental health of rural older adults has exactly this problem: Likert ratings of the environment sit alongside WHO-5. Image-derived exposures break this shared-method loop. Depressed mood cannot change the pixels.
Analysis. The main model is a multilevel regression of symptoms on frontage affluence, ring affluence, own income, age, tenure, housing type, and small-area deprivation, with random intercepts for neighborhoods. Mediation through subjective status is tested in the new cohort, with sensitivity analysis for unmeasured mediator–outcome confounding. Our own design remains cross-sectional, so causal language is limited. Two features give partial leverage. In UK Biobank, repeat assessments combined with street-view change over time give within-person contrasts. In both datasets, the ring radius can be varied to check that the effect concentrates in the distance people actually see.
Baselines
The naive reference point is a model with only demographics, own income and area deprivation. Every exposure is judged by the variance it adds on top of this model, on held-out neighborhoods.
The strongest existing method is the multidimensional environmental-profile approach from the UK Biobank urban study. It built 128 morphometric variables and used sparse canonical correlation analysis, a method that finds weighted combinations of two variable sets that correlate as strongly as possible. With it, the study explained up to 4.71% of psychiatric-symptom variance and found an affective profile correlating at r = 0.22. We rerun this pipeline on the same participants, symptom set and cross-validation folds as our models. It is the strongest competitor because it already captures density, deprivation, pollution and greenness, which are exactly the things a visual affluence score might merely re-encode. Beating it means a statistically reliable improvement in out-of-fold explained variance once our relative-deprivation term is added to its profiles. A gain that disappears when the profiles are included would count as failure.
The second strong baseline is census relative income: the Giorgi hierarchical model with the reference group set at the smallest available census unit rather than the county. On their own task, that model reached r = 0.55 against 0.52 for ordinary regression. We fit it with the same covariates and tuning budget as our model.
The ablation that isolates the key ingredient keeps every visual feature but removes the contrast. It enters frontage and ring affluence only additively, or uses absolute visual quality in the style of VoPham et al. (greenness, disorder, building type). If the contrast carries no signal beyond its parts, the core claim fails.
For the visual scorer, we compare against ResNet-50 and plain ViT backbones under matched training budgets. On UPDExplainer's binary disorder task, these reached 71.93% and 76.02% against the Swin model's 79.89%. For pairwise wealth-ranking accuracy on Place Pulse 2.0, we found no reported result on Lacuna, so we will report our own number with the backbone comparison.
Why not just…?
Why not just use small-area census income inequality instead of images? This is the real threat. If neighbors' incomes are what matter, a census gradient might do everything the visual gradient does at far lower cost. Two considerations favor images. Visible affluence is what is actually seen and compared. And Freitas et al. found visual wealth only moderately correlated with block income (r ≈ 0.41), so the two carry different information. The nested comparison in the Baselines settles the question. If the visual contrast adds nothing once the census contrast is included, we will report that and conclude the visual channel is redundant.
Why not just ask residents how their home compares with the street? In the new cohort, we do ask, as a secondary measure. But self-rated comparison is part of the appraisal we want to explain, not an independent exposure. Using it as a predictor reintroduces the mood-congruent bias that undermines the Jintang and CFPS-style designs. Self-report also cannot scale to UK Biobank. The experiment that tests this directly is to correlate the image-based gap with the self-rated gap, check whether the correlation weakens among high-CESD respondents (a sign of mood bias), and confirm that the image-based gap still predicts symptoms.
Why not just use absolute visual quality (greenness, disorder) at the home? VoPham et al. already showed that absolute street-view features predict disease. The ablation above decides whether the contrast matters. Its distinctive prediction is the conditional sign flip, which an absolute-quality account cannot produce: a purely absolute account predicts that nicer surroundings are always better.
Why not train a model end-to-end from images to depression scores? With a few thousand psychometric labels and heavy confounding by income, such a model would mostly learn neighborhood wealth. It would also give no testable mechanism. We fit it only as an upper-bound reference for predictive accuracy, not as an explanation.
Decisive Checks
Five results decide whether the idea holds. The first is the sign flip itself. The ring-affluence coefficient should be non-negative alone and negative once frontage affluence and own income are included, in both UK Biobank and the new cohort.
The second is incremental validity. The contrast term must improve held-out explained variance over the UK Biobank profile pipeline and over the census relative-income model.
The third is mediation. In the new cohort, a substantial share of the contrast's effect on CESD-8 should run through the subjective status ladder, while perceived stress serves as a secondary pathway.
The fourth is spatial specificity. The effect should be strongest for walking-distance rings and fade for rings beyond everyday sight. We also run a negative-control outcome, one with no plausible comparison pathway, to detect residual confounding.
The fifth concerns selection. Renters, long-tenure older residents and recent movers sort into modest homes on affluent streets for different reasons. We stratify by tenure and by length of residence. An effect present only among recent movers would point to selection rather than comparison.
If the sign flip appears, survives the census baseline and runs through subjective status, we will have a scalable, bias-resistant measure of relative deprivation tied to what people see. If it fails, that will show the comparison mechanism operates at a scale imagery does not capture.
References
- Well-Being Depends on Social Comparison: Hierarchical Models of Twitter Language Suggest That Richer Neighbors Make You Less Happy
- Google Street View Images as Predictors of Patient Health Outcomes, 2017-2019
- Effects of urban living environments on mental health in adults
- Characterizing the perception of urban spaces from visual analytics of street-level imagery
- UPDExplainer: an Interpretable Transformer-based Framework for Urban Physical Disorder Detection Using Street View Imagery
- AutoLTS: Automating Cycling Stress Assessment via Contrastive Learning and Spatial Post-processing
- SIM: A mapping framework for built environment auditing based on street view imagery
- The association between subjective socioeconomic status and depressive symptoms in women of reproductive age: the chain mediating effects of marital satisfaction and well-being
- Social isolation, mental health, and use of digital interventions in youth during the COVID-19 pandemic: A nationally representative survey
- Three Mental Health Symptoms of Frontline Medical Staff Associated With Occupational Stressors During the COVID-19 Peak Outbreak in China: The Mediation of Perceived Stress and the Moderation of Social Support
- The mental health of rural older adults: the role of the built environment in Jintang County