When Does Another Urban Data Source Help a City With Few Labels?

City analysts increasingly predict neighborhood outcomes, such as income, social vulnerability, or consumer spending power, by fusing many data streams: taxi trips, points of interest (shops, schools, transit stops), road networks, and phone-derived population counts. The usual story is that more sources give a fuller picture of the city. That story was told mostly in data-rich cities. The cities that most need these models are smaller or newly instrumented. They may have only a few dozen labeled neighborhoods, and in that regime every added source also adds dimensions to fit. This proposal asks a plain question that the literature has not answered: for a target city with $N$ labeled regions, which data sources are worth fusing, and at what $N$ does fusing everything stop helping or start hurting?

The core claim is that the value of a data source depends on the target city's label budget, and that this dependence can be predicted before spending target labels. It can be read off a cheap quantity: how consistently that source's mapping to the outcome holds across data-rich cities. We would build a fusion model that pulls each source's target-city weights toward weights learned in other cities. The pull for each source is set by how much those cities agree. We pair the model with an evaluation protocol that makes small-$N$ comparisons trustworthy. If the claim holds, practitioners get a label-budget-aware rule for which sources to collect and fuse. If it fails, we learn that well-tuned concatenation is already enough, which is itself worth knowing.

What exists and what is missing

Cross-city transfer under label scarcity is well studied, so the obvious idea of transferring multi-source embeddings to a small city is already done. Smart City Development with Urban Transfer Learning lays out a pipeline in three stages: identify a source, link it to the target, then refine on the target. It does not say how much target data the refinement stage needs. STAN aligns flow representations across cities with adversarial and distribution-matching losses, and is tested with only 10 days of target data. CcFTL does privacy-preserving federated transfer that fills in a missing data modality and fine-tunes on 20% of target regions. Its authors note that they do not study smaller label fractions. MobiCLR learns region embeddings from taxi flows by contrastive self-supervision and transfers them from NYC to DC. It uses a single source and does not report the variance of its downstream metrics. All four papers motivate this proposal. Each fixes one label budget and one set of sources, so none can show how the value of a source changes with $N$. That relationship is exactly what we measure and model.

Two papers from the sample-size literature constrain the design. Scaling laws for language encoding models in fMRI studies brain-encoding models with a fixed amount of training data. It finds that widening the input embedding eventually stops helping, while adding observations helps a great deal. The authors call this a dimensionality bottleneck. We expect fusion of many urban sources to behave the same way. Algorithmic Accountability in Small Data proves that classification metrics are jagged and sometimes undefined on small evaluation sets. It reports that these artifacts fade around $N \ge 100$, and it proposes Cross-Prior Smoothing, a correction that borrows a confusion matrix from a data-rich group as a Dirichlet prior. Most realistic target-city evaluations sit below that threshold. Any claimed gain there must therefore be shown to exceed metric noise. We reuse Cross-Prior Smoothing for that purpose. Deep Neural Networks for High Dimension, Low Sample Size Data shows that gradients become unstable when features greatly outnumber samples. This informs our choice to keep the target-side model simple, rather than fine-tuning deep adversarial networks on a few dozen regions.

Core mechanism

Each region in a city is described by $G$ source groups. Examples are taxi-flow embeddings from a MobiCLR-style encoder, point-of-interest category counts, road-network statistics, and bike-share flows. Source group $g$ contributes a feature block $x_g$. Several data-rich cities $c = 1..C$ each yield fitted weights $\beta_g^{(c)}$. For a target city with $N$ labeled regions we fit

$$\hat\beta = \arg\min_\beta \sum_{i=1}^{N} \ell\big(y_i, \textstyle\sum_g x_{ig}^\top \beta_g\big) + \sum_{g=1}^{G} \lambda_g \lVert \beta_g - \bar\beta_g \rVert^2,$$

where $y_i$ is the outcome for region $i$, $\ell$ is squared or logistic loss, and $\bar\beta_g$ is the average source-city weight for group $g$. The penalty $\lambda_g$ is set by empirical Bayes: it is inversely proportional to the spread of $\beta_g^{(c)}$ across source cities. A source whose relationship to income holds steadily from city to city is pulled hard toward the shared weights, so it costs almost no target labels. A source whose relationship varies from city to city gets a weak pull and must be fit locally, so it only pays off once $N$ is large.

From this follows a testable prediction. The $N$ at which adding source $g$ starts to help should grow with its cross-city spread, scaled by its effective dimension. A simple variance-bias argument for this kind of shrinkage puts the break-even point roughly where $N$ exceeds the source's effective dimension times the ratio of its cross-city variance to the noise variance. From the fitted model we derive a rule: given $N$, include source $g$ only if its predicted break-even point is below $N$.

Research Program

The work has three parts. First, we assemble a multi-source, multi-city regional benchmark from the US setting used by MobiCLR: Chicago, NYC, and DC. Regions are census tracts. Sources are taxi-flow embeddings, OpenStreetMap points of interest and road statistics, and bike-share flows where they exist. Targets are American Community Survey median income and education, plus the CDC Social Vulnerability Index. Where the four-city Chinese data from CcFTL can be obtained, we add it as a classification setting: five-class consumption power, with grid cells as regions. H4M (Yaping Zhao et al.) provides a single-city check on Beijing house prices. There we vary $N$ by subsampling within the city, using spatially separated folds.

Second, we measure learning curves. For every combination of sources and every $N$ in {10, 20, 50, 100, 200, all}, we hold out each city in turn as the target. We draw labeled regions using spatially blocked sampling, because nearby tracts resemble each other and random splits make the effective $N$ look larger than it is. Every curve point is averaged over at least 50 draws. Classification metrics are reported both raw and with Cross-Prior Smoothing, and regression metrics carry spatial-block bootstrap intervals.

Third, we fit the shrinkage model and test whether cross-city spread, measured only on source cities, predicts the observed break-even $N$ for each source in the held-out city.

Baselines

The naive reference point is a target-only ridge regression on the single cheapest source, point-of-interest counts, fit with $N$ labels. We also run "fuse everything" target-only ridge. It is the same model on all source blocks concatenated, with the penalty tuned by nested spatial cross-validation using the same tuning budget as our method. This is the strongest simple competitor. A single tuned penalty might already absorb the dimensionality cost, and if it matches our method then our contribution narrows to the measurement study.

The strongest existing transfer method on the classification task is CcFTL, run on identical splits. Its paper reports an F1 gain of 8.9% over the strongest transfer baseline at a 20% label fraction in Xuchang. It also reports a 0.3–2% F1 cost from federated rather than centralized training. Because we compare centralized against centralized, the privacy machinery does not affect the comparison. On the regression tasks, the strongest existing method is MobiCLR embeddings with a linear probe. MobiCLR reports SVI improvements over GAT and TS2Vec of 12.54% in Chicago and 31.56% in DC. We found no result on Lacuna for any of these methods at $N < 100$ labeled regions, so we must establish those points ourselves.

Two ablations isolate the key ingredient. In the first, we shrink each block toward zero with per-group penalties, a group ridge, instead of toward the source-city weights. In the second, we shrink toward the source-city weights with one shared penalty rather than per-source ones.

Beating a baseline means a gap in held-out error at $N \le 50$ that exceeds the 95% spatial-bootstrap interval. For classification, the gap must also survive Cross-Prior Smoothing.

Why not just…?

Why not just concatenate all sources and let a cross-validated ridge penalty handle it? This is the real threat. With one penalty, a stable low-dimensional source and an unstable high-dimensional one are regularized identically. The fMRI bottleneck result suggests that wide inputs hurt when samples are fixed, even under ridge. Whether per-source, transfer-centered penalties matter in practice is an empirical question. The "fuse everything" baseline and the shared-penalty ablation settle it directly. If they tie with our method, we will report that.

Why not just fine-tune a deep transfer model like STAN or CcFTL with fewer labels? These methods train adversarial alignment end to end. The low-sample results in Deep Neural Networks for High Dimension, Low Sample Size Data suggest such training becomes unstable on a few dozen samples, and no paper in our file reports performance below a 20% label fraction. Running CcFTL across our $N$ grid answers the question. A fine-tuned deep model winning at $N = 20$ would refute our design choice.

Why not just pick the best single source by cross-validation? With $N = 20$, cross-validation scores are exactly the jagged, high-variance metrics Briscoe et al. warn about, so selection among sources becomes noisy. Our rule chooses sources using only source-city information, so it spends no target labels on selection. We include CV-selected best-single-source as a further baseline to test whether this advantage is real.

Decisive checks

The proposal succeeds if three things hold. First, held-out cities show a crossover: fusing all sources is worse than the shrinkage-selected subset at small $N$, and the two converge at large $N$. Second, break-even $N$ predicted from source-city spread correlates with observed break-even $N$ across sources and targets. Third, at least some published-scale gains at small $N$ prove to lie within metric noise once smoothing and spatial blocking are applied. If no crossover appears, more sources simply help at every budget, and the per-source machinery is unnecessary. We would report that as a negative result, along with the corrected small-$N$ learning curves.