Evidence-Anchored Delphi: Closing the Urban Monitoring Loop Without Herding

Cities now measure a great deal about their streets, including foot traffic, shop openings and closures, citizen complaints, and sensor readings. Very little of this measurement turns into decisions. Writers on smart-city systems call this the "translation gap": data are collected and visualized but never routed into a repeatable process that decides which street needs which intervention next. The Delphi method is a candidate for that process. It is a decades-old protocol in which a panel rates items anonymously, sees a statistical summary of the group's ratings and anonymized reasons, and re-rates over several rounds until the ratings stabilize. Delphi was built for forecasting and consensus under uncertainty, which describes deciding what a renovated district needs. It carries a known weakness, though. Showing people the group median pulls them toward it whether or not the median is right. In a city this can bury a correct minority view, such as the one resident who knows an informal street market has quietly died.

We propose to change what flows back to the panel between rounds. In an evidence-anchored Delphi, each round returns the newest monitoring data for the items under debate alongside, or instead of, the peer distribution. The panel's consensus then becomes the ratification step of a closed loop that repeats every time new telemetry arrives. Our central empirical claim can be tested directly. In urban forecasting-and-prioritization tasks, the accuracy gains attributed to Delphi iteration come mainly from evidence feedback. Peer-statistic feedback mainly produces conformity, and it erodes correct dissent. If the claim holds, the practical recommendation for cities is specific: loop the telemetry, not the median.

What exists and what is missing

Work on closed-loop urban governance supplies the monitoring half. Street Vitality Evaluation of the Mengzi East Street Historical District couples street-network centrality measures with point-of-interest density. It defines a per-street "deviation degree," the mismatch between how accessible a street is and how much activity it actually hosts. It reports that renovation weakened the space–function correlation from r=0.42 to r=0.32 and left severe localized imbalances, such as Renmin Middle Road at 4.160. That paper motivates our work, because the monitoring detected problems with no downstream decision process. Its authors also note that structural metrics miss informal economies and lived experience, which is exactly where a human panel might add information. Gamifying Data Visualization in Smart Cities roughly doubled citizen submissions, from 180–250 to 370–480, but explicitly leaves those submissions untranslated into policy. The Preface by Daly argues for bidirectional loops without specifying a mechanism.

Work on participatory AI supplies deliberation that is not closed-loop. Large Language Model for Participatory Urban Planning simulates 1,000 census-based resident agents who critique a planner agent's zoning in an open, rotating "fishbowl" discussion. AI Agent as Urban Planner turns consensus into a multi-agent reinforcement-learning reward on Kendall Square parcels. Both are one-shot and offline, and neither uses anonymity or controlled feedback. HABERMOLT aggregates AI-representative statements through a single ranked vote. Two of its findings constrain our design. First, agent opinions collapse toward the model's own prior, with pairwise cosine similarity of 0.745 versus 0.649 for human interviews, so simulated panels cannot stand in for people. Second, it finds that broadly agreeable statements become vague while concrete ones fracture consensus. EuroCon shows language models defaulting to the dominant party's stance, which is the machine analogue of herding. Clinical Delphi studies such as the behavioral-trials priorities study and the CVR-based panel in VISION-Cog supply the protocol itself: anonymous rounds, median and interquartile-range feedback, mandatory rationales from outliers, and threshold-based retention. None of this work feeds live measurements into rounds, and none separates why Delphi panels converge.

Our contribution is therefore a causal decomposition, not another pipeline. We ask what each ingredient of Delphi feedback does once the loop carries real telemetry. We also score the answer against outcomes that are measured later, rather than against satisfaction ratings.

Core Claim and mechanism

Panelist $i$ rates item $j$ (for example, "how much will this street's deviation degree rise over the next 12 months, and how urgently does it need intervention"). Between rounds, the rating change is modeled as

$$\Delta x_{ij} = \alpha\,(m_j - x_{ij}) + \beta\,(e_j - x_{ij}) + \varepsilon_{ij},$$

where $x_{ij}$ is the current rating, $m_j$ is the panel median shown as feedback, $e_j$ is the rating implied by the newly released telemetry (a simple extrapolation of the item's measured deviation trend), and $\varepsilon_{ij}$ is residual change. The coefficient $\alpha$ measures herding and $\beta$ measures evidence uptake. Randomizing what feedback panels receive identifies both coefficients. We predict three things. Peer-only feedback yields large $\alpha$ and improves accuracy only where the median happens to be right. Evidence feedback yields large $\beta$ and higher accuracy. Combined feedback lies between the two unless outlier rationales carry information the telemetry lacks. We test the third prediction by coding whether rationales cite local knowledge absent from the sensor feed.

The consensus rule borrows the Content Validity Ratio logic from clinical Delphi: an intervention item is ratified only if a fixed proportion of panelists independently endorse it. We add a minority channel. Any item held by a persistent, justified outlier group through the final round is forwarded to the next monitoring cycle as a "watch" item and not discarded. The next telemetry release then adjudicates it. This is where the loop closes: dissent is tested by data rather than suppressed by the median.

Research Program

We will build a lightweight asynchronous Delphi platform with round-by-round logging. It will be fed by monthly point-of-interest and street-network pulls (OpenStreetMap tooling from SJTU-CILAB/OSM-dataset, plus commercial map APIs as in the Mengzi study). The pipeline recomputes Mao et al.'s deviation degree per street segment each month. The core study is prospective, which avoids leakage because the outcome does not yet exist when panelists rate. It covers three renovated or renovating historic districts, each with roughly 60–150 street segments. Panels of about 12 people, a mix of municipal planners, shop owners, and residents, are randomized to one of four feedback arms: independent re-rating with no feedback, peer statistics only (classic Delphi), telemetry only, and peer statistics plus telemetry. Each district has three panels per arm, about 430 participants in total. Every quarter, panels run three rounds on the same forecasting-and-priority items as new data arrive. Outcomes are scored against the realized deviation change 12 months later.

In parallel, we run LLM persona panels built from census profiles as in Zhou et al. under the same four arms plus an open-discussion fishbowl arm. We treat these panels as a cheap stress test of the feedback mechanics, never as evidence about residents, given HABERMOLT's prior-collapse result. Comparing $\alpha$ and $\beta$ between LLM and human panels is itself informative. If LLM panels show higher herding, that is a concrete warning against using AI delegates inside governance loops. A retrospective replay on the Mengzi 2021→2025 snapshots supports pilot and protocol debugging only, because local participants already know the outcome.

Baselines

The naive reference is the telemetry-only extrapolation forecaster with no humans. It ranks segments by their measured deviation trend and applies Mao et al.'s threshold zoning to decide what gets flagged. This is the "just automate it" point. If panels cannot beat it, deliberation adds nothing measurable here. We will give it the same monthly data the evidence arms see and tune its smoothing window on the first two quarters only.

The second reference is a single anonymous survey: the round-one median, with no iteration. It isolates whether any iteration helps at all, and it costs a single round of the same participants' time.

The strongest existing method for panel-based urban planning is the open fishbowl deliberation of Large Language Model for Participatory Urban Planning, which reports 11.2%–12.9% higher resident satisfaction and 8.0%–11.9% higher inclusion than reinforcement-learning and genetic-annealing baselines. Those are simulated satisfaction scores, not forecast accuracy. We found no reported result on Lacuna for any deliberation method on prospective street-level forecast accuracy. We will run the fishbowl protocol with the same personas, items, and round budget in the LLM arm, and as an open moderated discussion with matched time for one human panel per district.

The key ablation compares classic peer-only Delphi against telemetry-only feedback. The proposal beats classic Delphi if telemetry-fed panels achieve a higher rank correlation with realized change and a higher correct-dissent survival rate, with both confidence intervals excluding zero across districts. Beating the extrapolation forecaster means that a blend of panel and forecaster outperforms the forecaster alone on held-out quarters.

Why not just…?

Why not just trigger interventions whenever the deviation degree crosses a threshold? This is the most serious threat. Mao et al.'s metric already yields explicit thresholds, and if the metric captured everything, panels would be redundant. The same paper, however, lists informal street economies, pedestrian surges, and lived experience as blind spots of structural metrics. Our blended-forecaster test settles the question empirically. If panel input adds no accuracy over the extrapolator, we will report that the closed loop should be automated, and the Delphi component should shrink to ratifying which type of intervention to apply.

Why not just run one anonymous survey and take the median? The wisdom-of-crowds literature suggests independent averaging is often as good as iteration, and it is cheaper. Our no-feedback and single-survey arms measure this directly. If iteration adds nothing beyond the round-one median, then classic Delphi iteration is not worth its well-known attrition cost, and only evidence re-rating needs justifying.

Why not just hold open discussion, as in fishbowl or town-hall formats? EuroCon's dominant-party bias and the Delphi literature's case against unblinded committees predict that open discussion suppresses minority knowledge. Our correct-dissent survival metric, the share of round-one outliers who were later proven right and who kept their position, measures this rather than assuming it.

Why not just have a large model synthesize all complaints and sensor data into a plan? HABERMOLT's homogeneity finding and UrbanLLM's 49.9% complex-task accuracy for GPT-4o (UrbanLLM) argue against trusting single-shot synthesis. The decisive answer is still empirical: we include a single-shot model forecast as an additional scored baseline.

Decisive checks

Three results would settle the claim. First, telemetry-fed arms must beat the peer-only arm on the Spearman correlation between panel priority and realized 12-month deviation change. Second, the peer-only arm must show a significantly larger herding coefficient $\alpha$ and a lower correct-dissent survival rate. Third, the blended panel-plus-forecaster must beat the telemetry extrapolator alone. The third check determines whether humans belong in the loop at all. If the first two hold but the third fails, the contribution narrows to a warning that classic Delphi feedback herds. If all three fail, evidence-anchored Delphi offers nothing over simpler options in this setting, and we will say so.