Virtual Worlds as Ground-Truth Testbeds for Hidden-Population Estimators

Methods for counting hidden populations, such as people who inject drugs, undocumented migrants, or sexual minorities under stigma, share an awkward weakness: they are almost never checked against a true count. These methods rely on respondent-driven sampling (RDS), a chain-referral design in which each participant hands a few coupons to peers. Estimators then correct the resulting chains for over-representation of well-connected people and extrapolate a total size. They are validated on synthetic graphs or on static social-network snapshots. In the field they are compared against other estimates, never against the truth. Meanwhile, research on locomotion habituation shows that people in virtual worlds keep their offline movement habits. They walk instead of fly, stay on paths, crowd at social destinations, and slowly unlearn these norms with experience. Those habits create exactly the clustered, bottlenecked contact structure that is known to break hidden-population estimators.

This note proposes running real chain-referral studies inside a persistent virtual world. The world's position telemetry gives a passive, complete census of who was present and who came near whom. That census supplies the ground truth the field has lacked. The study can then measure how often and why population-size estimators fail on real human referral behavior, and whether habituated movement predicts those failures. The finished testbed also supports two modest method changes, both evaluated against that ground truth: spatially defined separation of referral trees, and aggregate movement covariates.

Core Claim

Two claims are tested. First, a virtual region polled for avatar positions supplies a known population size $N$ and an observed encounter graph. A coupon-based RDS run inside that region therefore produces the first field dataset where estimator error can be computed directly. Second, the failures of current estimators on such data are structured, not random. Regions where movement is strongly channeled, meaning high path adherence and heavy gathering, should show larger underestimation by degree-weighted estimators. They should also show the most benefit from estimators that discount matches between referral trees that are spatially entangled. If either claim fails, the field learns something valuable: synthetic-graph validation is adequate, or habituation is irrelevant to estimator bias.

Grounding in Prior Work

The habituation side supplies the measurement technique and the structural hypothesis. Please keep off the grass polled avatar positions every 0.5 seconds in Second Life's Hyde Park, covering 1,088 avatars and about 45,000 points. It found over 80% of movement at ground level, path adherence of 58% for avatars three days old or younger versus 39% for veterans ($p < 0.033$), and a collapse in adherence when social destinations sat on grass. We reuse that polling design to build the census. We also use its adherence measure as a per-region index of how strongly movement is channeled. Its noted "alt account" problem, where veterans open new accounts that look like novices, constrains our estimand: the census counts accounts, not people, and we say so. Something In The Way We Move computed place entropy and step entropy (how evenly a player spreads over map sectors, and how unpredictable their steps are) and showed that these predict offline gender better than avatar choice. We borrow the entropy definitions as candidate auxiliary covariates. We deliberately do not use them for individual profiling, for reasons given below.

The sampling side supplies the estimators under test and motivates the gap. One-step estimation of networked population size estimates size from a single chain using hashed participant traits. It reports that its degree-adjusted estimator $n_2$ significantly underestimated size on the clustered Brightkite network, while $n_3$, which counts matches only across referral trees grown from different seeds, was "highly accurate." The file reports no exact number for this. That result motivates our hypothesis, but Brightkite is a static friendship graph with simulated referral. Hidden Population Estimation with Indirect Inference and Auxiliary Information treats RDS as stochastic exploration of a graph. It uses indirect inference, which calibrates a simulator so that a summary statistic (here the maximum-likelihood estimate) matches the data. It reports that this cut arrival-rate mean squared error to less than half the MLE's in low-sampling, high-degree settings, and improved the Kohtla-Järve PWID size estimate by about 20% using auxiliary covariates such as ART status. That paper is both our strongest baseline and the source of an assumption we can test directly: exponential recruitment waiting times. The WebRDS deployments in Vietnam (676 participants, 24 waves, equilibrium for 16 of 17 variables within 7 waves) and Sweden (123 participants from 13 seeds, with chains vulnerable to channel friction) supply the coupon protocol we copy. They also show what a realistic chain looks like. Reconstructing Patterns of Information Diffusion from Incomplete Observations is a caution: incomplete observation makes trees look "stringy." Our telemetry lets us check observed referral trees against the encounter graph to see how much branching went unobserved.

Research Program

The build has three parts. First, a census layer. A LibOpenMetaverse-style client polls a set of contiguous regions at 0.5-second intervals for four to six weeks. It logs hashed account identifiers and positions, and from these derives a time-stamped co-presence graph, with an edge when two avatars are within a fixed radius for a minimum dwell time. Second, a referral layer. In-world kiosks seed a short survey, and each respondent receives three or four unique coupon codes redeemable by any avatar, as in WebRDS. Coupons are matched to accounts only through a one-way hash, following the Khan et al. design. Third, an analysis layer that computes every estimator on the referral data alone and compares it to the census $N$ for the same regions and window.

The method contribution is narrow. In addition to $n_3$'s rule of counting matches only across different seed trees, we add a spatial-separation variant. Matches count only between trees whose members' movement footprints (region-sector occupancy histograms) overlap below a threshold. The reasoning is that, in a channeled world, two seed trees can be distinct as referral trees yet share the same paths and gathering points. Separately, we add region-level aggregates as auxiliary information in indirect inference: mean place entropy, path adherence, and gathering concentration. Because these are aggregates, they carry no individual biometric signal. This addresses the tension the file flags between anonymity hashing and movement fingerprinting.

Baselines

The naive reference point is the uniform capture-recapture estimator $n_1$ from Khan et al., together with the raw count of unique respondents. Neither corrects for network structure, so both show how much any correction buys. The standard network-aware baseline is $n_2$, the harmonic-mean degree adjustment. It is the default practitioners would reach for, and we predict it will underestimate most in high-adherence regions. The strongest existing structural competitor is $n_3$. It already targets the clustering failure we study, so our spatial-separation variant must improve on it specifically, not just on $n_2$. No numeric Brightkite accuracy for $n_3$ was reported, so beating it will be judged only on our own ground truth. The strongest inference-based competitor is the indirect-inference estimator of Naggar Weltz et al. We run it with its published structure and with self-reported survey covariates only, giving it the same simulation budget and tuning grid as our variant. Its reported reference points are arrival-rate MSE below 50% of the MLE's and roughly 20% better size estimation on Kohtla-Järve. The MLE itself is also run.

All estimators see identical referral data, identical hashed matches, and the same seed sets. Beating a baseline means lower relative absolute error $|\hat N - N|/N$, judged by bootstrapping over seed subsets with confidence intervals that exclude zero difference, across at least eight regions with differing adherence levels. The key ablations are two. The first is spatial-separation $n_3$ with footprints computed from shuffled telemetry, which isolates whether the spatial information or merely the extra pruning helps. The second is indirect inference with movement aggregates replaced by random region labels.

Why not just…?

Why not just simulate RDS on synthetic or fitted geometric graphs? This is the real threat, and the design tests it directly. We will simulate referral on the observed co-presence graph and on a random geometric graph fitted to its degree distribution and density. We then compare the simulated estimator errors with the errors from real referral. If simulation on the fitted graph reproduces the real errors within bootstrap uncertainty, the virtual-world fieldwork adds nothing beyond a better graph generator, and we will report that. The prior evidence on synthetic topologies from Khan et al. and Naggar Weltz et al. cannot settle this, because neither had real referral behavior over a known population.

Why not just validate on Brightkite or similar static networks? Brightkite has a known node count. However, its referral is simulated and its edges are declared friendships, not encounters, so it cannot reveal non-exponential waiting times, coupon abandonment, or channeled co-presence. The Sweden study shows that real chain fragility is a dominant failure, and no static dataset contains it.

Why not just recruit through in-world advertising and count respondents? The Sweden WebRDS paper describes banner recruitment as reaching only those who visit particular sites at particular times. Our census allows a direct test: a kiosk-only arm without coupon referral, whose respondent count and composition we compare to the census, giving a measured coverage rate rather than an assumed one.

Why not just use individual movement entropy to verify eligibility or reweight respondents? Murray et al. show that movement acts as a fingerprint, which defeats the hashing that makes stigmatized people willing to participate. We use only region-level aggregates. A side analysis will report how well individual-level entropy re-identifies hashed respondents, so the privacy cost is quantified rather than assumed away.

Decisive Checks

Three results decide the program. First, across regions, the correlation between path adherence or gathering concentration and $n_2$'s signed error: a clear negative relation supports the habituation hypothesis, and a null result rejects it. Second, whether spatial-separation $n_3$ reduces relative error versus standard $n_3$ in high-adherence regions without hurting low-adherence ones, with the shuffled-footprint ablation ruling out a pruning artifact. Third, a goodness-of-fit test of observed coupon-redemption times against the exponential assumption in Naggar Weltz et al., and whether a decaying-rate alternative changes their size estimate's error against the census. We state the limitation plainly: Second Life users are not stigmatized hidden populations. The testbed validates estimator mechanics under real referral and real clustered contact, not willingness to participate under threat.