Abstract
We develop methods for estimating the size of hard-to-reach populations from data collected using network-based questions on standard surveys. Such data arise by asking respondents how many people they know in a specific group (e.g. people named Michael, intravenous drug users). The Network Scale up Method (NSUM) is a tool for producing population size estimates using these indirect measures of respondents' networks. We extend previous maximum likelihood estimators by treating personal network sizes as random effects, yielding principled statements of uncertainty. This allows us to generalize the model to account for variation in people's propensity to know people in particular subgroups (barrier effects), such as their tendency to know people like themselves, as well as their lack of awareness of or reluctance to acknowledge their contacts' group memberships (transmission bias). NSUM estimates also suffer from recall bias, in which respondents tend to underestimate the number of members of larger groups that they know. We propose a data-driven adjustment method to deal with this. Our methods perform well in simulation studies and in back estimates of real sample data. We apply them to data from a study of HIV/AIDS prevalence in Curitiba, Brazil. Our results show that when transmission bias is present, external information about its likely extent can greatly improve the estimates.
Introduction: The Challenge of Hidden Populations
Estimating the size of marginalized or "hidden" populations is notoriously difficult because members often avoid traditional census or survey methods. The Network Scale-Up Method (NSUM) bypasses this by asking a representative sample of the general public: "How many people do you know in group X?" (e.g., "How many people named Michael do you know?" or "How many intravenous drug users do you know?").
However, traditional NSUM models suffer from three major flaws:
- Barrier Effects: Social networks aren't random; people tend to associate with others similar to them in age, race, or status.
- Transmission Bias: Respondents might not know that a contact belongs to a stigmatized group, or they may be reluctant to admit it to a surveyor.
- Recall Bias: People systematically forget contacts in large groups (under-recall) and over-count contacts in small, memorable groups.
Methodology: A Bayesian Hierarchical Framework
To address these limitations, this research moves away from traditional "fixed effects" approaches toward a Bayesian hierarchical framework consisting of four nested models that account for increasing levels of social complexity.
- Random Degree Model: Instead of assuming a person's total network size ("degree") is a fixed number, it is treated as a random effect following a log-normal distribution. This "regularizes" the estimates, preventing extreme answers from skewing the results.
- Barrier Effects Model: This introduces a Beta distribution to model the probability of knowing someone in a specific group, accounting for "overdispersion"—the fact that some people know significantly more (or fewer) people in a subgroup than random chance would suggest due to nonrandom mixing.
- Transmission Bias Model: A multiplier ($\tau_K$) is added to represent the proportion of contacts that are actually reported, accounting for lack of awareness or stigma.
- Combined Model & Recall Adjustment: The combined model integrates these effects but requires external information for calibration. Additionally, a data-driven post-processing method was developed for recall bias, using "back-estimates" to find a log-linear relationship between estimated and true sizes for known groups.
Data and Validation
The models were validated using three primary sources:
- McCarty et al. (2001): A US-based survey involving 1,375 adults asked about 29 groups with known sizes (e.g., people named "Walter," diabetics, or postal workers).
- Curitiba, Brazil Study (Salganik et al., 2011): A real-world application estimating the prevalence of heavy drug users, which included a "game of contacts" to independently estimate transmission bias.
- Simulated Data: Used to test how the models handle varying levels of bias and social mixing.
Key Insights and Results
- Improved Accuracy: In simulations with barrier effects, the new model reduced the Mean Absolute Relative Error (MARE) by 12% compared to traditional methods.
- The Stigma Gap: Transmission bias is the most significant hurdle. In the Curitiba study, the estimate for heavy drug user prevalence jumped from 3.3% (without accounting for transmission bias) to 6.3% (with the correction). This suggests that previous methods may have been underestimating these populations by nearly half.
- Better Coverage: The Bayesian credible intervals correctly captured the true population size in 90-97% of simulations, whereas older models often had 0% coverage when bias was present.
- Systematic Recall: The research identified a consistent linear relationship on the log scale between reported contacts and actual population size, allowing for a straightforward mathematical correction for recall bias.
Conclusion
The primary limitation of this framework is its dependence on external information. To correct for transmission bias, researchers must use informative priors or separate studies (like a "game of contacts") to estimate group visibility. Without this external data, the model cannot fully account for hidden information. However, by providing a principled way to incorporate such data, the NSUM R package now offers a more robust tool for public health researchers to estimate the size of populations that are otherwise invisible to standard survey techniques.
Summary updated