When we collect data through surveys, clinical trials, or online platforms, we rarely obtain a perfect miniature replica of our target population. Some groups are easier to reach, more likely to respond, or intentionally oversampled to ensure sufficient data. For instance, in large-scale health studies, certain demographic groups are deliberately oversampled to ensure statistical power within those subgroups. If we analyze this data without adjustment, our conclusions will be biased toward the overrepresented groups.
To bridge the gap between a collected sample and a target population, researchers rely on a family of statistical techniques designed to rebalance the data. The core intuition is simple: if a specific demographic group is underrepresented in a sample relative to the general population, each individual from that group should carry more weight in the final analysis. Conversely, individuals from overrepresented groups should carry less weight.
Post-Stratification and Calibration
The most straightforward way to rebalance a sample is post-stratification. Imagine dividing both your sample and the true population into distinct buckets, or "strata," based on known characteristics like age and sex. Instead of calculating a simple sample average, you calculate the average within each bucket and then combine these averages using the true population proportions as weights.
This approach works beautifully when the joint distribution of all demographic variables is known. However, as the number of variables increases, the number of buckets grows exponentially, leading to "empty cells" where a specific combination (e.g., 18–24 year old male college graduates in a specific rural county) contains no respondents.
To overcome this, researchers use calibration weighting and propensity score matching. For example, Baobao Zhang demonstrated how propensity score matching can weight online survey samples against national census data to accurately capture public opinion on complex topics in U.S. Public Opinion on the Governance of Artificial Intelligence. Similarly, Dasom Lee et al. utilized calibration weighting in Doubly robust estimators for generalizing treatment effects on survival outcomes from randomized controlled trials to a target population to adjust the covariate distributions of randomized controlled trials, ensuring that survival outcome estimates could be safely generalized to a broader target population without directly modeling participation probabilities.
Raking and Multilevel Regression
When the full joint distribution of the population is unavailable, but marginal distributions (e.g., the total percentage of women, the total percentage of seniors) are known from census data, researchers turn to Iterative Proportional Fitting, commonly known as raking.
Raking operates by adjusting the weights along one demographic margin at a time. First, it scales the weights so the sample's gender distribution matches the population. This adjustment might throw off the age distribution, so the next step scales the weights along the age margin. The algorithm iterates back and forth across all margins until the sample's marginal distributions converge to the known population benchmarks.
Jacy Reese Anthis et al. applied this technique in What Do People Think about Sentient AI? to align longitudinal survey data on public perceptions of AI minds with broader public demographics. Similarly, Daniel A. Antiporta et al. used raking in Depressive symptoms among Peruvian adult residents amidst a National Lockdown during the COVID-19 pandemic to correct for selection bias in opt-in digital surveys during pandemic lockdowns, ensuring that estimates of depressive symptoms accurately reflected the national population.
An alternative, highly flexible approach is Multilevel Regression and Post-stratification (MRP). Instead of adjusting individual weights directly, MRP fits a multilevel model to predict the outcome of interest based on demographic predictors. This model "borrows strength" across related groups to estimate the expected outcome for every single demographic cell—even those with few or no respondents in the sample. Finally, these cell-level predictions are aggregated using the true population counts of each cell.
Fumeng Yang et al. leveraged MRP in Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms to evaluate how election forecast visualizations influence public trust across diverse sub-groups while maintaining overall national representativeness. In a different domain, Rock Yuren Pang et al. used MRP in Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems to generate representative cross-cultural estimates of human-annotated labels, mitigating demographic imbalances in recommendation system training data.
Downstream Inference via Weighted Regression
Once population-representative weights are calculated, they must be integrated into downstream statistical models to evaluate relationships between variables. In standard regression models, every observation contributes equally to the loss function. In a weighted regression framework, each observation's contribution to the objective function is multiplied by its assigned survey weight.
For instance, when fitting a weighted logistic regression model, the log-likelihood function is modified so that individuals representing under-sampled segments of the population have a greater impact on the optimization gradient. This ensures that the resulting model coefficients reflect the true population-level associations rather than the idiosyncratic correlations present in an unadjusted sample.
This mechanism is widely applied across epidemiological and clinical research. For example, You Lü et al. used weighted logistic regression in Quitting smoking after a cancer diagnosis is associated with high-risk neutrophil-to-lymphocyte ratio among tobacco use-related cancer survivors to uncover associations between smoking cessation timing and high-risk immune biomarkers among cancer survivors, adjusting for unequal selection probabilities in complex survey designs.
Similarly, Qingjia Zeng et al. applied weighted multivariate logistic regression in Association of Blood Selenium Levels with Diabetes and Heart Failure in American General Adults: a Cross-sectional Study of NHANES 2011–2020 pre to national health datasets to untangle the complex, non-linear relationships between blood selenium levels, diabetes, and heart failure. By incorporating multi-stage probabilistic sampling weights, the authors ensured that the identified clinical thresholds remained valid for the general adult population.