In empirical research, the ideal way to understand a population is to sample its members randomly. However, practical constraints—such as high costs, logistical hurdles, and data privacy regulations—often force researchers to rely on convenience sampling, where participants are recruited based on their accessibility rather than their representativeness. Anthis et al. argue that while these methods are scalable, they introduce inherent biases that limit the ability to apply findings to the broader world.
The Mechanism of Selection Bias
The primary failure of convenience sampling is selection bias. When researchers recruit from a single institution, a specific online platform, or a localized geographic area, the resulting data reflects the unique characteristics of that environment rather than the general population.
For example, Payne et al. note that small convenience samples in organizational studies often suffer from self-selection bias, where individuals with the strongest opinions are the most likely to respond. Similarly, Hullman found that recruiting via social media can overrepresent practitioners who are already highly engaged with the research topic, skewing the results toward an "expert" perspective that does not reflect the broader field.
The Representativeness Gap
This bias creates a "representativeness gap" that undermines external validity—the extent to which a study's results can be generalized. Across diverse fields, researchers have documented how this gap manifests:
- Public Health: Ansumana et al. found that sampling from a single medical center captures only those with the means or motivation to seek hospital care, potentially misrepresenting infection rates in the wider community. Stojanovic et al. further observed that convenience samples in vaccine studies often overrepresent women and highly educated individuals, leading to an underestimation of hesitancy in the general public.
- Psychology and Education: Ma et al. and Dong et al. highlight that findings from a single school or region cannot account for the socio-economic and regional variations essential to understanding adolescent behavior or language learning.
- Technology Adoption: Gupta and Quaddus argue that non-random sampling prevents the statistical extension of adoption drivers (like trust or ease of use) to entire national or cultural groups.
Missing the Margins
Convenience sampling frequently overlooks marginalized or less accessible subgroups. Zhang et al. point out that in studies of marginalized communities, recruiting from specific physical or online locations misses individuals who do not frequent those spaces. Fu et al. similarly note that institutional sampling may miss those not enrolled in formal systems, such as education, which can lead to a fundamental misunderstanding of ecological risk factors.
Even in large-scale population modeling, the source of the sample matters. Abu‐Akel et al. demonstrate that online convenience samples may fail to reflect the clinical and demographic diversity found in representative epidemiological data, potentially leading to inaccurate mathematical models of human traits.
Moving Toward Rigor
To address these limitations, the authors of these studies consistently advocate for a shift toward randomized sampling and larger, more diverse cohorts. Naveed et al. suggest that future research must prioritize randomized techniques to ensure results are representative of diverse socio-economic environments. While convenience sampling remains a useful tool for generating initial theories—as Moore suggests—it cannot serve as a substitute for the statistical rigor required to make broad claims about human behavior, health, or society.