Location Inference for Hidden Population with Online Text Analysis

Chuchu Liu, Ziqiang Cao, Xin LüResearch 2020 · DOI

Abstract

Understanding the demographics of hidden populations, such as men who have sex with men (MSM), sex workers, or injecting drug users, is of great importance for the adequate deployment of intervention strategies and public health decision-making. However, due to hard-to-access properties—including lack of a sampling frame, sensitivity issues, and reporting errors—traditional survey methods are largely limited. This study utilizes data from an active online MSM community in China to develop location inferring methods, achieving high-resolution mapping at a national level. We compare popular inference algorithms and propose a new hybrid model that achieves superior accuracy based solely on text content. This approach overcomes sparse location labeling in user profiles and can be extended to other hidden populations.

Introduction

Public health officials face significant challenges when studying "hidden populations." Groups like MSM often face social stigma or legal risks, leading them to avoid traditional surveys. The absence of a comprehensive sampling frame makes it difficult to deploy targeted interventions, such as HIV prevention programs, where they are needed most.

While social media provides a potential data source, privacy concerns lead many users to leave profile location fields blank or provide inaccurate information. However, within niche communities, users frequently disclose locations in the body of their posts to facilitate offline connections. The challenge lies in accurately extracting these locations from short, informal, and often "noisy" text containing internet slang.

Methodology

The research team evaluated three standard Natural Language Processing (NLP) approaches: Gazetteer-based matching, Part-of-Speech (POS) tagging, and Named Entity Recognition (NER).

The methodological workflow for data collection and location inference.

To improve upon these baselines, the authors introduced the Hybrid Voting Algorithm for Location Inference (HVA-LI). This model addresses a key weakness of simple gazetteer matching: the tendency to flag locations mentioned casually (e.g., "I want to visit Shanghai") rather than as a residence. By implementing "Strong Constraints"—requiring location words to appear alongside specific context keywords (e.g., "dating," "from")—and using an ensemble "voting" system between Gazetteer and NER methods, the researchers significantly filtered out false positives.

Data and Validation

The study analyzed over 13 million records from the "gay-bar" sub-community on Baidu Tieba, representing 628,360 unique users. A "ground truth" set of 10,000 manually labeled posts was used for benchmarking, while GPS coordinates from a subset of 30,948 users validated the final geographic distribution.

Results

Standalone evaluations revealed that the Gazetteer-based method was the most robust baseline, outperforming more complex NER and POS models in the informal text environment of social media. This is largely because internet slang often breaks the grammatical rules that more sophisticated models rely on.

Performance metrics (Success ratio S, Coverage ratio C, Targeting ratio T, and Partial accuracy P) for baseline methods.

The proposed hybrid Gazetteer & NER model significantly improved performance. While the baseline Gazetteer method achieved an absolute accuracy (Success Rate) of 50.3%, the hybrid approach reached 71.3%. Furthermore, the "Targeting Ratio"—the precision of identified locations—reached 91.4%. This allowed the team to infer locations for over 156,000 users who had no location data in their public profiles, effectively mapping 91% of the previously "invisible" community.

Discussion and Conclusion

The study highlights a "dating disclosure paradox": users who hide locations in formal profiles for privacy often voluntarily share them in informal text to meet social goals. The success of the HVA-LI model demonstrates that ensemble filtering is essential for handling the linguistic noise of internet slang.

Geographic analysis revealed that MSM populations in China are heavily clustered in economically developed and culturally open "hub" cities like Chengdu, Wuhan, and Chongqing, rather than being strictly tied to total population density. While posting frequency remains a limitation for "lurkers," this text-based inference tool provides a vital new capability for public health resource allocation where traditional data is scarce.

Summary updated

Abstract

Abstract Understanding the demographics of hidden population, such as men who have sex with men (MSM), sex workers, or injecting drug users, are of great importance for the adequately deployment of intervention strategies and public health decision making. However, due to the hard-to-access properties, e.g., lack of a sampling frame, sensitivity issue, reporting error, etc., traditional survey methods are largely limited when studying such populations. With data extracted from the very active online community of MSM in China, in this study we adopt and develop location inferring methods to achieve a high-resolution mapping of users in this community at national level. The performances of popular inference algorithms are compared to elucidate the most suitable approach. In addition, we propose a new hybrid model, which is proven to achieve the highest accuracy for inferring locations of online users only based on text content. This method is conducive to overcoming the sparse location labeling problem in user profiles, and can be extended to the inference of geo-statistics for other hidden populations.
BibTeX
@inproceedings{liu2020location,
  author = {Chuchu Liu and Ziqiang Cao and Xin Lü},
  title = {{Location Inference for Hidden Population with Online Text Analysis}},
  booktitle = {Research},
  year = {2020},
  doi = {10.21203/rs.3.rs-56873/v1},
}
Figure