Correcting Biases in Online Social Media Data Based on Target Distributions in the Physical World

Zhu Wang, Zhiwen Yu, Renjie Fan, Bin GuoIEEE Access 2020 · OpenReview

Abstract

Social media is an important data source for understanding public opinion and enabling better decision-making in fields like public policy, transportation, and healthcare. However, online social media data collected with common mechanisms are usually biased compared with the distribution of related features in the physical world. Sampling issues, especially selection bias, can have far-reaching implications for data analysis. In this paper, we propose to address the bias calibration issue by adopting a data resampling approach. Specifically, we develop a data resampling algorithm based on the stochastic stability theory of Markov Chains to collect data samples from a biased dataset to calibrate these biases. By regarding the data resampling process as status transitions of a stochastic variable, the algorithm leverages the stationary distribution of Markov Chains to build an acceptance matrix to control the resampling process, optimizing the original dataset towards target distributions in the physical world. Experimental results demonstrate that the proposed algorithm can effectively output datasets with similar distributions to the target ones.

The Challenge of the "Digital Mirror"

We often treat social media as a proxy for "what the world thinks," but there is a massive gap between the digital and physical worlds. This is known as selection bias. For example, while platforms like Twitter or Weibo are popular for research, their user bases are disproportionately younger and tech-savvy compared to the general population. If a researcher uses these platforms to study public health or transportation without correction, their results will be heavily skewed toward the opinions and behaviors of specific cohorts, such as those born in the 1990s, while neglecting older demographics.

Figure 1: Comparison of age distribution between virtual (social media) and physical (real-world census) datasets, highlighting significant selection bias.

As shown in Figure 1, the "Virtual" distribution (yellow) peaks significantly in the 24-33 age bracket, whereas the "Physical" world (purple) has a much higher representation of individuals over 45. Existing solutions often involve complex regression models tailored to a single task. This research seeks a more universal solution: a way to transform a biased dataset into a representative one that can be used for any downstream analysis.

Methodology: Target Distribution based Resampling

The proposed Target Distribution based Resampling algorithm moves beyond simple weighting. Instead, it uses a resampling approach inspired by Markov Chain Monte Carlo (MCMC) methods to manage "status transitions" between data points.

Figure 2: The proposed resampling model framework, illustrating the integration of target distribution statistics and transition-acceptance matrices.

Key Components of the Model:

  1. State Mapping: Different feature intervals (e.g., age groups) are treated as "states" in a Markov Chain.
  2. Transition-Acceptance Matrix: The core innovation is building a matrix that controls the resampling process. By leveraging the stationary distribution theory of Markov Chains, the algorithm calculates the probability of "accepting" a new data point into the final set.
  3. Stochastic Stability: The algorithm ensures that as more samples are collected, the distribution of the new dataset converges to the target distribution (the real-world ground truth).
  4. Balancing Logic: The system automatically handles both under-sampling (dropping samples from over-represented groups) and over-sampling (retaining and potentially duplicating samples from under-represented groups like the elderly).

Experimental Evaluation and Results

The researchers validated their approach using an online dataset of 15,940 Sina Weibo users and compared it against the China Population Census as the physical world ground truth. The study primarily focused on age group distributions.

  • Rapid Convergence: The difference between the resampled data and the real-world census—measured by Jensen-Shannon Divergence—dropped sharply. After collecting just 500–1000 samples, the divergence fell below 0.02, indicating a very high match.
  • Visual Alignment: Before the algorithm, the Weibo data showed a massive spike for users in their 20s. After resampling, the new dataset’s age curve closely aligned with the offline physical world distribution.
  • Universal Utility: Unlike previous methods that fix bias during a specific prediction task, this method produces a new, cleaned dataset. This allows researchers to "fix" the data once and then use it for multiple independent studies.

Limitations and Future Work

While effective, the algorithm faces a "duplication trap." If a specific demographic is extremely rare in the online data (e.g., very elderly users on a niche app), the algorithm will over-sample them, leading to many duplicate records. This can reduce the variety of the data even if the distribution is statistically correct. Future work will focus on refining the acceptance matrix to minimize duplicates while maintaining distributional accuracy, particularly when the desired sample size is large relative to the original dataset.

Summary updated

Abstract

Social media is an important data source. Billions of posts, likes, and connections are created by people all around the world every day. The promises of such social media data are plentiful, including understanding “what the world thinks” about a social issue, brand, product, celebrity, or other entity, as well as enabling better decision-making in a variety of fields including public policy, transportation, healthcare, and economics. However, while the validity of these data-driven researches are largely dependent on the accuracy and representativeness of the used data, online social media data collected with common mechanisms are usually biased compared with the distribution of related features in the physical world. For example, sampling issues, especially selection bias, associated with such data sources can have far reaching implications for data analysis and interpretation. Therefore, how to calibrate biases in the online social media data set to achieve unbiased results becomes a significant and urgent problem. In this paper, we propose to address the bias calibration issue by adopting a data resampling approach. Specifically, we develop a data resampling algorithm based on the stochastic stability theory of Markov Chains to collect data samples from the given biased data set to calibrate possible biases. By regarding the data resampling process as status transitions of a stochastic variable, the algorithm leverages the stationary distribution of Markov Chains to build an acceptance matrix to control the resampling process, and thus optimize the original dataset towards target distributions in the physical world. Experimental results demonstrate that the proposed algorithm can effectively output data sets with similar distributions to the target ones.
BibTeX
@inproceedings{wang2020correcting,
  author = {Zhu Wang and Zhiwen Yu and Renjie Fan and Bin Guo},
  title = {{Correcting Biases in Online Social Media Data Based on Target Distributions in the Physical World}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=1QLxfXPtDS},
}
Figure