Abstract
Social media is an important data source for understanding public opinion and enabling better decision-making in fields like public policy, transportation, and healthcare. However, online social media data collected with common mechanisms are usually biased compared with the distribution of related features in the physical world. Sampling issues, especially selection bias, can have far-reaching implications for data analysis. In this paper, we propose to address the bias calibration issue by adopting a data resampling approach. Specifically, we develop a data resampling algorithm based on the stochastic stability theory of Markov Chains to collect data samples from a biased dataset to calibrate these biases. By regarding the data resampling process as status transitions of a stochastic variable, the algorithm leverages the stationary distribution of Markov Chains to build an acceptance matrix to control the resampling process, optimizing the original dataset towards target distributions in the physical world. Experimental results demonstrate that the proposed algorithm can effectively output datasets with similar distributions to the target ones.
The Challenge of the "Digital Mirror"
We often treat social media as a proxy for "what the world thinks," but there is a massive gap between the digital and physical worlds. This is known as selection bias. For example, while platforms like Twitter or Weibo are popular for research, their user bases are disproportionately younger and tech-savvy compared to the general population. If a researcher uses these platforms to study public health or transportation without correction, their results will be heavily skewed toward the opinions and behaviors of specific cohorts, such as those born in the 1990s, while neglecting older demographics.
As shown in Figure 1, the "Virtual" distribution (yellow) peaks significantly in the 24-33 age bracket, whereas the "Physical" world (purple) has a much higher representation of individuals over 45. Existing solutions often involve complex regression models tailored to a single task. This research seeks a more universal solution: a way to transform a biased dataset into a representative one that can be used for any downstream analysis.
Methodology: Target Distribution based Resampling
The proposed Target Distribution based Resampling algorithm moves beyond simple weighting. Instead, it uses a resampling approach inspired by Markov Chain Monte Carlo (MCMC) methods to manage "status transitions" between data points.
Key Components of the Model:
- State Mapping: Different feature intervals (e.g., age groups) are treated as "states" in a Markov Chain.
- Transition-Acceptance Matrix: The core innovation is building a matrix that controls the resampling process. By leveraging the stationary distribution theory of Markov Chains, the algorithm calculates the probability of "accepting" a new data point into the final set.
- Stochastic Stability: The algorithm ensures that as more samples are collected, the distribution of the new dataset converges to the target distribution (the real-world ground truth).
- Balancing Logic: The system automatically handles both under-sampling (dropping samples from over-represented groups) and over-sampling (retaining and potentially duplicating samples from under-represented groups like the elderly).
Experimental Evaluation and Results
The researchers validated their approach using an online dataset of 15,940 Sina Weibo users and compared it against the China Population Census as the physical world ground truth. The study primarily focused on age group distributions.
- Rapid Convergence: The difference between the resampled data and the real-world census—measured by Jensen-Shannon Divergence—dropped sharply. After collecting just 500–1000 samples, the divergence fell below 0.02, indicating a very high match.
- Visual Alignment: Before the algorithm, the Weibo data showed a massive spike for users in their 20s. After resampling, the new dataset’s age curve closely aligned with the offline physical world distribution.
- Universal Utility: Unlike previous methods that fix bias during a specific prediction task, this method produces a new, cleaned dataset. This allows researchers to "fix" the data once and then use it for multiple independent studies.
Limitations and Future Work
While effective, the algorithm faces a "duplication trap." If a specific demographic is extremely rare in the online data (e.g., very elderly users on a niche app), the algorithm will over-sample them, leading to many duplicate records. This can reduce the variety of the data even if the distribution is statistically correct. Future work will focus on refining the acceptance matrix to minimize duplicates while maintaining distributional accuracy, particularly when the desired sample size is large relative to the original dataset.
Summary updated