On predicting religion labels in microblogging networks

Minh Thap Nguyen, Ee-Peng LimSIGIR 2014 · DOI

TL;DR

This paper tackles the challenge of predicting users' religious affiliations from their microblogging data, a task previously unaddressed. By combining content and network structure features, including novel "representative user" metrics, the authors achieve high accuracy in classifying Christian and Muslim users on a large Twitter dataset, paving the way for more nuanced user profiling.

What Problem Does This Paper Address?

Traditionally, understanding the religious demographics of a population relies on costly and time-consuming census studies. However, online social media platforms offer a rich, albeit implicit, source of information about users' personal attributes. This paper addresses the gap in predicting users' religious labels from their microblogging activity, a task that is difficult due to the sparsity of explicitly declared religious information in user profiles. The authors aim to develop an accurate and scalable method for inferring religious affiliation, which can be valuable for various social applications.

Methods

The core of this paper lies in its feature engineering and classification approach. The authors formulate religion label prediction as a classification task. They extract a comprehensive set of features, categorized into:

  • Content Features: Derived from the textual content of users' tweets. This includes TF-IDF representations of words in a user's tweet document.
  • Structural Features: Based on the user's connectivity within the social network (follow links).
  • Aggregated Features: Summaries of content or structural features, often derived from the user's neighborhood.

A key innovation is the introduction of "representative user" measures (label-degree, label-indegree, and representative indegree ratio - RInDeg). These metrics go beyond standard network centrality measures by considering a user's importance within a specific religious group. RInDeg, in particular, measures the dominance of a particular religion among a user's followers, indicating their influence within that community. These representative measures are then used to derive social features, leveraging the idea that users might follow influential figures within their religious group.

The classification itself is performed using Support Vector Machines (SVMs), a standard and powerful machine learning algorithm.

Datasets

The primary dataset used is a newly constructed Twitter dataset comprising over 111,767 users located in Singapore. This dataset includes users' profiles, tweets, and follow links.

Crucially, the authors manually annotated religion labels for 1029 users from this dataset, identifying 581 Christians and 448 Muslims. This manual annotation process involved searching for religion-specific keywords in user biographies and then manually verifying the declared religion. This dataset is a significant contribution as it addresses the lack of publicly available datasets with religion labels.

Key Insights and Ideas

  • Implicit Religion: Users' religious beliefs are often implicitly embedded in their online behavior and social connections, rather than explicitly stated.
  • The Power of Neighborhood: Social features derived from a user's network (especially influential users within their religious group) significantly improve prediction accuracy compared to using self-content alone.
  • Representative Users Matter: Standard network centrality measures are insufficient. Novel metrics like RInDeg, which capture a user's influence within a specific religious community, are crucial for effective feature engineering.
  • Homophily in Action: The assumption that users tend to follow others with similar attributes (homophily) holds true for religious affiliation, making network structure a strong predictor.
  • Singapore as a Case Study: The Singapore dataset, with its diverse religious landscape, provides a unique context for studying religion prediction, though the observed distribution of online religious self-declaration differs from census data.

Key Results

The paper reports impressive results using SVM classifiers:

  • High Accuracy: The best F1-scores achieved for Christian and Muslim classes are 0.909 and 0.880, respectively, when using a comprehensive set of features including social and aggregated ones.
  • Feature Effectiveness: Social content features, particularly those derived using the RInDeg metric, significantly outperform self-content features. Combining content, structural, and aggregated features yields the best performance.
  • Manual Validation: The authors manually judged the top-scored users predicted by their model on the larger unlabeled dataset, finding a 100% precision rate for their manual checks, indicating the model's reliability.

Limitations and Future Work

  • Limited Religions: The study primarily focuses on predicting Christian and Muslim labels due to the sparsity of self-declared labels for other religions (Buddhism, Taoism, Hinduism) in their dataset.
  • Dataset Specificity: The findings are based on a dataset of Twitter users in Singapore. While the methods are generalizable, the specific feature distributions and keyword relevance might vary in other cultural contexts.
  • Manual Annotation Effort: The manual annotation of religion labels is labor-intensive, which is a bottleneck for creating larger, more diverse labeled datasets.

Future work could involve:

  • Expanding the study to include more religious groups.
  • Investigating the prediction of other latent user attributes.
  • Exploring different social media platforms and languages.
  • Developing more automated methods for religion label annotation.

Summary updated

BibTeX
@inproceedings{nguyen2014predicting,
  author = {Minh Thap Nguyen and Ee-Peng Lim},
  title = {{On predicting religion labels in microblogging networks}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2014},
  doi = {10.1145/2600428.2609547},
}
Figure