Booming Up the Long Tails: Discovering Potentially Contributive Users in Community-Based Question Answering Services

Juyup Sung, Jae-Gil Lee, Uichin Lee

Abstract

Community-based question answering (CQA) services such as Yahoo! Answers have been widely used by Internet users to get the answers for their inquiries. The CQA services totally rely on the contributions by the users. However, it is known that newcomers are prone to lose their interests and leave the communities. Thus, finding expert users in an early phase when they are still active is essential to improve the chances of motivating them to contribute to the communities further. In this paper, we propose a novel approach to discovering "potentially" contributive users from recently-joined users in CQA services. The likelihood of becoming a contributive user is defined by the user's expertise as well as availability, which we call the answer affordance. The main technical difficulty lies in the fact that such recently-joined users do not have abundant information accumulated for many years. We utilize a user's productive vocabulary to mitigate the lack of available information since the vocabulary is the most fundamental element that reveals his/her knowledge. Extensive experiments were conducted with a huge data set of Naver Knowledge-In (KiN), which is the dominating CQA service in Korea. We demonstrate that the top rankers selected by the answer affordance outperformed those by KiN in terms of the amount of answering activity.

Introduction

Community-based Question Answering (CQA) platforms like Yahoo! Answers or Naver Knowledge-In (KiN) are powered by a "heavy user" elite—a small group of experts who provide the bulk of high-quality answers. However, newcomers ("light users") are prone to churn; if they don't feel useful or engaged within their first week, they often leave the platform. This creates a classic cold-start problem: identifying a future expert when they have only written a handful of answers. Traditional ranking systems rely on long-term statistics, such as "best answer" ratios or total points, which newcomers have not yet accumulated. To prevent these potential experts from leaving, platforms need a way to spot talent early so they can route them relevant questions and keep them engaged. By "booming up" the long tail of light users, platforms can diversify their expert base and reduce the burden on a few overworked heavy users.

Proposed Framework: Answer Affordance

The authors propose a framework called Answer Affordance, which combines two metrics: Estimated Expertise and Availability.

The Vocabulary Bridge

The core innovation of this approach is shifting the unit of analysis from the user profile to the word. Specialized knowledge is inextricably linked to specialized language; even in a single sentence, an expert reveals themselves through their choice of terminology.

The Vocabulary Bridge: Propagating expertise from heavy users to light users via shared vocabulary.

The process involves four key steps:

  1. Expertise Calculation: Calculate the expertise of "heavy users" using established metrics like category concentration and selection counts.
  2. Word Level Assignment: Assign a "Word Level" to specific terms based on the expertise of the heavy users who use them.
  3. Expertise Propagation: Propagate these word levels to light users based on their usage.
  4. Estimation: A newcomer’s expertise is estimated by the weighted average of the "Word Levels" found in their limited vocabulary.

Availability

Availability measures how likely a user is to answer now. It uses a sigmoid-smoothed recency function, giving higher weight to users who have been active in the very recent past. By combining expertise and availability, the system ranks users who are both highly knowledgeable and currently active.

Availability Metric: Comparing the recent activity levels of two users over a timeline.

Experimental Evaluation

The study utilized a massive longitudinal dataset from Naver Knowledge-In (KiN), covering 10 years of data (2002–2012). The evaluation focused on two distinct categories: Computers (factual expertise) and Travel (subjective/opinion-based expertise).

Key Results

  • Expertise Prediction: In the Computers category, the EstimatedExpertise model achieved a P@20 of 0.833, outperforming KiN’s own internal ranking metrics for light users.
  • Activity Retention: Users identified as high-potential were significantly more active in the future. In the Computers category, the average user availability for the proposed method was 0.650, compared to 0.278 for users selected by KiN’s standard ranking.
  • Robustness: The system remained highly effective even when users had as few as five answers on record, maintaining a correlation coefficient of approximately 0.7 with the ground truth.

Conclusion

The proposed method is significantly more effective in factual domains like Computers, where vocabulary is highly specialized. In subjective domains like Travel, word choice is less distinct, making it harder to distinguish experts. Future work includes exploring more sophisticated ways to calculate initial expertise and refining word-level propagation to better handle the nuances of subjective categories.

Summary updated

Abstract

Community-based question answering (CQA) services such as Yahoo! Answers have been widely used by Internet users to get the answers for their inquiries. The CQA services totally rely on the contributions by the users. However, it is known that newcomers are prone to lose their interests and leave the communities. Thus, finding expert users in an early phase when they are still active is essential to improve the chances of motivating them to contribute to the communities further. In this paper, we propose a novel approach to discovering "potentially" contributive users from recently-joined users in CQA services. The likelihood of becoming a contributive user is defined by the user's expertise as well as availability, which we call the answer affordance. The main technical difficulty lies in the fact that such recently-joined users do not have abundant information accumulated for many years. We utilize a user's productive vocabulary to mitigate the lack of available information since the vocabulary is the most fundamental element that reveals his/her knowledge. Extensive experiments were conducted with a huge data set of Naver Knowledge-In (KiN), which is the dominating CQA service in Korea. We demonstrate that the top rankers selected by the answer affordance outperformed those by KiN in terms of the amount of answering activity.
BibTeX
@inproceedings{sung2021booming,
  author = {Juyup Sung and Jae-Gil Lee and Uichin Lee},
  title = {{Booming Up the Long Tails: Discovering Potentially Contributive Users in Community-Based Question Answering Services}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v7i1.14387},
}
Figure