Retrieving Relevant Conversations for Q&A on Twitter

Jose Miguel Herrera, Denis Parra, Barbara Poblete

Introduction

Twitter serves as a massive, real-time repository of human knowledge, yet its "firehose" nature often results in valuable information being buried within minutes. While dedicated Community Question Answering (CQA) platforms like Stack Overflow or Yahoo! Answers provide mechanisms to highlight "best answers," Twitter lacks an explicit "mark as resolved" feature. This research proposes a method to "recycle" historical knowledge by retrieving and ranking past conversation threads to answer new user queries, effectively treating Twitter as a searchable Q&A repository.

Methodology

The proposed system bridges the gap between social networking and structured Q&A through a three-step process:

  1. Retrieval: Using the Twitter Search API, the system identifies past questions ($q_j$) that exhibit high cosine similarity to a new incoming query ($q^*$).
  2. Thread Reconstruction: Since the Twitter API does not natively provide full conversation trees, specialized scraping tools are employed to reconstruct the hierarchical thread structure ($th_j$).
  3. Ranking via Positive Reinforcement Feedback (PRF): The core innovation lies in using the original asker's feedback as a proxy for answer quality. Instead of relying on general popularity metrics like retweets, the system prioritizes threads where the asker signaled satisfaction through:
  4. Favorites: Marking a specific reply as a favorite.
  5. Sentiment Analysis: Using the Natural Language Toolkit (NLTK) to analyze the asker’s response. High positive polarity in replies like "thanks dude!" or "fab reads!" indicates a successful resolution.
Twitter UI elements used as feedback signals, including favorites and direct replies, to identify high-quality answers.

Experimental Setup and Datasets

The researchers conducted a focalized crawl between March and April 2015, resulting in a historical repository of 152,721 conversation threads. From this, 721 "new" questions were used for testing. The dataset was filtered to target factual or recommendation-based queries (e.g., "good restaurants in London?") while excluding time-sensitive or non-informational posts.

Key Insights and Results

The study reveals that approximately 10% of the Twitter stream consists of Q&A interactions, confirming its status as a "hidden" CQA platform. The primary findings include:

  • The Asker as the Ultimate Judge: Feedback from the person who initiated the question is a significantly more reliable indicator of thread value than standard social features.
  • Sentiment as Metadata: In the absence of formal metadata, sentiment analysis of closing remarks serves as a powerful substitute for a "Correct Answer" button.
  • Retrieval Patterns: The authors identified three recurrent cases of successful retrieval, ranging from threads with explicit thanks (Case 1) to single-tweet threads where a URL provided the answer (Case 2).
Examples of retrieved threads where asker feedback (e.g., 'Ooh yes!!', 'ty') indicates successful resolution of the information need.

Conclusion and Future Work

Preliminary results demonstrate the feasibility of using historical Twitter conversations to satisfy new information needs. Future work will focus on automating the ranking process through machine learning and incorporating temporal awareness to ensure the system does not recommend outdated information. The ultimate goal is to create a real-time knowledge base that preserves human understanding as it flows through social networks.

Summary updated

BibTeX
@inproceedings{herrera2015retrieving,
  author = {Jose Miguel Herrera and Denis Parra and Barbara Poblete},
  title = {{Retrieving Relevant Conversations for Q\&A on Twitter}},
  booktitle = {SPS @ SIGIR},
  year = {2015},
}
Figure