Expert Finding for Community-Based Question Answering via Ranking Metric Network Learning

Zhou Zhao, Qifan Yang, Deng Cai, Xiaofei He, Yueting Zhuang

Introduction

Expert finding is a critical task for Community-Based Question Answering (CQA) platforms like Quora and Stack Overflow. Effectively routing new questions to qualified experts and identifying high-quality answers improves user engagement and platform utility. However, traditional expert finding methods face three primary challenges:

  1. Data Sparsity: Most questions receive few answers, making it difficult to build comprehensive user profiles based solely on answering history.
  2. Semantic Representation: "Bag-of-words" models often fail to capture the sequential intent and context of questions.
  3. Vote Bias: Absolute vote counts are frequently influenced by question popularity rather than answer quality.

To address these issues, this paper proposes Ranking Metric Network Learning (RMNL), a framework that integrates deep learning with social network analysis to match experts with relevant questions.

The RMNL Framework

The RMNL framework approaches expert finding as a graph-based embedding problem, combining semantic understanding with structural network information.

Overview of the RMNL framework, including the heterogeneous CQA network, random-walk sampling, and LSTM-based learning architecture.

Heterogeneous CQA Network

The model constructs a graph incorporating multiple node types and relations. Beyond standard question-answer pairs, it includes relative quality ranks (capturing that User A’s answer was preferred over User B’s for a specific question) and social relations. By integrating external social signals, such as "following" relationships from Twitter, the model can bridge gaps in activity data.

Q-LSTM and Random-Walk Learning

To capture semantic nuances, the framework employs Q-LSTM (Question-Long Short-Term Memory). Unlike traditional models, LSTMs process questions sequentially, preserving the semantic meaning derived from word order.

To learn from the network structure, RMNL utilizes Random-Walk Learning. By performing random walks across the heterogeneous graph, the model leverages social homophily—the tendency for connected individuals to share expertise—to infer expertise even in the absence of direct answering history.

Ranking Metric Loss

The model is optimized using a triplet loss function. This "Ranking Metric Loss" aims to minimize the distance between a question's embedding and its high-quality experts while maximizing the distance to lower-quality users.

Experimental Setup

The researchers evaluated RMNL using a large-scale real-world dataset from Quora, comprising 444,138 questions, 95,915 users, and 887,771 answers. Social relationships were extracted by mapping Quora users to their corresponding Twitter accounts, providing a robust social layer for the network.

Experimental Results

RMNL was compared against several state-of-the-art baselines, including AuthorityRank and DeepWalk. The framework consistently outperformed these methods across multiple metrics:

  • nDCG (Ranking Quality): RMNL achieved 0.741, surpassing the best baseline (GRMC-AGM at 0.734).
  • Precision@1: RMNL reached 0.540, a significant improvement over the 0.512 achieved by competing methods.
  • Accuracy: RMNL attained 0.641, compared to the 0.55–0.61 range of other models.
Performance of RMNL across different lengths of sampled paths for nDCG, Precision@1, and Accuracy.

The results indicate that supervised information regarding relative answer quality is significantly more valuable for expert finding than unsupervised network structure alone. Furthermore, the length of sampled paths in the random walk process impacts performance, with optimal results typically found at moderate path lengths (e.g., length 5).

Conclusion and Future Work

RMNL demonstrates the power of combining sequential semantic modeling with heterogeneous network embeddings for expert finding. By leveraging social ties and relative ranking, it effectively mitigates data sparsity and vote bias.

Future research may address the model's dependency on cross-platform social data and the computational overhead of training LSTMs on massive graphs. Additionally, incorporating temporal dynamics to model evolving user expertise presents a promising direction for further development.

Summary updated

Abstract

Expert finding for question answering is a challenging problem in Community-based Question Answering (CQA) site, arising in many applications such as question routing and the identification of best answers. In order to provide high-quality experts, many existing approaches learn the user model mainly from their past question-answering activities in CQA sites, which suffer from the sparsity problem of CQA data. In this paper, we consider the problem of expert finding from the viewpoint of learning ranking metric embedding. We propose a novel ranking metric network learning framework for expert finding by exploiting both users' relative quality rank to given questions and their social relations. We then develop a random-walk based learning method with recurrent neural networks for ranking metric network embedding. The extensive experiments on a large-scale dataset from a real world CQA site show that our method achieves better performance than other state-of-the-art solutions to the problem.
BibTeX
@inproceedings{zhao2016expert,
  author = {Zhou Zhao and Qifan Yang and Deng Cai and Xiaofei He and Yueting Zhuang},
  title = {{Expert Finding for Community-Based Question Answering via Ranking Metric Network Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure