In digital collaboration platforms, from academic peer review systems to Q&A communities like Quora, finding the right expert is only half the challenge. The real bottleneck is often the "willingness" of the expert to accept an invitation. This paper introduces a novel framework designed to match tasks with experts by simultaneously evaluating competence and predicting the probability of acceptance.
The Challenge of Expert Matching
Traditional expert finding systems primarily focus on competence: identifying individuals with the requisite knowledge for a specific task. However, in real-world scenarios, high-competence experts are often over-solicited, leading to a high rate of declined invitations. This "willingness bottleneck" results in delayed tasks and inefficient resource allocation.
The authors identify three core challenges:
- The Expertise-Willingness Trade-off: Balancing a "perfect" match who is likely to decline against a "good" match who is more likely to accept.
- Data Sparsity: The lack of labeled data explaining why experts decline invitations (e.g., workload, topic disinterest).
- Real-world Validation: The difficulty of evaluating these models outside of controlled environments.
The Weakly Supervised Factor Graph (WeakFG) Model
To address these challenges, the researchers developed the Weakly Supervised Factor Graph (WeakFG) model. This approach decomposes the matching problem into two integrated components: expertise matching and acceptance prediction.
1. Expertise Matching
The model moves beyond simple keyword matching by employing advanced embedding techniques to capture semantic context:
- Word Mover’s Distance (WMD): Measures the semantic distance between a task description and an expert’s historical contributions.
- Document Vector (D2V): Learns dense numeric representations of entire documents to facilitate high-level semantic comparison.
2. Acceptance Prediction via Social Identity Theory
The core innovation of WeakFG is its use of Social Identity Theory to predict behavior. The model assumes that experts who share a social identity (e.g., similar professional roles, seniority, or organizational affiliations) exhibit correlated behavioral patterns.
By utilizing Weak Supervision, the model propagates known "decline" labels across a social graph. If an expert with a specific social profile declines a task, the model increases the predicted decline probability for other experts with similar profiles. The factor graph framework then combines these social correlations with local features—such as an expert's h-index or recent activity—to produce a final matching score.
Experimental Evaluation
The WeakFG model was rigorously tested across three diverse datasets:
- QA-Expert: Data from a large-scale Q&A challenge involving ~44,000 invitations.
- Paper-Reviewer: Bidding and assignment data from a major Computer Science conference.
- Online Evaluation: A real-world deployment via a Chrome extension used by 50 journal editors to find reviewers for 540 papers.
Key Results and Performance Analysis
The experimental results demonstrate that WeakFG significantly outperforms traditional ranking algorithms, such as RankSVM and LDA, by 1.5% to 10.7% in Mean Average Precision (MAP).
Key findings include:
- Impact of Social Correlations: Ablation studies (removing social identity components) resulted in a 4% performance drop on the QA dataset, confirming that social context is a critical predictor of professional behavior.
- Robustness to Data Sparsity: The model maintained high performance even with limited training data, showcasing the effectiveness of the weak supervision approach.
- Real-World Utility: In the online journal system, the model achieved a Precision@3 of 66.2%, matching human-level relevance while effectively identifying "serial decliners."
Conclusion and Future Directions
The WeakFG model represents a significant step forward in expertise matching by acknowledging and modeling the human element of willingness. By leveraging social identity as a proxy for behavior, the system overcomes data sparsity issues common in professional networks.
Future work may explore integrating temporal factors, such as seasonal workload fluctuations, and expanding the types of social correlations used to further refine prediction accuracy. While the model's computational complexity is higher than simpler alternatives, its performance gains and real-world applicability make it a valuable tool for modern collaborative platforms.
Summary updated