Abstract
[This corrects the article DOI: 10.1371/journal.pone.0064601.]. This paper introduces a novel framework for improving archived question retrieval in Community Question Answering (cQA) platforms by bridging the "lexical gap." By identifying key phrases in a user's query and generating high-quality paraphrases through multi-engine "pivot translation," the system reformulates questions to better match existing high-quality answers, significantly outperforming traditional word-level or sentence-level retrieval methods.
The Lexical Gap in Community Question Answering
In Community Question Answering (cQA) portals like Yahoo! Answers or Quora, users often ask the same question using vastly different terminology—a phenomenon known as the lexical gap. For example, one user might ask about "folic acid for expectant mothers," while a perfectly good archived answer exists for "vitamin B9 for pregnant women."
Standard search engines often struggle with this because they rely heavily on exact keyword matching. Previous attempts to solve this used word-level synonyms (which lack context) or sentence-level paraphrasing (which is often too rigid or data-sparse). The authors tackle the challenge of finding a "middle ground": phrase-level paraphrasing that is flexible enough to capture meaning but structured enough to maintain grammatical sense.
Methodology: A Three-Stage Pipeline
The authors propose a three-stage pipeline for question reformulation designed to maximize both semantic accuracy and linguistic fluency.
1. Key-phrase Detection
Instead of treating all words equally, the system identifies "indispensable" phrases. It uses a combination of statistical weighting (based on BM25 and Language Modeling) and linguistic constraints (Part-of-Speech tagging and dependency parsing). They employ three specific mathematical thresholds—quadratic, arithmetic, and geometric means—to decide which chunks of a sentence are "key."
2. Paraphrase Extraction via Pivot Translation
This is the most novel part of the approach. To find synonyms for a phrase like "expectant mother," the system sends the phrase to three different online translation engines (Baidu, Youdao, and Dict.cn), translates it into Chinese, and then translates the results back into English. This "pivot" through another language uncovers natural variations in phrasing that static dictionaries like WordNet might miss.
3. Reformulation Selection (Viterbi Decoding)
Once they have a list of paraphrased phrases, they don't just randomly swap them. They use the Viterbi algorithm—a dynamic programming tool—to find the most fluent and probable combination of these phrases, ensuring the final reformulated question sounds like natural English.
Datasets and Evaluation
The researchers evaluated their model using a real-world dataset from Yahoo! Answers, consisting of:
- 1,123,134 unique questions used as the searchable archive.
- A test set of 200 questions (140 for testing, 28 for development, and the rest filtered for quality).
- Human judgments from two native English speakers were used to evaluate the "correct meaning" and "correct grammar" of the generated paraphrases.
Key Insights and Ideas
- The "Goldilocks" Granularity: The paper argues that phrases are the optimal unit for paraphrasing. Words are too small and lose context (e.g., "cold" can mean a temperature or an illness), while sentences are too unique to find frequent matches in a database.
- Translation as a Paraphrase Tool: Using translation engines as a "black box" for paraphrasing is a clever hack. It leverages the massive, human-curated data already baked into commercial translation tools to solve a difficult NLP problem.
- Linguistic Rules Matter: The authors found that simply having high-weight keywords wasn't enough. Incorporating dependency parsing (understanding which words are subjects or objects) was crucial to ensure that the "key phrases" actually carried the core meaning of the question.
Key Results
The proposed model, referred to as VD-TLM (Viterbi Decoding-based Translation Language Model), showed substantial improvements over baseline methods:
- Retrieval Accuracy: The Mean Average Precision (MAP) jumped from 0.2889 (baseline) to 0.4188, a significant gain in finding relevant archived questions.
- Precision at 1 (p@1): The probability that the very first result returned was the correct one increased from 19.28% to 27.86%.
- Paraphrase Quality: Their method achieved a "Both Correct" (meaning and grammar) score of 81.66%, compared to only 46.17% for the baseline phrase-extraction method.
Limitations and Future Work
While effective, the authors noted a few hurdles. Sometimes the system struggled to define exactly where a phrase starts and ends, leading to awkward overlaps. Additionally, their method produced fewer total paraphrases (4,750) compared to some baselines (7,752), though their paraphrases were of much higher quality.
Future directions include combining their bilingual "pivot" approach with monolingual corpora (like Wikipedia) to further expand the variety of paraphrases. They also aim to explore joint estimation of paraphrase probability to further refine the ranking of reformulated questions.
Summary updated