TL;DR
This paper tackles the challenge of recommending relevant citations for academic papers, a task hindered by the vocabulary mismatch between a paper's content and the context where a citation is needed. The authors propose using a "translation model," originally designed for machine translation, to bridge this linguistic gap. Their experiments show this approach significantly outperforms existing methods.
What Problem Does This Paper Address?
When writing a paper, authors often need to cite previous work to support their claims, methods, or data. However, it's common to forget the exact source or struggle to find the most relevant supporting papers. This is where citation recommendation systems come in, aiming to suggest appropriate citations for "citation placeholders" (like "citation needed" in Wikipedia). The core problem is that the language used in the citation context (e.g., a sentence or two describing the need for a citation) is often very different from the language used in the actual paper being cited. This vocabulary mismatch makes it difficult for traditional information retrieval methods, which rely on keyword matching, to perform well.
Methods
The paper's central innovation is the application of a translation model to citation recommendation. This model, inspired by machine translation, treats the citation context and the paper's content as two different "languages." The goal is to learn a probabilistic mapping between words in the citation context and words in the paper's content.
Specifically, they estimate the translation probability $P(w_c|w_d)$, representing the likelihood of a word $w_c$ in the citation context appearing given a word $w_d$ from the document's content. This is typically done using an Expectation-Maximization (EM) algorithm, but for efficiency, they use a simplified version based on word co-occurrence counts.
To address the issue of low self-translation probabilities (i.e., a word translating to itself), they incorporate a self-translation boosting mechanism. This boosts the probability of a word translating to itself, improving performance.
For ranking documents, they use a query likelihood model, but instead of directly using the citation context as the query, they incorporate the translation model to bridge the vocabulary gap. The final score for a document $d$ given a citation context $c$ is calculated using a smoothed version of the query likelihood, where the probability of a word in the context being generated by the document is influenced by the translation model.
Datasets
The authors used a dataset consisting of 5,183 papers primarily from the information retrieval and text mining fields, published between 1988 and 2010. For each paper, they extracted citation placeholders and their surrounding three sentences as citation contexts. The collection of all papers formed the document collection, and a randomly selected set of 200 citation contexts were used as queries for evaluation.
Key Insights and Ideas
- Vocabulary Gap is Key: The fundamental challenge in citation recommendation is the significant vocabulary difference between the context needing a citation and the content of the cited paper. Traditional methods struggle with this.
- Translation Model as a Bridge: The translation model is a powerful tool for bridging this vocabulary gap, effectively acting as a "translator" between the two linguistic domains.
- Abstract vs. Full Text: The translation model performs better on paper abstracts than on full texts. This is because abstracts are concise and capture the core ideas, making them less noisy for translation. Full texts, while comprehensive, can introduce too much noise that hinders the translation process.
- Self-Translation Boosting is Beneficial: Explicitly boosting the probability of words translating to themselves improves the performance of the translation model.
- Parameter Sensitivity: While the model's performance is generally robust to the choice of parameters like the number of translated words, careful tuning of parameters like the self-translation boosting factor ($\beta$) is important for optimal results.
Key Results
The paper reports significant improvements using their translation model approach.
- Overall Performance: The translation models (TM) consistently outperformed both the query likelihood language model (LM) and the context-aware relevance model (CRM) across various configurations.
- TM vs. LM/CRM: The best translation model variant (TMsa, with self-boosting on abstract) achieved a Mean Average Precision (MAP) of 0.571, significantly higher than the best LM variant (LMf, full text) at 0.211 and the CRM at 0.238.
- Abstract vs. Full Text: Translation models performed better when using abstracts (TMsa: 0.571) compared to full text (TMsf: 0.535). Conversely, language models performed better on full text (LMf: 0.211) than abstracts (LMa: 0.122).
- Self-Boosting Advantage: The self-boosting variants (TMsa, TMsf) generally outperformed their non-boosting counterparts (TMa, TMf), indicating the value of this technique.
Limitations and Future Work
- Parameter Tuning: While the paper explores parameter tuning, the optimal values for parameters like $\alpha$ (collection model weight) and $\beta$ (self-translation boosting) are determined empirically. The paper notes that $\alpha$ can be set to a very small value, suggesting the collection model smoothing is almost useless in this scenario.
- Computational Cost: The paper acknowledges that translation models can be computationally expensive, both in storing the translation probability matrix and in online processing. They suggest a heuristic of storing only the top K translated words for each word to reduce complexity.
- Future Work: The authors suggest extending their work by incorporating additional features, such as other citations within the paper and the paper's global content, to further enhance citation recommendation. They also mention that their translation model can potentially be combined with existing citation-based methods.
Summary updated