Introduction
Wikipedia serves as a primary gateway to scientific knowledge for millions. However, maintaining the integrity of its "References" section remains a significant challenge. While the text of articles evolves rapidly, the citations often lag behind, leading to pages that are informative but lack robust, up-to-date sourcing. This paper introduces WikiRef, an automated system designed to bolster Wikipedia's reliability by recommending high-quality scientific references through the analysis of internal wikilinks.
The Reference Gap
The motivation for WikiRef stems from a fundamental imbalance in how Wikipedia is edited. While users frequently update article text and add internal wikilinks to navigate between topics, they rarely add or update citations. Research indicates that while approximately 32% of edits involve wikilinks, a mere 1% focus on references. This "reference gap" suggests that adding citations is perceived as a significantly more difficult task for the community.
To bridge this gap, the authors propose leveraging the existing network of wikilinks. Since wikilinks are updated more frequently and reflect topical connections, they can serve as a "fresher" signal for identifying relevant source material.
The WikiRef Framework
WikiRef operates through a two-step pipeline:
1. Wikilink Classification
Not every link in an article points to a page that can provide useful scientific references. For instance, a page on "Machine Learning" might link to "Linear Algebra" (a rich source of foundational papers) but also to "Silicon Valley" (unlikely to contain relevant academic citations). WikiRef employs a Random Forest classifier to distinguish between these "potential sources" and irrelevant links. Features used for this classification include Tf-idf similarity of article summaries, link overlap (comparing inlinks and outlinks), and semantic vector representations generated via bi-directional LSTMs.
2. Reference Recommendation and Ranking
Once a relevant source page is identified, the system must determine which specific references from that page should be "inherited" by the target article. This is framed as a learning-to-rank problem using SVMRank. The model prioritizes references based on:
- Citation Context: The similarity between the sentence containing the reference in the source article and the sentence containing the wikilink in the target article.
- Title Similarity: Matching the reference title against the context of the target page.
Evaluation and Results
The researchers tested WikiRef on extensive datasets from Computer Science (CS) and Physics (PH), using existing Wikipedia references as the ground truth.
WikiRef consistently outperformed traditional baselines. In the Computer Science domain, the system achieved a Precision@1 of 0.44, marking a 10-15% improvement over previous methods. Human evaluation further validated these results; in a study with 10 experts, WikiRef's suggestions received a relevance score of 0.63, meaning that roughly 60% of the "new" references suggested by the system were deemed highly appropriate by subject matter experts.
Conclusion
WikiRef demonstrates that Wikipedia's internal link structure is a powerful, underutilized resource for automated curation. By treating wikilinks as proxies for topical relevance, the system can effectively "crowdsource" references from well-sourced articles to those in need. Future work aims to implement WikiRef as a real-time tool for Wikipedia editors and expand its application to broader fields like medicine and biology.
Summary updated