Abstract
Wikipedia is a primary information gateway, yet the extent to which readers interact with its underlying citations remains largely unquantified. This study presents the first large-scale analysis of reader engagement with citations on English Wikipedia. By instrumenting client-side logging for one month, we tracked interactions such as footnote clicks and hovers. Our findings reveal a low overall engagement rate: only 0.29% of page views result in a citation click. Engagement is significantly higher on shorter, lower-quality articles, suggesting citations serve as a critical resource when Wikipedia's own content is insufficient. We also observe a preference for open-access sources and life-event references. These results deepen our understanding of Wikipedia's role in the global information ecosystem.
Introduction
Wikipedia’s core philosophy is built on verifiability: every claim should be backed by a reliable source. While the community has extensive data on how editors curate these citations, there has been a significant knowledge gap regarding how the "silent majority"—the billions of monthly readers—actually use them. This research addresses whether Wikipedia serves as a "final destination" or a "bridge" to academic and primary resources. Understanding these interactions is crucial for measuring the public demand for secondary sources and improving the encyclopedia’s interface.
Methodology
To capture reader behavior without compromising privacy, the researchers utilized Wikimedia’s EventLogging tool for client-side instrumentation. This allowed for the tracking of specific micro-interactions that define how a user navigates from the text to a source.
Interaction Types
The study defined six specific interaction types to categorize user engagement with citations:
- pageLoad: The initial loading of an article.
- fnClick: Clicking a footnote number (e.g., [1]).
- fnHover: Hovering over a footnote to trigger a preview tooltip.
- refClick: Clicking an external link within the reference section.
- extClick: Clicking external links in sidebars or infoboxes.
- upClick: Returning to the main text from a footnote link.
Metrics and Classification
Two primary metrics were introduced: Global Click-Through Rate (gCTR), which measures micro-level engagement across the entire site, and Page-specific CTR (pCTR), a macro-average giving every page equal weight regardless of traffic volume. Articles were classified using the ORES (Objective Revision Evaluation Service) machine learning models to categorize content by topic and quality (from "Stub" to "Featured Article").
Dataset
The analysis utilized a dataset of 96 million citation-related events collected over two 4-week periods (October 2018 and April 2019). To isolate reader behavior, all logged-in editors and bots were excluded. Due to high traffic, pageLoad events were sampled at a 33% rate.
Key Results
The study yielded several surprising insights into reader behavior:
- Low Overall Engagement: Only 0.29% of page views involve a citation click. Furthermore, 93% of citation links are never clicked within a given month.
- The Quality Paradox: Counter-intuitively, higher-quality and longer articles (e.g., "Featured Articles") exhibit lower citation engagement. This suggests that comprehensive summaries satisfy reader needs, reducing the drive to verify facts externally.
- The Gateway Function: Engagement is highest on "Stubs" and "Start" class articles, where Wikipedia acts as a true gateway to more detailed external sources.
- Device Disparity: Desktop users engage with citations nearly four times as often as mobile users (0.56% vs 0.13%), likely due to the availability of "hover" tooltips on desktop.
- Open Access Preference: References pointing to open-access scientific papers see higher engagement than paywalled sources, highlighting the public value of the Open Access movement.
Discussion and Conclusion
The findings suggest that citations serve as a "safety net" for readers when Wikipedia's content is lacking. While the study provides a massive-scale view of engagement, it does not capture user intent or the perceived utility of the external sources. Future work could leverage these insights to develop recommendation engines, guiding editors to improve citations in "Stub" articles where reader curiosity and demand for external verification are highest.
Reference: Piccardi, T., Redi, M., Colavizza, G., & West, R. (2020). Quantifying Engagement with Citations on Wikipedia. In Proceedings of The Web Conference 2020 (WWW ’20).
Summary updated