A Deeper Investigation of the Importance of Wikipedia Links to the Success of Search Engines

Nicholas Vincent, Brent HechtarXiv 2020 · arXiv

TL;DR

Major search engines like Google, Bing, and DuckDuckGo are deeply dependent on Wikipedia, with its links appearing in up to 84% of common search results. By using a novel spatial analysis approach, researchers found that Wikipedia content frequently occupies prime "above-the-fold" real estate, particularly through Knowledge Panels, highlighting a massive reliance of multi-billion dollar technologies on volunteer-created "data labor."

What Problem Does This Paper Address?

For years, search engine research treated results as a simple list of "ten blue links." However, modern Search Engine Results Pages (SERPs) are far more complex, featuring Knowledge Panels, social media carousels, and featured snippets. While previous studies hinted at Wikipedia's importance, they were often limited to Google desktop results and failed to account for the spatial layout of modern search.

The authors set out to quantify exactly how much search engines rely on Wikipedia across different platforms (desktop vs. mobile) and providers (Google, Bing, DuckDuckGo). They also wanted to move beyond simple rankings to understand spatial prominence: how much of the user's immediate field of view is actually occupied by Wikipedia content? This is a critical question for understanding the "data as labor" ecosystem, where volunteer work fuels highly profitable proprietary technologies.

Methods

The researchers moved away from rank-based analysis to a spatial analysis approach. They developed a tool extending the se-scraper library, using Puppeteer to automate headless browsers that simulate both desktop (Chrome on Windows 10) and mobile (Safari on iPhone) environments.

Key technical innovations include:

  • Coordinate Tracking: Instead of just recording a link's rank, the software captures the exact (x, y) pixel coordinates of every clickable hyperlink.
  • Spatial Metrics: They defined new metrics like "above-the-fold incidence" (visible without scrolling) and "right-hand incidence" (a proxy for Knowledge Panel presence).
  • Visual Validation: To ensure the automated scraper wasn't missing structural changes in opaque search algorithms, they manually validated the coordinate data against screenshots for a subset of queries.

Datasets

The study utilized 432 unique queries across three categories:

  1. Common Queries: The top 100 queries by volume from Ahrefs (e.g., "Facebook", "Amazon").
  2. Trending Queries: 282 queries from Google’s 2018 top trending list (e.g., "World Cup", "Alexandria Ocasio-Cortez").
  3. Medical Queries: 50 important medical queries previously made public by Bing (e.g., "acid reflux", "common cold symptoms").

Key Insights and Ideas

  • The Knowledge Panel Powerhouse: Wikipedia is the primary engine for the "Knowledge Panel"—the rich information box usually found on the right side of desktop search results. As seen in Figure 1, for a query like "Instagram," Wikipedia provides the essential summary that users see immediately.
Figure 1. Shows a Google SERP for the common query “Instagram” with special components, including a right column “Knowledge Panel” with a Wikipedia link.
  • The Mobile Prominence Gap: Wikipedia is significantly less prominent "above the fold" on mobile devices compared to desktop. This is likely because mobile screens lack the right-hand column for Knowledge Panels, forcing that content further down the page.
  • Navigational Exposure: Even when users have "navigational intent" (e.g., searching for "Facebook" just to log in), they are still exposed to Wikipedia content. This suggests Wikipedia serves as a ubiquitous "fact-checking" layer for almost all web activity.
  • Medical Outliers: While Wikipedia is ubiquitous for common and trending topics, search engines (especially Google and Bing) appear to deliberately favor specialized medical sources over Wikipedia for health-related queries. DuckDuckGo was the exception, showing Wikipedia links for 54% of medical queries.

Key Results

The quantitative data confirms Wikipedia's massive footprint:

  • Full-Page Incidence: Wikipedia appeared in 80-84% of common queries and 67-72% of trending queries across all three search engines.
  • Desktop Prominence: On desktop, Wikipedia links were "above the fold" for roughly 80% of common queries.
  • Mobile Drop-off: For common queries on Google, Wikipedia's above-the-fold incidence dropped from 80% on desktop to just 2% on mobile.
Figure 2: Full-page incidence rates for Wikipedia links. Left plot shows desktop results and right plot shows mobile results. Results are faceted by query category (along the x-axis) and search engine (color).

As shown in Figure 2, while the overall presence of Wikipedia is high across engines, the specific query category (common vs. medical) is the strongest predictor of whether a Wikipedia link will appear.

Limitations and Future Work

The authors acknowledge that their study is a "snapshot" of a rapidly changing landscape. Limitations include:

  • Geographic Bias: The study focused exclusively on U.S.-based, English-language search results.
  • Query Sensitivity: Incidence rates are highly dependent on the specific query set chosen.
  • The "AI Assistant" Factor: The researchers noted that features like "Siri Suggestions" often bypass the search results page entirely, sending users directly to Wikipedia. Future work should investigate how these "zero-click" interventions impact the ecosystem.

Ultimately, the paper serves as a reminder that the "magic" of modern search is built on the back of a massive, volunteer-run encyclopedia. As search engines become more profitable, the question of how that value is shared with the creators of the underlying data becomes increasingly urgent.

Summary updated

Abstract

A growing body of work has highlighted the important role that Wikipedia's volunteer-created content plays in helping search engines achieve their core goal of addressing the information needs of millions of people. In this paper, we report the results of an investigation into the incidence of Wikipedia links in search engine results pages (SERPs). Our results extend prior work by considering three U.S. search engines, simulating both mobile and desktop devices, and using a spatial analysis approach designed to study modern SERPs that are no longer just "ten blue links". We find that Wikipedia links are extremely common in important search contexts, appearing in 67-84% of all SERPs for common and trending queries, but less often for medical queries. Furthermore, we observe that Wikipedia links often appear in "Knowledge Panel" SERP elements and are in positions visible to users without scrolling, although Wikipedia appears less in prominent positions on mobile devices. Our findings reinforce the complementary notions that (1) Wikipedia content and research has major impact outside of the Wikipedia domain and (2) powerful technologies like search engines are highly reliant on free content created by volunteers.
BibTeX
@misc{vincent2020deeper,
  author = {Nicholas Vincent and Brent Hecht},
  title = {{A Deeper Investigation of the Importance of Wikipedia Links to the Success of Search Engines}},
  year = {2020},
  eprint = {2004.10265},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.10265},
}
Figure