How the structure of Wikipedia articles influences user navigation

Daniel Lamprecht, Kristina Lerman, Denis Helic, Markus StrohmaierNew Rev. Hypermedia Multim 2017 · DOI · OpenReview

TL;DR

This paper investigates how the structural organization of Wikipedia articles influences user navigation. By analyzing clickstream data and simulated "wikigames," the authors found that article structure, particularly the lead section and infobox, significantly guides user clicks, even more so than link popularity or textual similarity in many cases. This highlights the importance of well-structured content for effective information discovery.

What Problem Does This Paper Address?

Navigating large information networks like the World Wide Web or Wikipedia can be challenging. While search engines help, they aren't always effective for complex or ill-defined information needs. This leads to a reliance on navigation and exploration. The problem this paper addresses is understanding how the inherent structure of information within articles, specifically Wikipedia articles, influences the way users navigate through these networks. Specifically, it asks: to what extent does the way an article is organized (e.g., the placement of information in the lead section or infobox) affect where users click and how they find information?

Methods

The researchers employed a two-pronged approach:

  1. Wikipedia Clickstream Analysis: They analyzed a month's worth of all clicks within the English Wikipedia. This provided a broad view of "free-form" navigation, encompassing all user behaviors without specific goals.
  2. Wikigames Analysis: They utilized data from "wikigames," where users are tasked with navigating from a starting Wikipedia article to a target article using only internal links. This allowed for the study of "goal-directed" navigation with explicit objectives.

To model and compare the influence of different factors on link selection, they developed link selection models. These models assigned probabilities to links based on various influences and were then compared to actual user click data using Kullback-Leibler (KL) divergence to measure how well each model explained user behavior.

Datasets

  • Wikipedia Clickstream Data: A dataset of all clicks within the English Wikipedia for one month (February 2015). This dataset contains aggregated link click counts and does not reveal individual user sessions or navigation paths.
  • Wikispeedia Dataset: Log files from approximately 75,000 wikigames, involving around 475,000 clicks. This dataset provides detailed, step-by-step navigation paths for goal-directed tasks. The games were played on a subset of ~4600 Wikipedia articles chosen for educational purposes.

Key Insights and Ideas

  • Article Structure Reigns Supreme (Mostly): The most surprising finding is how strongly article structure, particularly the lead section and infobox, influences navigation. Users disproportionately click on links within these top sections, even though they might contain a smaller percentage of the total links.
  • Generality Guides the Zoom-Out: In the initial stages of goal-directed navigation (the "zoom-out" phase), users tend to click on links leading to more general articles, likely to find a "landmark" to orient themselves. This is well-explained by measures of article generality like indegree.
  • Textual Similarity Wins at the End: As users get closer to their target in goal-directed navigation (the "home-in" phase), textual similarity (TF-IDF) becomes a more dominant factor in link selection.
  • Middle Ground is Structure-Driven: For the majority of navigation steps (excluding the very first and last), article structure is the best predictor of user clicks, outperforming generality and textual similarity.
  • Wikigames Reflect Real-World Navigation: Despite being a game, the wikigame data showed remarkably similar navigational influences to the broader Wikipedia clickstream data, suggesting that these games offer valuable insights into real-world navigation.

Key Results

  • Lead and Infobox Importance: For the Wikipedia clickstream, models that heavily weighted the lead section (32% of clicks) or infobox (18% of clicks) showed the lowest KL divergence, significantly outperforming models based solely on link counts (lead: 17% of links, infobox: 4% of links).
  • Generality Models Underperform: Models based on article generality (indegree, view count, n-grams) consistently showed large differences (high KL divergence) compared to actual user clicks, indicating they are poor explanations for aggregate navigation.
  • Wikigame Phases: In wikigames, indegree best explained the initial steps, TF-IDF best explained the final steps, and article structure (lead/infobox) best explained the intermediate steps.
  • Article Structure Dominance: Across all steps in wikigames, article structure models explained the largest fraction of navigational decisions, particularly in the middle phases where they accounted for about 64% of best fits.

Limitations and Future Work

  • Wikigame Bias: The authors acknowledge that the game-like nature of wikigames and the fact that users knew they were being recorded could introduce biases. However, their step-by-step analysis showed that only the first and last clicks differed significantly from the clickstream, suggesting the middle steps are representative.
  • Mobile vs. Desktop: The current analysis focused on the desktop view of Wikipedia. Future work could explore navigation on mobile devices, where article layouts (e.g., infobox placement) differ.
  • More Granular Models: The current structure-based models assigned weights to entire sections. Future research could explore more detailed positional models or combine influences from multiple sections.
  • User Types and Scenarios: Further studies could compare navigation behavior across different user types and a wider range of usage scenarios.
  • Unsuccessful Games: Analyzing unsuccessful wikigames could provide insights into how users struggle and potentially inform the development of intelligent systems.

Summary updated

Abstract

In this work we study how people navigate the information network of Wikipedia and investigate (i) free-form navigation by studying all clicks within the English Wikipedia over an entire month and ...
BibTeX
@article{lamprecht2017how,
  author = {Daniel Lamprecht and Kristina Lerman and Denis Helic and Markus Strohmaier},
  title = {{How the structure of Wikipedia articles influences user navigation}},
  journal = {New Rev. Hypermedia Multim.},
  year = {2017},
  doi = {10.1080/13614568.2016.1179798},
}
Figure