Why we search: visualizing and predicting user behavior

Eytan Adar, Daniel S. Weld, Brian N. Bershad, Steven D. GribbleWWW 2007 · DOI

TL;DR

This paper introduces a novel framework and visualization tool (DTWRadar) to analyze and correlate user behavior across diverse online platforms like search engines, blogs, and news sites. By modeling user interest as time-series data and using Dynamic Time Warping, they uncover predictive relationships between different online activities, offering insights into how events ripple across the web.

What Problem Does This Paper Address?

Understanding how user behavior on the internet changes over time and how these changes correlate across different platforms is a significant challenge. While individual platform behaviors (like search queries or blog posts) have been studied, correlating these diverse activities to understand the broader impact of events or trends has been largely unexplored. The authors aim to bridge this gap by developing methods to quantify and visualize these cross-platform behavioral patterns, enabling predictions about future user actions.

Methods

The core of this work lies in two main contributions:

  • Data Modeling: They formalize user behavior as "event streams," represented as tuples of <text, weight, time, dataset>. These streams are then transformed into time-series data.
  • Dynamic Time Warping (DTW): To compare time-series data that may be out of sync or have different temporal scales, they employ DTW. This algorithm finds an "optimal" alignment between two time series, accounting for variations in speed and magnitude. They also incorporate cross-correlation delays as a pre-processing step for DTW.
  • DTWRadar: This is a novel visualization artifact designed to summarize the differences between two time series after DTW alignment. It plots warp vectors on a normalized axis, providing a visual representation of lead/lag times and magnitude differences. This allows for a quick understanding of how one behavior stream relates to another.

Datasets

The study utilizes six diverse datasets:

  • MSN Query Logs: 7.51 million search sessions from May 1-31, 2006.
  • AOL Query Logs: 2.6 million search sessions from May 1-31, 2006.
  • Blog Dataset (BLOG): 14 million blog posts from May 1-24, 2006.
  • News Datasets (NEWS & BLOG-NEWS): Over 12,000 BBC articles and ~950 CNN articles from May 1-31, 2006. The NEWS dataset uses in-links as a weight, while BLOG-NEWS uses daily in-links from blogs.
  • TV Dataset (TV): 2,457 TV show records from TV.com for May 2006, weighted by user votes.

Key Insights and Ideas

  • Cross-platform behavior is predictable: The study demonstrates that events on one platform (e.g., a news article) can predict or correlate with user activity on others (e.g., search queries).
  • DTWRadar is an effective visualization: This tool provides an intuitive way to grasp complex temporal relationships between different user behavior streams, going beyond simple correlation coefficients.
  • Blogs can anticipate news: Bloggers often react to or anticipate events (like "news of the weird") before they become mainstream search queries.
  • Different platforms have different response characteristics: The study highlights how various platforms react to events with different speeds and magnitudes, offering insights into user engagement patterns.
  • Noise reduction is crucial: Techniques like binning and Gaussian smoothing are essential for making sense of noisy time-series data from diverse sources.

Key Results

  • The paper successfully generated time-series representations for a large number of topics across multiple datasets, with high overlap between search engine logs.
  • They found that a significant percentage of correlated time-series pairs exhibited delays, suggesting predictive potential. For instance, an average of 38% of delays were centered around 0 days, but many others showed consistent shifts.
  • The DTWRadar visualization was shown to be effective in summarizing these relationships, with users able to interpret lead/lag times and magnitude differences.
  • Qualitative analysis revealed interesting patterns, such as bloggers being more predictive for "news of the weird" topics, while search engines might react more to anticipated events like horse races.

Limitations and Future Work

  • Manual Annotation: While the DTWRadar aids visualization, the initial identification of interesting topics and the annotation of query categories required manual effort.
  • Data Source Limitations: The TV dataset's weighting by votes might not perfectly reflect viewership, and the blog dataset can contain noise from irrelevant posts.
  • Correlation vs. Causation: The study primarily identifies correlations, and distinguishing true causation remains a challenge.
  • Future Work: The authors plan to automate predictions by applying machine learning techniques to the discovered behavioral patterns. They also aim to incorporate more sophisticated measures of behavior and explore portal-specific differences in search engine front pages. Further refinement of the DTWRadar and its variants is also suggested.

Summary updated

BibTeX
@inproceedings{adar2007why,
  author = {Eytan Adar and Daniel S. Weld and Brian N. Bershad and Steven D. Gribble},
  title = {{Why we search: visualizing and predicting user behavior}},
  booktitle = {The Web Conference},
  year = {2007},
  doi = {10.1145/1242572.1242595},
}
Figure