HistoryComparator: Interactive Across-Time Comparison in Document Archives

Adam Jatowt, Marc BronCOLING 2016

TL;DR

This paper introduces HistoryComparator, an interactive framework for comparing document collections across different time periods. It addresses the challenge of understanding historical trends and cultural shifts by offering novel visualization tools that highlight temporal differences in term usage and relationships.

What Problem Does This Paper Address?

In the digital age, we have access to vast archives of historical documents. While traditional methods like keyword searching and browsing are useful, they often fall short when trying to understand how concepts, events, or entities have evolved over time. Historians and social scientists frequently rely on comparative analysis to gain insights, but computational tools for this specific task, especially for large-scale document collections, are limited. The paper aims to bridge this gap by providing an interactive system that allows users to visually compare query results from different historical periods, making temporal shifts in language and meaning more apparent.

Methods

HistoryComparator employs a dual-column visualization approach, presenting two time periods side-by-side for immediate comparison. The core of the system is built upon a retrieval engine that fetches documents relevant to user queries within specified time constraints. It offers two primary comparison views:

  • Contrastive Term Cloud View: This view generates term clouds for each time period, where term size indicates frequency and color highlights differences in frequency between the two periods (blue for left column prevalence, red for right). Users can adjust the number of terms, ordering, and grouping.
  • Contrastive Graph View: This view visualizes inter-word relationships as force-directed graphs. Nodes represent terms, with size indicating frequency. Edges connect co-occurring terms, with width reflecting their Jaccard coefficient. Node colors again indicate frequency differences, and edge colors show affinity changes.

Key functionalities include:

  • Keyword in Temporal Context: Clicking a term reveals example sentences and related information, allowing for direct comparison of how a term is used in different eras.
  • Similar Terms: The system identifies and displays terms with similar co-occurrence patterns.
  • Time-based Term Adjustment: This feature helps distinguish genuine temporal shifts from changes due to general time passage by using background document collections for normalization.
  • Sentiment Analysis: Temporal sentiment trends are visualized, showing shifts in positive vs. negative connotations associated with query terms.

Datasets

The paper does not explicitly list specific datasets used for training. However, it mentions using the New York Times Article Archive on Solr, Google News Archive, and Google Blog search engines as underlying data sources for their system. The content of collected documents undergoes stop word removal, tokenization, and normalization.

Key Insights and Ideas

  • Visualizing Temporal Nuance: The system's strength lies in its ability to visually represent subtle changes in language and meaning over time, which is crucial for historical and cultural analysis.
  • Beyond Simple Frequency: By using color coding for term clouds and edge coloring for graphs, HistoryComparator goes beyond just showing what terms are present, but how their prominence and relationships change.
  • Context is King: The "Keyword in Temporal Context" feature is vital, as it grounds the abstract term clouds and graphs in concrete examples, allowing users to understand why a term's prominence or sentiment might have shifted.
  • Distinguishing Time Passage Effects: The method for adjusting for general time passage (e.g., increased use of "computer") is a sophisticated way to isolate changes directly related to the query's subject matter.

Key Results

The paper presents a framework and its interactive views rather than quantitative results from a specific experiment. The "key results" are the functionalities and visualizations themselves, which are demonstrated through an example query ("world trade center"). The figures showcase:

  • Figure 1: A comparison of term clouds for "world trade center" immediately after 9/11 and about 10 years later, highlighting shifts in associated terms.
  • Figure 2: Popup windows illustrating how a term like "construction" is used differently in the two time periods.
  • Figure 3: A term cloud example with visual adjustments for time passage effects.
  • Figure 4: A comparison of contrastive graphs, showing how the network of related terms for "world trade center" has evolved.

The qualitative findings suggest that the system effectively highlights differences in term usage and sentiment across time periods.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that term clouds and graphs can become cluttered with too many nodes or links, though sliders are provided to manage this.
  • The effectiveness of the system relies on the quality and comprehensiveness of the underlying document collections.
  • Future Work:
    • The authors plan to suggest relevant and interesting time periods for contrasting entities by comparing term distributions over time.
  • Further exploration of how to automatically identify and present the most significant temporal shifts.

Summary updated

BibTeX
@inproceedings{jatowt2016historycomparator,
  author = {Adam Jatowt and Marc Bron},
  title = {{HistoryComparator: Interactive Across-Time Comparison in Document Archives}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
}

Loading related content...

Figure