Mail Explorer - Spatial and Temporal Exploration of Electronic Mail

Daniel A. Keim, Florian Mansmann, Christian Panse, Jörn Schneidewind, Mike Sips

TL;DR

This paper introduces "Mail Explorer," an interactive visual analytics tool designed to help users understand and manage their vast email communications. By combining spatial and temporal visualization techniques, it allows for the identification of communication patterns, spam sources, and important contacts, addressing the overwhelming nature of modern email inboxes.

What Problem Does This Paper Address?

In today's digital world, email has become a primary communication channel for both personal and professional life. However, the sheer volume of emails received daily, especially by individuals in large organizations, makes effective management and analysis a significant challenge. Users struggle to keep up with incoming messages, identify important contacts, and distinguish between legitimate and unwanted (spam) emails. Traditional email clients offer limited tools for exploring these large datasets, leading to frustration and potential missed information. This paper aims to bridge this gap by providing a visual analytics approach that allows users to explore their email communication in a more intuitive and insightful way.

Methods

The Mail Explorer employs a three-layer architecture to facilitate the analysis of email data.

  • Retrieval and Processing Layer: This layer interacts with IMAP servers to retrieve email headers. It then extracts relevant information, including sender/receiver addresses, timestamps, and network-related data, which are crucial for spatial and temporal analysis.
  • Visual Analytics Layer: This is the core of the Mail Explorer, offering several visualization techniques:
  • Geo-spatial Visualizations: Utilizes a modified HistoMap algorithm (a space-filling value-by-area technique) to visualize the geographic distribution of email senders. This approach addresses the issue of overlapping data points in dense areas and efficiently uses screen space.
  • Temporal Visualizations: Employs a "recursive pattern" technique to visualize the temporal distribution of emails over time. This method is particularly effective for identifying trends, peak hours, and seasonal patterns in email traffic.
  • Distorted Spatial Grids (HistoMap): This technique maps spatial data onto a grid where the size of each cell represents the volume of emails from that location, preserving geographic relationships.
  • Interaction Layer: This layer provides users with interactive capabilities, including zooming, filtering, and brushing, to explore the visualizations and drill down into specific data points or patterns.

The system is designed to be flexible, allowing for the integration of new visualization techniques.

Datasets

The paper describes experiments conducted on email data from several IMAP accounts, including those of the authors' working group, which contained over 100,000 emails. The data used for spatial analysis is derived from email headers, which are then geolocated using a geo-IP database (specifically mentioning the Maxmind GeoIP City Database). No new benchmarks or datasets were introduced; rather, existing email archives were used for evaluation.

Key Insights and Ideas

  • Visualizing the Overwhelming: The core idea is that visual analytics can make the massive volume of email data comprehensible, transforming a daunting task into an exploratory process.
  • Spatial Patterns Reveal More Than Just Location: Visualizing sender locations can help identify important business partners (e.g., "gold" or "silver" customers) and detect anomalies that might indicate spam or fraudulent activity.
  • Temporal Patterns Uncover Communication Rhythms: The recursive pattern visualization effectively highlights daily, weekly, and seasonal email traffic patterns, aiding in understanding communication flow and identifying periods of high activity.
  • Spam is a Global Phenomenon: The HistoMap visualization of spam folders revealed a surprisingly wide international distribution of spam senders, far exceeding the typical geographic spread of solicited email.
  • User-Centric Analysis: The Mail Explorer empowers users to analyze their own data, providing insights into their communication habits and allowing them to fine-tune their email filters more effectively.

Key Results

The paper presents qualitative results demonstrating the utility of the Mail Explorer. For instance, the visualization helped identify that a significant portion of emails for one user originated from Halle, Germany, and that spam emails came from over 40 different countries. The recursive pattern visualization was effective in showing a clear upward trend in spam for a re-opened mailbox, followed by a sharp decrease. The authors report that initial user feedback and experimental results with large email collections were "very promising."

Limitations and Future Work

  • Data Quality and Bias: The paper acknowledges that data extracted from email headers can be incomplete or biased due to the nature of network circumstances, which are beyond the user's control.
  • Preprocessing Challenges: While the system integrates preprocessing, handling all potential issues with header data remains a challenge.
  • Future Work: The authors suggest that the modular design of the Mail Explorer allows for the integration of additional visualization and interaction techniques. Further research could explore more advanced methods for analyzing temporal and spatial patterns, as well as potentially incorporating content analysis.

Summary updated

Abstract

In today s world, e-mail has become one of the most important means of communication in business and private lives due to its efficiency. However, the problems start as soon as mail volumes go beyond the scope of human information processing capabilities. Firstly, time does not allow for leaving certain messages unanswered for a long time, and in certain cases, for reading all messages. Secondly, the dilemma of electronic filters leaves a choice of too many junk mails getting through versus a risk of solicited mails being dumped. In this paper we present a new interactive visual data mining approach for analyzing individual e-mail communication. It combines classical visual analytics (help to identify pattern such as peaks and trends over time) with geo-spatial map distortions (help to understand the routes of e-mails). Experiments show that our visual e-mail explorer produces useful and interesting visualizations of large collections of e-mail and is practical for exploring temporal and geo-spatial patterns hidden in the e-mail data.
BibTeX
@inproceedings{keim2005mail,
  author = {Daniel A. Keim and Florian Mansmann and Christian Panse and Jörn Schneidewind and Mike Sips},
  title = {{Mail Explorer - Spatial and Temporal Exploration of Electronic Mail}},
  booktitle = {EuroVis},
  year = {2005},
  doi = {10.2312/vissym/eurovis05/247-254},
}
Figure