Short and Long-term Pattern Discovery Over Large-Scale Geo-Spatiotemporal Data

Sobhan Moosavi, Mohammad Hossein Samavatian, Arnab Nandi, Srinivasan Parthasarathy, Rajiv Ramnath

TL;DR

Researchers at The Ohio State University have developed a new framework to uncover complex "cause-and-effect" patterns in massive traffic and weather datasets. By distinguishing between short-term "propagation" (e.g., rain leading to an accident) and long-term "influence" (e.g., how a months-long construction project changes local traffic), the study provides a more semantically accurate way to understand how our cities move.

What Problem Does This Paper Address?

Traditional spatiotemporal data mining often relies on "simplistic" definitions of a neighborhood—usually just a circle on a map and a shared window of time. This approach fails in the real world. For example, a car crash on the northbound side of a divided highway rarely affects southbound traffic, yet a standard algorithm would group them together. Similarly, a heavy snowstorm might end at noon, but its impact on traffic (slush, ice, slow speeds) lasts for hours afterward, breaking the "temporal overlap" rule many models require.

The authors address this by creating a framework that uses a more rigorous, semantically aware definition of spatiotemporal neighborhoods, allowing them to find patterns that actually represent causal chains rather than just random coincidences.

Methods

The framework splits pattern discovery into two distinct tracks:

  1. Short-term (Propagation) Patterns: The goal here is to find "cascading" events.
  2. The authors first identify Child-Parent Relationships (e.g., Event A happens near Event B, and A starts before B).
  3. These relationships are organized into Relation Trees.
  4. They then use the SLEUTH algorithm (a frequent tree-mining technique) to find common subtrees across the entire country. This reveals "recipes" for traffic chaos, like Rain → Accident → Heavy Congestion.
  5. Long-term (Influential) Patterns: This track looks at "long" entities (defined as the top 1% of durations, roughly >5 hours).
  6. They compare the frequency of traffic incidents in a neighborhood during a long-term event (like a major construction project) against the periods immediately before and after that event.
  7. They use two-sample t-tests across six different scenarios to determine if the long-term entity had a statistically significant impact on its surroundings.

Datasets

The study utilizes a massive, real-world dataset collected over two years (August 2016 – August 2018) across the contiguous United States:

  • Traffic Data: 13.1 million entities (accidents, congestion, construction, etc.) collected via the MapQuest API.
  • Weather Data: 2.2 million entities (rain, snow, fog, etc.) from nearly 2,000 airport weather stations.
  • The "LSTW" Dataset: The authors have made this integrated dataset publicly available for the research community.

Key Insights and Ideas

  • Weather as a Catalyst: 50 out of the 90 most common short-term patterns were initiated by a weather event. Interestingly, weather-induced traffic issues are more frequent in the morning, while general traffic-induced congestion peaks in the afternoon.
  • The "Construction Paradox": The study found that after very long construction projects (over 28 hours), traffic flow often became smoother than it was even before the construction started. The authors suggest two reasons: the road infrastructure actually improved, or drivers permanently adapted by finding alternative routes.
  • State-Level "Personalities": By clustering states based on their traffic patterns, the authors found that geography isn't everything. For instance, California (Cluster 4) stands alone with unique, complex congestion and fog-related patterns that don't mirror any other state.
  • Adaptation: For long-term entities lasting more than 40 hours, the "impact" on traffic often diminishes. This suggests that drivers are highly adaptable, adjusting their habits to "the new normal" of a long-term closure or weather condition.

Key Results

  • 90 Unique Propagation Patterns: The framework identified nearly 100 distinct ways traffic and weather events cascade.
  • Four State Clusters: The US states were categorized into four groups based on driving habits and infrastructure characteristics. Cluster 1 (e.g., Ohio, Minnesota) features states where weather impacts are well-mitigated by effective road crews, while Cluster 2 (e.g., Pennsylvania, North Carolina) sees significantly more weather-related chain reactions.
  • Quantifiable Impact: The study proved that shorter "long-term" entities actually have a more volatile impact on traffic than very long ones, likely due to the lack of time for driver adaptation.

Limitations and Future Work

The authors acknowledge that while they've identified these patterns at a state and national level, the impact can vary wildly between specific cities. Future work aims to "zoom in" on top-tier cities to see how urban density changes these propagation rules. Additionally, they plan to refine the "lagging impact" models to better predict exactly how long a region will remain affected after a major weather event concludes.

Summary updated

Abstract

Pattern discovery in geo-spatiotemporal data (such as traffic and weather data) is about finding patterns of collocation, co-occurrence, cascading, or cause and effect between geospatial entities. Using simplistic definitions of spatiotemporal neighborhood (a common characteristic of the existing general-purpose frameworks) is not semantically representative of geo-spatiotemporal data. We therefore introduce a new geo-spatiotemporal pattern discovery framework which defines a semantically correct definition of neighborhood; and then provides two capabilities, one to explore propagation patterns and the other to explore influential patterns. Propagation patterns reveal common cascading forms of geospatial entities in a region. Influential patterns demonstrate the impact of temporally long-term geospatial entities on their neighborhood. We apply this framework on a large dataset of traffic and weather data at countrywide scale, collected for the contiguous United States over two years. Our important findings include the identification of 90 common propagation patterns of traffic and weather entities (e.g., rain --> accident --> congestion), which results in identification of four categories of states within the US; and interesting influential patterns with respect to the "location", "duration", and "type" of long-term entities (e.g., a major construction --> more traffic incidents). These patterns and the categorization of the states provide useful insights on the driving habits and infrastructure characteristics of different regions in the US, and could be of significant value for applications such as urban planning and personalized insurance.
BibTeX
@inproceedings{moosavi2019short,
  author = {Sobhan Moosavi and Mohammad Hossein Samavatian and Arnab Nandi and Srinivasan Parthasarathy and Rajiv Ramnath},
  title = {{Short and Long-term Pattern Discovery Over Large-Scale Geo-Spatiotemporal Data}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2019},
  url = {https://openreview.net/forum?id=QVAc7S-oXE},
}

Loading related content...

Figure