Pairwise Location-Aware Publish/Subscribe for Geo-Textual Data Streams

Ying Zhong, Shunzhi Zhu, Yan Wang, Jianmin Li, Xinxin Zhang, Jedi S. ShangIEEE Access 2020 · DOI

Introduction

The explosion of social media and mobile services has created a "firehose" of geo-textual data—streams of information containing location coordinates, text snippets, and timestamps. Traditional publish/subscribe (pub/sub) systems are designed to notify users when a single item matches their interests. However, this approach often leads to high redundancy and low-quality notifications. Researchers have proposed a novel system that matches pairs of related objects to users, effectively filtering noise and providing context-rich information at scale.

The Challenge of Single-Item Matching

Traditional location-aware pub/sub systems suffer from three primary flaws:

  1. Redundancy: Multiple users reporting the same event (e.g., a goal at a stadium) result in repetitive notifications.
  2. Low Quality: Individual posts are often too brief or contain typos to be useful on their own.
  3. Lack of Context: Single items fail to capture the broader relationship between concurrent events in the same vicinity.

To address these, the system must move from "item-matching" to "relationship-matching" without sacrificing performance under high data velocity.

System architecture for pairwise geo-textual pub/sub

Methodology: A Two-Phase Matching Scheme

The proposed framework handles the complexity of pairwise matching through two distinct phases:

Phase 1: Online Geo-Textual Object Join (OGT-Join)

Instead of processing items in isolation, the system utilizes a time-based sliding window to identify pairs of objects that are similar in location, text, and time. To maintain real-time performance, an Incremental Online Join (IOJ) algorithm was developed. IOJ avoids redundant calculations by only processing the newest incoming item and removing the oldest as the window slides.

Phase 2: Subscription Matching

Once a high-quality pair is identified, it must be matched against millions of user subscriptions. The system supports two subscription types:

  • Region-Based (RLS): Matches pairs within a specific circular area containing target keywords.
  • Score-Based (SLS): Matches based on a weighted relevancy score of spatial and textual similarity.

Efficiency is maintained via a grid-based index that prunes the search space. By mapping subscriptions to specific grid cells, the system only evaluates subscriptions that overlap with the pair's location.

Grid-based index for efficient subscription matching

Experimental Evaluation

The system was evaluated using two real-world datasets: YEL (Yelp business reviews) and TWI (geo-tagged tweets). These datasets represent the high-velocity, noisy streams typical of modern social media.

Key Results

  • Performance: The proposed algorithms (OMPR and OMPS) achieved a 70-80% reduction in processing time compared to baseline methods.
  • Throughput: The system maintained stable performance even at high arrival rates where baselines struggled.
  • Scalability: The framework demonstrated linear scalability, supporting up to 4 million subscriptions without performance degradation.

Discussion and Future Directions

The "join" operation acts as a natural deduplicator, ensuring that users receive more "evidence" (e.g., two independent reports of a traffic jam) rather than a single, potentially unreliable post.

Future work may explore extending this to "clusters" of objects beyond pairs to further reduce redundancy. Additionally, integrating advanced NLP techniques like Transformers could improve semantic matching, though this would require significant optimization to maintain the system's real-time throughput.

Summary updated

BibTeX
@article{zhong2020pairwise,
  author = {Ying Zhong and Shunzhi Zhu and Yan Wang and Jianmin Li and Xinxin Zhang and Jedi S. Shang},
  title = {{Pairwise Location-Aware Publish/Subscribe for Geo-Textual Data Streams}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3038921},
}
Figure