Sensing Urban Transportation Events from Multi-Channel Social Signals with the Word2vec Fusion Model

Hao Lu, Kaize Shi, Yifan Zhu, Yisheng Lv, Zhendong Niu

TL;DR

This paper introduces a novel Word2vec-based Event Fusion (WBEF) model to detect and fuse urban traffic events by leveraging multi-channel social signals from news and social media. By semantically integrating information from these diverse sources, the WBEF model achieves significantly improved accuracy and recall in urban traffic incident detection compared to traditional methods.

What Problem Does This Paper Address?

Intelligent transportation systems (ITS) rely on accurate and timely information about traffic conditions. While physical sensors are expensive and have limited coverage, social media and online news offer a vast, low-cost source of real-time information. However, existing research primarily focuses on single-channel social signals, missing the richer context available from cross-platform data. This paper addresses the challenge of effectively sensing, processing, and fusing information from multiple social media sources (specifically, Sina Weibo and news articles) to gain a more comprehensive understanding of urban traffic events. The core problem is how to semantically represent and fuse these multi-channel signals into meaningful traffic event descriptions.

Methods

The proposed approach, the Word2vec-based Event Fusion (WBEF) model, involves several key steps:

  • Multi-channel Social Sensor Network: A keyword-based network is established to collect data from news and Weibo. This network categorizes keywords into traffic event, urban identity, road identity, and domain assistant types to guide data collection.
  • Data Preprocessing: Raw data undergoes noise filtering, meta-data extraction, word segmentation (crucial for Chinese text), and aggregation into "data blocks" associated with specific urban roads.
  • Transportation Event Detection:
    • News: A standard Latent Dirichlet Allocation (LDA) model is used to extract topics from news articles.
  • Weibo: A modified LDA model, termed w-LDA, is employed to handle the unique characteristics of short, user-generated content from Weibo, focusing on user profiles.
  • Transportation Event Representation: To capture semantic similarity between topics from different platforms, words are represented using 200-dimensional word embeddings trained with the Continuous Bag-of-Words (CBOW) model and Negative Sampling for efficiency.
  • Transportation Events Fusion:
    • Event Alignment: Topics from news and Weibo are aligned based on word embedding similarity to create topic pairs.
  • Event Fusion: Aligned topic pairs are fused into a unified event description. Anomalous words are identified and removed, and candidate words are used to replace them for a more objective description.

Datasets

The study utilized a multi-channel Qingdao transportation dataset collected from August 1, 2015, to August 4, 2017. This dataset comprises approximately 1.15 million texts, including:

  • 301,684 News articles
  • 839,587 Weibo posts

The dataset was divided into:

  • Training Dataset: Approximately 2 years of data for model parameter learning.
  • Testing Dataset: 7 days of data for model performance evaluation.
  • Case Study Dataset: 1 day of data for qualitative analysis in an open scenario.

The paper does not introduce a new benchmark dataset but uses a real-world dataset from Qingdao.

Key Insights and Ideas

  • The Power of Multi-Channel Data: Combining news (objective, structured) and Weibo (timely, opinionated) provides a more comprehensive view of traffic events than either source alone.
  • Semantic Fusion is Crucial: Simply matching keywords isn't enough. Using word embeddings to capture semantic similarity between topics from different platforms is key to effective fusion.
  • Adapting Topic Models: A standard LDA model works for news, but a modified w-LDA model is necessary to effectively process the unique characteristics of social media data like Weibo.
  • Word Embeddings for Traffic: Training word embeddings specifically on transportation-related text captures essential semantic relationships, improving event representation.

Key Results

  • Improved Detection Accuracy: The WBEF model achieved an F₁ score of 88.1% in urban traffic incident detection, significantly outperforming the baseline "standard LDA + keywords matching" model, which achieved 70.9%.
  • Higher Recall: The WBEF model demonstrated a recall of 85.1%, substantially higher than the baseline's 57.4%, indicating it misses fewer events.
  • Precision Trade-off: While the WBEF model's precision (91.4%) was slightly lower than the baseline (92.6%), the overall improvement in F₁ score highlights its superior performance.
  • Qualitative Case Study: A case study in Qingdao showed that the WBEF model provided more comprehensible and understandable descriptions of traffic events compared to the baseline.

Limitations and Future Work

  • Limitations:
    • The current social sensor network relies on keywords, which can lead to false positives from non-traffic-related content that happens to use traffic keywords.
  • The precision is slightly lower than the baseline, potentially due to noise in the keyword-based data collection.
  • Future Work:
    • Incorporate more diverse social signal sources (e.g., Instagram, Facebook, Quora) to enhance accuracy and comprehensiveness.
  • Explore powerful deep learning methods to further improve accuracy and robustness.
  • Extend the WBEF model to heterogeneous recommending systems for more personalized transportation services.
  • Integrate social sensors with physical sensors to create Cyber-Physical-Social Systems (CPSS) for more holistic intelligent transportation management.

Summary updated

BibTeX
@inproceedings{lu2018sensing,
  author = {Hao Lu and Kaize Shi and Yifan Zhu and Yisheng Lv and Zhendong Niu},
  title = {{Sensing Urban Transportation Events from Multi-Channel Social Signals with the Word2vec Fusion Model}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=QJyLD4qHAM},
}
Figure