Introduction
Social media platforms like Twitter offer a wealth of real-time feedback for transportation companies and urban planners. However, extracting actionable complaints from the vast volume of "noise" is a significant challenge. Traditional supervised learning methods often struggle due to the rapid evolution of language (slang, new infrastructure issues) and the inherent class imbalance where complaints represent a small fraction of total data. This paper presents a semi-supervised, iterative pipeline designed to identify transport-related complaints with minimal human intervention.
Methodology: The Iterative Pipeline
The proposed framework moves from specialized domain knowledge to general social media data through a multi-stage process. Initially, the researchers "bootstrapped" the system by scraping 500 posts from The Verge’s transportation forum. Using tf-idf, they identified 50 core "seed" terms to query Twitter, yielding an initial pool of 19,300 tweets.
A small subset of 1,500 tweets was manually annotated to identify "informative" complaints. These served as the basis for an iterative loop designed to refine the dataset.
As shown in Figure 1, the loop involves extracting new seed phrases from informative tweets, filtering redundant content via cosine similarity, and calculating a Domain Relevance Score to ensure phrases remain contextually appropriate. This process was repeated over four iterations, eventually growing the transport-relevant dataset to 7,200 tweets.
Feature Analysis and Results
The researchers evaluated the effectiveness of various feature sets—Linguistic, Sentiment, and Information-specific—using Logistic Regression with Elastic Net regularization.
The results, detailed in Figure 2, reveal several key insights:
- Linguistic Superiority: Simple Bag-of-Words (BoW) features achieved the highest performance with 75.3% accuracy and a 0.71 F1-score. This suggests that specific "problem" keywords are more indicative of a complaint than complex emotional tone.
- The Role of Intensifiers: Information-specific markers, particularly "Intensifiers" (e.g., excessive capitalization or punctuation), proved highly effective, reaching 72.5% accuracy.
- Sentiment Limitations: While sentiment analysis (specifically the Stanford Sentiment model) provided a signal, it generally underperformed compared to linguistic and structural markers.
Conclusion and Future Work
This research demonstrates that a semi-supervised, iterative approach can effectively build a robust dataset for domain-specific complaint identification. By evolving the lexicon through multiple cycles, the system captures linguistic variations that static models miss.
Future work will explore the integration of neural networks, which may offer higher performance if larger labeled datasets can be generated. Additionally, the researchers plan to incorporate demographic analysis to better understand the populations affected by service issues and test the framework's generalizability in other sectors like healthcare and retail.
Summary updated