Introduction
During large-scale crises, social media platforms like Twitter become critical channels for real-time information. However, disaster response agencies such as FEMA or the Red Cross often struggle with "information overload," making it difficult to identify credible sources amidst the noise. While official organizational accounts are relatively easy to track, a significant gap exists in identifying organization-affiliated users—individuals like field officers or volunteers who provide timely, ground-level insights but lack official branding. This paper proposes a machine learning framework to bridge this gap by classifying users based solely on profile metadata.
Methodology: Real-Time Credibility Assessment
To meet the demands of real-time emergency response, the proposed system avoids computationally expensive analysis of tweet history. Instead, it utilizes a supervised learning approach using features extracted exclusively from user profiles.
Feature Engineering
The model categorizes features into three functional groups:
- Social Features: Metrics like "Sociability" (the ratio of friends to followers) to gauge network influence.
- Activity Features: "Favorability" (likes relative to total tweets) and "Survivability" (account age) to measure engagement and longevity.
- Representation Features: "Informality" (use of emojis/hashtags in bios) and semantic embeddings (Word2Vec) of the bio text to capture how users describe their roles.
The researchers employed Gradient Boosted Decision Trees (GBDT) within One-vs-All (OVA) and One-vs-One (OVO) binarization frameworks to handle the multiclass classification of users into organization, organization-affiliated, and non-affiliated categories.
Behavioral Insights Across Disasters
The study evaluated the model using data from three distinct events: the 2016 Louisiana Floods, Hurricane Matthew, and global discussions on Gender-Based Violence (GBV). Analysis revealed that organization-affiliated users occupy a unique behavioral "middle ground."
As shown in the figure above, the distribution of social and activity metrics varies significantly between classes. For instance, affiliated users often exhibit higher network connectivity than official accounts during natural disasters, serving as vital information hubs. Furthermore, the importance of specific features shifts with the scale of the event; "listed counts" (the number of public lists a user belongs to) became more discriminative as the crisis moved from a local to a national level.
Key Results
The integrated model achieved high performance, reaching an F1-score of up to 92%. Key findings include:
- Feature Synergy: Combining Social, Activity, and Representation features consistently outperformed single-category baselines, confirming that credibility is a multi-faceted trait.
- Classification Strategy: The One-vs-One (OVO) scheme proved most effective at distinguishing the subtle differences between organization-affiliated individuals and the general public.
- Bio-Based Signaling: Even without tweet content, profile bios ("Founder of...", "Volunteer at...") provide sufficient signal for high-accuracy classification.
Conclusion and Future Directions
This research demonstrates that profile metadata is a powerful, low-latency signal for identifying credible voices during disasters. While highly effective on Twitter, future work should explore the model's generalizability to other platforms and its resilience against sophisticated bots. Additionally, investigating how users dynamically update their profiles during active crises could further refine real-time identification.
Summary updated