Abstract
Geotagging social media posts at a fine-grained level is challenging due to noisy text and significant class imbalance among Points-of-Interest (POIs). We present HierBERT, a two-level hierarchical classification model that leverages BERT for semantic text analysis and one-hot encoding for temporal data. By first categorizing posts into broad themes before predicting specific landmarks, HierBERT achieves over 90% accuracy (Acc@5) on Flickr and Twitter datasets, significantly outperforming flat classification baselines and reducing mean distance error.
Introduction: The Challenge of Fine-Grained Geolocation
Social media data is a valuable resource for location-based services, yet most posts lack explicit geotags. Predicting a post's location—geotagging—is complicated by the "noisy" nature of social media text, which is often filled with slang, typos, and abbreviations. While previous research has focused on coarse-grained location (e.g., city-level), this work addresses fine-grained geolocation at the landmark level. This task faces a severe "class imbalance" problem: popular landmarks like the Eiffel Tower have thousands of associated posts, while niche sites have very few. Standard flat classifiers often struggle to distinguish between hundreds of potential landmarks under such skewed data distributions.
Methods: The HierBERT Architecture
The authors introduce HierBERT, which utilizes a 2-level hierarchical structure combined with BERT (Bidirectional Encoder Representations from Transformers) to process multi-modal inputs.
Hierarchical Structure
Instead of a single-step prediction across all possible landmarks, HierBERT employs a "divide and conquer" strategy:
- Level 1 (Classifier): Predicts a broad "POI Theme" (e.g., "Leisure/Recreation").
- Level 2 (Sub-classifiers): For each identified theme, a specific sub-classifier predicts the actual landmark (e.g., "Carlton Gardens South").
Input Fusion
The model processes three primary data types:
- Textual Data: Post titles and descriptions are processed by BERT to generate semantic representations that handle context and noisy language effectively.
- Temporal Data: Information such as the hour, day of the week, and month is converted into one-hot encoded vectors.
- Fusion: BERT embeddings and temporal vectors are concatenated and passed through a fully connected (FC) layer and a softmax layer to produce final probabilities.
Experimental Evaluation
The model was evaluated on two large-scale datasets: Twitter (266,000 tweets) and Flickr (202,000 photos). Posts were mapped to specific POIs based on proximity to known landmarks.
Key Results
HierBERT consistently outperformed all baselines, including 1-level BERT models and various CNN-based approaches. As shown in the performance table, HierBERT achieves the highest accuracy and lowest mean distance error across both datasets.
- Accuracy (Acc@5): HierBERT achieved an accuracy of 91.19% on Flickr and 90.50% on Twitter, representing an improvement of 10.6% to 35.3% over baseline models.
- Mean Distance Error (MDist): The model significantly reduced physical distance error. On Flickr, the error dropped to 264.2 meters, compared to 403.5 meters for a standard 1-level BERT model.
Discussion and Future Work
The success of HierBERT highlights the importance of hierarchical filtering in mitigating class imbalance. Furthermore, the inclusion of temporal data proved crucial, as visit patterns for different POIs (e.g., nightclubs vs. parks) vary significantly over time.
Future work could enhance the model by integrating visual data from images, accounting for social network connections, and improving granularity in dense urban centers where landmarks frequently overlap.
Summary updated