Introduction
The rapid growth of online text data, including news and social media, has made the extraction of geographical information increasingly vital for fields such as disaster relief, urban planning, and transportation. Identifying and classifying texts that contain relevant geographical entities is a high-value task, yet it remains computationally challenging, particularly in the context of Chinese natural language processing (NLP). Traditional models like Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks often struggle with the non-Euclidean structure of text-based graphs and the complexities of Chinese word segmentation.
To address these limitations, the authors propose geoGAT, a Graph Attention Network (GAT) model specifically designed for Chinese geographic text classification. By representing text as a graph and employing an attention mechanism to dynamically weight the importance of neighboring nodes, geoGAT achieves a 95% Macro-F score, significantly outperforming traditional architectures.
Methodology
The geoGAT framework treats the geographic text classification task as a node classification problem within a heterogeneous graph. The process is divided into three primary stages: graph construction, the application of the attention mechanism, and multi-head attention aggregation.
Graph Construction
The entire dataset is converted into a large heterogeneous graph where nodes represent both individual documents (sequences) and unique words. The relationships (edges) are defined by:
- Word-Word Edges: Weighted using Pointwise Mutual Information (PMI) to capture global word co-occurrence.
- Word-Document Edges: Weighted using Term Frequency-Inverse Document Frequency (TF-IDF) to represent the importance of a word within a specific text.
Attention Mechanism
The core innovation of geoGAT is its departure from standard Graph Convolutional Networks (GCNs). While GCNs aggregate neighbor information using fixed weights, geoGAT utilizes an attention mechanism to learn the relative importance of neighboring nodes. This allows the model to focus on specific keywords that are highly indicative of geographical relevance.
As shown in the figure above, the model computes attention coefficients $\alpha_{ij}$ by concatenating transformed feature vectors ($Wh_i$ and $Wh_j$) and applying a learnable weight vector $a^T$. This process ensures that the model captures spatial and contextual correlations more effectively than "blurring" features across the graph.
Multi-head Attention
To stabilize the learning process and increase model capacity, geoGAT employs multi-head attention. Multiple independent attention "heads" learn features in different representation spaces. These representations are then averaged or concatenated to produce the final output, leading to faster convergence and more robust feature extraction.
Dataset and Experiments
The authors developed a specialized Chinese dataset comprising 55,000 samples from four major sources: ByteDance News, THUCNews, Fudan University Chinese Text, and SogouCA. The data is categorized into five domains: Geography, Games, Cars, Economy, and Entertainment. The "Geography" category was meticulously curated to include only texts with clear, non-virtual geographical locations.
A significant finding during experimentation was the model's robustness to segmentation. Unlike CNNs and RNNs, which show performance degradation when moving from character-level to word-level segmentation due to noise and data sparsity, graph-based models like geoGAT maintained high accuracy regardless of the segmentation method.
Key Results
The geoGAT model demonstrated superior performance across all primary metrics compared to baselines such as TextGCN, BiLSTM-Att, and DPCNN:
- Macro-F Score: Achieved a peak of 95.20%.
- Geographic Precision/Recall: Reached 98.34% Precision and 97.87% Recall specifically for the geography class.
- Efficiency: Multi-head attention allowed the model to converge within approximately 200 epochs, proving more efficient than simpler self-attention mechanisms despite the increased computational load per epoch.
Conclusion and Future Work
While geoGAT represents a significant advancement in geographic text classification, challenges remain regarding toponym ambiguity (e.g., distinguishing between names of people and places). Future research will focus on integrating entity disambiguation techniques and exploring multi-layer graph structures to deepen feature extraction and improve the resolution of complex geographical information.
Summary updated