A Coherent Unsupervised Model for Toponym Resolution

Ehsan Kamalloo, Davood Rafiei

Abstract

Toponym Resolution, the task of assigning a location mention in a document to a geographic referent (i.e., latitude/longitude), plays a pivotal role in analyzing location-aware content. However, the ambiguities of natural language and a huge number of possible interpretations for toponyms constitute insurmountable hurdles for this task. In this paper, we study the problem of toponym resolution with no additional information other than a gazetteer and no training data. We demonstrate that a dearth of large enough annotated data makes supervised methods less capable of generalizing. Our proposed method estimates the geographic scope of documents and leverages the connections between nearby place names as evidence to resolve toponyms. Our model is evaluated on three news corpora and compared to state-of-the-art unsupervised and supervised techniques, as well as commercial products like Google Cloud Natural Language API. The evaluation shows that our method outperforms unsupervised techniques and commercial APIs, showing performance close to supervised methods and surpassing them when test data contains many unseen toponyms.

Introduction

Identifying the correct geographic referent for a place name in text—such as distinguishing between Paris, France, and Paris, Texas—is a fundamental challenge in natural language processing known as Toponym Resolution. The GeoNames database lists nearly 100 interpretations for "Paris" alone, illustrating the extreme ambiguity machines must navigate. While supervised models exist, they often require expensive hand-annotated data and struggle to generalize to new regions. This paper introduces Context-Hierarchy Fusion (CHF), an unsupervised model that relies solely on document context and a standard gazetteer to achieve high-accuracy resolution without training data.

Methods

The CHF model integrates two distinct unsupervised strategies to resolve ambiguity:

  1. Context-Bound Hypotheses (CBH): A probabilistic approach leveraging geo-centre inheritance (assuming a document has a specific geographic scope) and near-location clues (assuming toponyms mentioned together are often physically close).
  2. Spatial-Hierarchy Sets (SHS): A structural approach that treats resolution as a "conflict-free set cover" problem. It seeks the minimum number of parent regions (e.g., states or countries) that can encompass all mentioned locations, enforcing geographic consistency across the document.

The final CHF model uses a confidence threshold ($\tau$). It prioritizes the probabilistic CBH model when confidence is high and falls back to the SHS model's structural logic to ensure spatial minimality when ambiguity persists.

Datasets

The model was evaluated on three news corpora with varying characteristics:

  • TR-News: A new dataset featuring "less dominant" interpretations to test robustness.
  • Local-Global Lexicon (LGL): Difficult local news articles with highly ambiguous small-town names.
  • CLUST: A large-scale dataset with over 13,000 documents.
Distribution of toponym types across the three evaluation corpora.

As shown in Figure 1, the datasets vary significantly in their composition. The LGL dataset, for instance, contains a higher proportion of specific city and county mentions compared to TR-News, making it a particularly challenging benchmark for resolution algorithms.

Key Results and Discussion

The CHF model demonstrated significant improvements over existing unsupervised methods and commercial solutions. On the TR-News dataset, CHF achieved a Precision of 79.3%, outperforming the previous state-of-the-art unsupervised model, TopoCluster (68.8%).

A critical finding of this research is the "Generalization Trap" of supervised models. While supervised systems perform well on data similar to their training sets, their performance drops sharply when encountering new locations.

Performance comparison between CHF and a supervised model (CustomAdaptive) across varying overlap ratios.

Figure 3 illustrates that when the "overlap ratio" (the percentage of toponyms in the test set that were also seen during training) falls below approximately 60%, the unsupervised CHF model begins to outperform the state-of-the-art supervised method. This highlights the robustness of the unsupervised approach for real-world applications where news content is geographically diverse. Furthermore, CHF outperformed commercial APIs like Google Cloud Natural Language and Reuters OpenCalais across all datasets.

Conclusion

The Context-Hierarchy Fusion model demonstrates that document-level coherence and spatial hierarchy are powerful signals for resolving geographic ambiguity. By avoiding the need for labeled training data, CHF provides a more generalizable solution for global news analysis. Future work will explore improving resolution for short texts and utilizing bounding boxes for larger regions to further refine accuracy.

Summary updated

Abstract

Toponym Resolution, the task of assigning a location mention in a document to a geographic referent (i.e., latitude/longitude), plays a pivotal role in analyzing location-aware content. However, the ambiguities of natural language and a huge number of possible interpretations for toponyms constitute insurmountable hurdles for this task. In this paper, we study the problem of toponym resolution with no additional information other than a gazetteer and no training data. We demonstrate that a dearth of large enough annotated data makes supervised methods less capable of generalizing. Our proposed method estimates the geographic scope of documents and leverages the connections between nearby place names as evidence to resolve toponyms. We explore the interactions between multiple interpretations of mentions and the relationships between different toponyms in a document to build a model that finds the most coherent resolution. Our model is evaluated on three news corpora, two from the literature and one collected and annotated by us; then, we compare our methods to the state-of-the-art unsupervised and supervised techniques. We also examine three commercial products including Reuters OpenCalais, Yahoo! YQL Placemaker, and Google Cloud Natural Language API. The evaluation shows that our method outperforms the unsupervised technique as well as Reuters OpenCalais and Google Cloud Natural Language API on all three corpora; also, our method shows a performance close to that of the state-of-the-art supervised method and outperforms it when the test data has 40% or more toponyms that are not seen in the training data.
BibTeX
@inproceedings{kamalloo2018coherent,
  author = {Ehsan Kamalloo and Davood Rafiei},
  title = {{A Coherent Unsupervised Model for Toponym Resolution}},
  booktitle = {The Web Conference},
  year = {2018},
  doi = {10.1145/3178876.3186027},
}
Figure