Joint Spatio-Textual Reasoning for Answering Tourism Questions

Danish Contractor, Shashank Goel, Mausam, Parag Singla

TL;DR

This paper introduces the first joint spatio-textual reasoning model for answering tourism questions that require understanding both location-based constraints and textual information from reviews. By combining a novel spatial reasoning network with a textual reasoner, the model significantly outperforms existing methods on a real-world POI recommendation task, demonstrating the importance of integrating these two modalities for complex question answering.

What Problem Does This Paper Address?

Answering questions like "Suggest a restaurant near Hotel Florida that's not too far" is challenging because it requires understanding not only the textual content of reviews but also spatial relationships between locations. Existing Question Answering (QA) systems often excel at either textual or spatial reasoning but struggle to combine them effectively. This gap is particularly evident in recommendation tasks where users express nuanced preferences involving both descriptive text and geographical constraints, often with ambiguous language. The authors aim to bridge this gap by developing a model that can jointly reason over geo-spatial data and textual corpora for more accurate and context-aware recommendations.

Methods

The core of the proposed solution is a Joint Spatio-Textual Reasoning Network composed of three main components:

  1. Geo-Spatial Reasoner (SPNET): This module is designed to handle spatial constraints. It includes:
  2. Distance-aware Question Encoder: This encoder processes question tokens, their location mentions, and crucially, the distance between these locations and candidate POIs. It also uses B-I-O (Begin-Intermediate-Other) labels to identify multi-token location names.
  3. Distance Reasoning Layer (DRL): This layer learns to infer the importance of different location mentions in the question based on the spatial constraints (e.g., "near," "far"). It transforms distance information into learned "distance-weights."
    • Spatial Relevance Scorer: This component calculates a "spatial relevance" score for each candidate POI by combining the learned distance-weights with the actual spatial distances.
  4. Textual Reasoner: This component utilizes the CRQA model, which is a Siamese-Encoder architecture. It attends to entity reviews and generates question-aware entity embeddings to compute a textual relevance score. To improve efficiency, it uses a subset of representative sentences from reviews clustered in a USE-embedding space.
  5. Joint Scoring Layer: This layer combines the scores from the spatial and textual reasoners. It uses learned weights (α and β) to dynamically balance the influence of spatial and textual information, allowing the model to rely more on textual reasoning for questions without explicit spatial constraints.

Datasets

The paper uses two primary datasets:

  1. Artificial Toy-Dataset: This dataset was created by the authors using linguistically diverse templates to generate questions with various spatial constraints (close to, far from, or a combination). It includes distractor locations to simulate real-world complexity. This dataset allowed for detailed analysis of the spatial reasoner's capabilities.
  2. Tourism Questions Dataset: This is a real-world dataset (previously released by the authors) containing over 47,000 tourism questions and POI recommendation pairs. It includes POI geo-coordinates and collections of reviews. This dataset was used for evaluating the joint spatio-textual model.

Key Insights and Ideas

  • Joint Reasoning is Crucial: The most significant insight is that effectively answering tourism questions requires a synergistic combination of spatial and textual reasoning. Neither modality alone is sufficient, and their integration leads to substantial performance gains.
  • Distance Matters, But Contextually: The DRL component highlights that simply knowing distances isn't enough. The model needs to learn how to interpret those distances based on the specific spatial constraints expressed in the question (e.g., "near" implies a different weight than "far").
  • Handling Ambiguity and Distractors: The model demonstrates robustness in handling ambiguous spatial language ("not too far") and effectively filters out irrelevant "distractor" locations that are present in the question but not pertinent to the recommendation.
  • Flexibility in Scoring: The joint scoring layer's ability to dynamically weigh spatial and textual scores is key. It allows the model to adapt to questions that might be purely textual or heavily reliant on spatial information.

Key Results

  • Significant Improvements: The proposed joint spatio-textual model achieved substantial improvements over models that used only spatial or only textual reasoning on the real-world Tourism Questions dataset.
  • State-of-the-Art Performance: The model established a new state-of-the-art on the task, particularly for questions involving location mentions.
  • Toy Dataset Performance: On the artificial toy dataset, the SPNET model (spatial reasoner) achieved high accuracy (e.g., 90.20% Acc@3 for "Close to Set X" with BERT, 98.00% Acc@3 for "Far from Set X").
  • Robustness to Distractors: While performance slightly degrades in the presence of distractor-locations, the joint model shows a less significant drop compared to baselines.
  • Human Evaluation: A blind human study confirmed the superiority of the joint model, with Acc@3 scores significantly higher than the textual-only baseline.

Limitations and Future Work

  • Candidate Space Size: The authors note that the model's performance can degrade with very large candidate spaces (over 1000 entities), which is a common challenge in POI recommendation.
  • Location Tagger Errors: Errors can arise from the location tagger itself or from inaccuracies in the geo-spatial data.
  • Complex Spatial Inference: While the model handles "near" and "far," future work could explore more complex spatial inferences like directional ("north of X") or topographical ("on the river beach") constraints.
  • Augmenting Neural IR: The authors suggest augmenting general-purpose neural IR methods with spatial reasoning capabilities, rather than relying solely on a separate spatial reasoner.
  • Scalability: For graph-based reasoning approaches, scalability remains a challenge due to the potentially thousands of nodes (entities) per question.

Summary updated

Abstract

Our goal is to answer real-world tourism questions that seek Points-of-Interest (POI) recommendations. Such questions express various kinds of spatial and non-spatial constraints, necessitating a combination of textual and spatial reasoning. In response, we develop the first joint spatio-textual reasoning model, which combines geo-spatial knowledge with information in textual corpora to answer questions. We first develop a modular spatial-reasoning network that uses geo-coordinates of location names mentioned in a question, and of candidate answer POIs, to reason over only spatial constraints. We then combine our spatial-reasoner with a textual reasoner in a joint model and present experiments on a real world POI recommendation task. We report substantial improvements over existing models without joint spatio-textual reasoning. To the best of our knowledge, we are the first to develop a joint QA model that combines reasoning over external geo-spatial knowledge along with textual reasoning.
BibTeX
@inproceedings{contractor2021joint,
  author = {Danish Contractor and Shashank Goel and Mausam and Parag Singla},
  title = {{Joint Spatio-Textual Reasoning for Answering Tourism Questions}},
  booktitle = {The Web Conference},
  year = {2021},
  url = {https://openreview.net/forum?id=LoM5PQU2wFx},
}
Figure