TL;DR
This paper introduces TESLA, a novel annotation tool designed to support the creation of geospatial language corpora. By integrating GIS databases and Google Earth visualization with synchronized playback of speech and GPS tracks, TESLA streamlines the process of grounding natural language descriptions of locations, movement, and orientation in real-world spatial data.
What Problem Does This Paper Address?
Understanding natural language descriptions of spatial locations, orientation, and movement is a challenging but crucial area for applications like automated geotagging, robot navigation, and NL-based localization. Developing robust algorithms for this requires high-quality, "gold-standard" annotated data. However, creating such corpora is labor-intensive, especially when needing to ground linguistic references to real-world entities and their dynamic positions. This paper addresses the need for an efficient and effective tool to facilitate this annotation process.
Methods
TESLA is built upon the NXT Toolkit and its data model, leveraging Google Earth for visualization and the TerraFly GIS database for geospatial entity search. The core innovation lies in its integrated approach:
- Synchronized Playback: TESLA allows for synchronized playback of audio (speech) and GPS tracks. This enables annotators to follow the movement of objects (like cars) in real-time while listening to descriptions, providing crucial positional context.
- GIS Integration: It supports searching a GIS database (TerraFly) for geospatial entities based on keywords or proximity to the current GPS location. This automates the retrieval of potential ground truth for annotated references.
- Google Earth Visualization: Search results and entity information are displayed in Google Earth, allowing annotators to visually verify the correct entity and its location.
- Annotation Schema: TESLA supports annotating referring expressions to locations. The annotation schema focuses on identifying the geospatial entity itself (e.g., a street name, a business) rather than just latitude/longitude coordinates, which aids in coreference resolution and richer entity information.
Datasets
The paper describes the PURSUIT Corpus, a newly annotated dataset for geospatial language understanding. This corpus consists of:
- 13 audio recordings of speech data.
- 7 paths along which the speech was recorded.
- Corresponding high-sampling-rate GPS tracks for each path.
The speech data was collected from subjects describing their real-time paths while traveling, with synchronized GPS recording. This differs from existing benchmarks by providing synchronized, real-time, first-person, ground-level path descriptions grounded by GPS data.
Key Insights and Ideas
- The Power of Synchronization: The synchronized playback of speech and GPS tracks is a critical feature. It allows annotators to intuitively link linguistic descriptions to the actual physical movement and context, significantly improving annotation accuracy and efficiency.
- Visual Grounding is Key: Using Google Earth for visualization is not just for display; it's essential for verification. Seeing the entity and its location in a familiar, interactive map interface helps annotators confirm their choices and understand potential ambiguities.
- Entity-Centric Annotation: Annotating the specific geospatial entity (e.g., "Main Street") rather than just its coordinates is a more robust approach. It facilitates coreference resolution and allows for richer metadata about the entity to be associated with the annotation.
- Extensibility: While currently focused on location references, TESLA is designed to be extended to other annotation tasks like movement and orientation, highlighting its potential as a general-purpose tool for geospatial language.
Key Results
The paper primarily focuses on the development and application of the TESLA tool. While specific quantitative results on annotation accuracy or speedup are not detailed, the paper demonstrates TESLA's utility through its use in annotating the PURSUIT Corpus. The annotation process involves:
- Identifying referring expressions to locations.
- Using TESLA's search and visualization features to find and verify corresponding geospatial entities in the TerraFly GIS database.
- Annotating the entire noun phrase referring to the location.
- The PURSUIT Corpus, as described, is a benchmark for real-time, spoken path descriptions grounded with GPS tracks.
Limitations and Future Work
Limitations:
- The current annotation schema does not include entities not present in the TerraFly database, such as fields or parking lots.
- The PURSUIT Corpus primarily consists of first-person, ground-level perspective descriptions.
Future Work:
- Extending TESLA to support annotation of movement, orientation, and more complex path descriptions.
- Using the PURSUIT Corpus as training and testing data for algorithms that automatically annotate geospatial language.
- Gathering spoken descriptions from an aerial perspective using TESLA's playback capabilities to compare different viewpoints of the same path.
Summary updated