TL;DR
The authors introduce ARABGEOTEXT, the first Arabic-centric benchmark for geospatial text spotting, featuring paired map and satellite-overlay views. They demonstrate that while large Vision-Language Models (VLMs) are powerful, a "tool-augmented" approach—using a specialized YOLOv10 detector to feed rectified crops to a LoRA-adapted Qwen3.5-4B reader—significantly outperforms direct end-to-end spotting.
What Problem Does This Paper Address?
Geospatial text spotting is the task of automatically locating and transcribing labels (like street names and landmarks) from raster maps. This is notoriously difficult because map text is often tiny, densely packed, and oriented at arbitrary angles.
For Arabic, the challenge is even steeper. The script's cursive nature, where letter forms change based on their position, and the reliance on small dots (diacritics) for character differentiation, makes it highly sensitive to low resolution and noise. While Arabic OCR has matured for documents and natural scenes, geospatial data has remained a major gap. Furthermore, existing models often struggle to generalize when text is overlaid on complex backgrounds, such as remote sensing (RS) satellite imagery.
Methods
The researchers compared two primary paradigms for solving this task:
- Direct VLM Spotting: A single Qwen3.5-4B model is prompted to output a structured list of all polygons and their corresponding transcriptions directly from a full 1024x1024 image.
- Tool-Augmented VLM Spotting: This pipeline decouples localization from recognition:
- Localization: A specialized YOLOv10 instance segmentation model identifies text regions.
- Crop Preparation: Detected regions are extracted, rectified using a projective transform to "un-rotate" the text, and upscaled to a standard size.
- Recognition: The processed crops are fed to a Qwen3.5-4B reader. The authors evaluated both a zero-shot version and one fine-tuned using Low-Rank Adaptation (LoRA) on the specific vocabulary and styles of the benchmark.
Datasets
The paper introduces the ARABGEOTEXT dataset, which consists of 970 geographic locations across Riyadh and Jeddah, Saudi Arabia.
- Paired Views: Each location includes a standard OpenStreetMap (OSM) rendering and a composite (RS+OSM) where the map is overlaid on satellite imagery. This allows researchers to test how background clutter affects performance.
- Annotations: 17,875 manually annotated text instances per view type, including Arabic, Latin, and mixed-script labels.
- Splits: Riyadh data is used for training and "in-city" testing (Test-I), while Jeddah is reserved for "city-disjoint" testing (Test-II) to evaluate geographic generalization.
Key Insights and Ideas
- The "Full-Image" Bottleneck: VLMs struggle to attend to dozens of tiny, rotated text instances simultaneously. By delegating the "where" to YOLO and the "what" to the VLM, the system achieves much higher precision.
- Rectification Matters: Feeding the VLM a "straightened" and upscaled crop (as seen in Figure 4) significantly lowers the character error rate (CER) compared to asking the model to read rotated text in a cluttered scene.
- Localization Interference: Interestingly, the authors found that asking a VLM to perform joint transcription and localization actually degrades its localization accuracy compared to a VLM trained only to detect regions.
- Background Robustness: The tool-augmented pipeline showed remarkably consistent performance across both clean OSM maps and noisy RS+OSM composites, suggesting that specialized detectors are better at filtering out satellite background noise than general-purpose VLMs.
Key Results
The quantitative gap between the direct and tool-augmented approaches is stark:
- Detection: YOLOv10 achieved an H-mean (a balance of precision and recall) of 88.2% on Riyadh OSM, while the direct VLM managed only 34.5%.
- End-to-End Spotting: The tool-augmented pipeline with a LoRA-adapted reader reached an E2E H-mean of 74.4% (Riyadh OSM). The direct VLM approach lagged far behind at 12.8%.
- Generalization: When moving to the unseen city of Jeddah (Test-II), performance dropped (E2E H-mean of 57.3%), highlighting the difficulty of handling new vocabularies and smaller text sizes in different urban layouts.
Limitations and Future Work
While the tool-augmented approach is superior in accuracy, it is computationally heavier, requiring multiple VLM calls (one per detected text crop). The authors noted that using a smaller 0.8B parameter model for the reader can speed up inference by 1.7x, though with a slight hit to accuracy.
Future work aims to improve efficiency and explore better ways to generalize to new cities where the vocabulary and map styles might differ significantly from the training data. The authors also suggest expanding the benchmark to include more diverse geographic regions.
Summary updated