RoadAnnoter: An AI-Assisted Desktop Tool for Semantic Segmentation Annotation of Road-Scene Images

Nazmul Haque, Arnob Protim RoyarXiv 2026 · arXiv

TL;DR

RoadAnnoter v1.0 is an open-source desktop application designed to slash the time required for high-fidelity semantic segmentation of road-scene images. By integrating six complementary annotation modes—ranging from classical edge-detection to the Segment Anything Model (SAM) and a self-training "TinySegNet"—it provides a flexible, offline-capable workflow for researchers building specialized transportation datasets.

What Problem Does This Paper Address?

Semantic segmentation—the process of labeling every pixel in an image (e.g., "this pixel is road," "this pixel is a car")—is the backbone of modern road safety research and autonomous driving. However, creating these datasets is a nightmare of manual labor. For context, the famous Cityscapes benchmark required roughly 1.5 hours of expert effort per image.

While tools like LabelMe or CVAT exist, they often lack deep, native integration with the latest AI models or require complex cloud setups. Furthermore, general-purpose AI models often fail to recognize specialized road features like rumble strips or specific types of roadside furniture common in developing regions. The authors tackle this "annotation bottleneck" by creating a unified, locally operated tool that adapts to the user’s specific project needs.

Methods

RoadAnnoter’s core strength lies in its six annotation modes, allowing users to pick the best tool for the specific geometry they are labeling:

  1. Manual Polygon: The "old school" way for complex, irregular shapes.
  2. SAM Prompt: Uses the Segment Anything Model (SAM). Users provide foreground/background points, and the AI generates a mask.
  3. Smart Select: A one-click variant of SAM that uses an adaptive bounding box to quickly grab large, continuous features like road surfaces.
  4. Edge Fill: A "no-AI" fallback for low-resource environments. It combines Canny edge detection with flood-filling in the CIELAB color space (which is better at mimicking human color perception than standard RGB).
  5. Pretrained SegFormer: Uses models trained on Cityscapes to provide "batch suggestions" for common classes (sky, road, sidewalk) in a single pass.
  6. TinySegNet: Perhaps the most novel feature. This is a lightweight MobileNetV3-Small network that can be trained inside the app using the labels the user has already created. As you annotate more, the tool gets better at suggesting labels for your specific dataset.

All modes feed into a shared mask-to-polygon pipeline. This pipeline uses morphological operations to clean up noise, Gaussian smoothing to prevent "staircase" edges, and the Douglas–Peucker algorithm to simplify the final polygon into a manageable number of vertices.

Datasets

The authors demonstrated the tool using a new collection of Google Street View (GSV) images captured at road accident locations across Bangladesh.

To power the underlying AI, the tool leverages:

  • Cityscapes: For the pretrained SegFormer models.
  • SA-1B: The massive dataset used to train the Segment Anything Model.
  • ImageNet: Used for pretraining the TinySegNet backbone.

Key Insights and Ideas

  • Geometry-Specific Tooling: One of the "aha" moments is that different AI models suit different shapes. SAM is excellent for distinct objects (cars, signs), but "Smart Select" is better for horizontally elongated features like lane markings or curbs.
  • The "Evolving" Annotator: By including TinySegNet, the tool moves from being a static interface to an active participant. It allows the annotation process to evolve alongside the dataset, potentially reducing domain-shift errors where a model trained in Germany (Cityscapes) might struggle with the road conditions in Bangladesh.
  • Local & Atomic: The tool is designed to work offline and uses an "atomic file-replacement" procedure. This ensures that if the app crashes or the power goes out, your annotation files don't end up corrupted—a small but vital detail for researchers in the field.

Key Results

While the paper focuses on the tool's architecture rather than a quantitative "speed" race, the qualitative results are promising:

  • Versatility: The tool successfully handled diverse geometries, from vertically oriented utility poles to complex grade-separated interchanges.
  • YOLO Compatibility: Annotations are exported directly in the YOLO segmentation format, making them immediately ready for training state-of-the-art object detection and segmentation models.
  • Benchmarking: A visual comparison with Roboflow showed that RoadAnnoter produced similar, and in some cases smoother, boundary characteristics for principal road regions.

Limitations and Future Work

The authors are transparent about several areas for improvement:

  • The "Hole" Problem: The current mask-to-polygon pipeline only extracts the external contour. This means it cannot currently handle "donut" shapes (e.g., a pothole inside a road or the sky visible through a bridge arch).
  • Quantitative Gaps: The study lacks a formal measurement of exactly how many minutes are saved per image compared to traditional tools.
  • Validation: TinySegNet currently evaluates itself on the training data; future versions need a proper "held-out" validation set to better estimate how well the model is actually learning.
  • Regional Fine-Tuning: The authors suggest that fine-tuning models specifically on South Asian road imagery would further improve the "out-of-the-box" suggestions for that region.

Source Code: GitHub - RoadAnnoter

Summary updated

Abstract

Accurate semantic segmentation of road-scene images is essential for road safety research; however, polygon-level annotation remains labor-intensive and limits labelled dataset development. This study presents RoadAnnoter v1.0, an open-source desktop application integrating six human-in-the-loop annotation modes: manual polygon delineation, SAM Prompt, Smart Select, Edge Fill, pretrained semantic segmentation using Cityscapes-trained SegFormer models, and task-specific segmentation using TinySegNet, a lightweight MobileNetV3-Small-based network trained with accumulated annotator-generated labels. SAM-based modes use promptable segmentation with SAM and MobileSAM, while Edge Fill combines Canny edge detection with colour-similarity-based flood filling in CIELAB colour space. SegFormer generates suggestions for common road-scene categories, whereas TinySegNet supports adaptation to project-specific features through iterative training. All six modes use a shared mask-to-polygon pipeline comprising morphological processing, Gaussian smoothing, contour extraction, and Douglas-Peucker simplification; annotations are exported in YOLO segmentation format. RoadAnnoter was used to annotate Google Street View images from recorded road accident locations in Bangladesh for a vision-language model-based investigation of free-flow speed estimation. Qualitative assessment across representative scenes demonstrated the complementary applicability of the annotation modes to diverse feature geometries. Visual comparison with Roboflow indicated similar qualitative boundary characteristics for principal road-scene regions, although no quantitative accuracy assessment was conducted. RoadAnnoter provides a locally operated framework for developing road-scene segmentation datasets for transportation engineering and road safety research.
BibTeX
@misc{haque2026roadannoter,
  author = {Nazmul Haque and Arnob Protim Roy},
  title = {{RoadAnnoter: An AI-Assisted Desktop Tool for Semantic Segmentation Annotation of Road-Scene Images}},
  year = {2026},
  eprint = {2611.22884},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22884},
}
Figure