ARIDA - Active railway infrastructure dataset annotation using neural network predictions

Maximilian Dornik, Christian Sieberichs, Simon Geerkens, Alexander BraunarXiv 2026 · arXiv

TL;DR

ARIDA is a semi-automated framework that drastically reduces the manual labor required to annotate complex railway infrastructure in images. By combining instance segmentation with a novel "no-ground-truth" quality gate and an active learning loop, it slashes human labeling effort by up to 93% while maintaining the high precision needed for autonomous rail systems.

What Problem Does This Paper Address?

Training robust computer vision models for autonomous trains requires massive, high-quality datasets. However, railway infrastructure is notoriously difficult to label. Unlike standard bounding boxes for cars or pedestrians, tracks require complex geometric primitives like polyline-pairs to represent rail geometry accurately.

Current general-purpose tools (like CVAT or LabelMe) aren't tailored for these railway-specific needs, making manual annotation prohibitively expensive and slow. Furthermore, simply "assisting" a human labeler isn't enough; researchers need a way to prioritize which images actually need human intervention to maximize model performance gains per hour of work.

Methods

The authors propose the Active Railway Infrastructure Dataset Annotation (ARIDA) framework. It consists of three core components:

  1. The Backbone: It uses a Mask Scoring R-CNN (MS R-CNN). Unlike standard segmentation models, this network explicitly predicts a "maskIoU" score, giving the system a built-in estimate of how reliable its own mask predictions are.
  2. Annotation Module & Quality Gating: The system converts raw masks into structured primitives (bounding boxes, polygons, polylines, and polyline-pairs). To ensure these are reliable without human oversight, the authors introduced the Contour F Score (CFS). This metric assesses geometric consistency by comparing the predicted rail contour to the reconstructed polyline-pair. If the CFS or the model's confidence falls below a threshold, the image is flagged for human review.
  3. Active Learning (AL) Loop: ARIDA uses Triplets based Active Learning (TBAL) to rank unlabeled images by uncertainty. A novel Query Quantity Parameterization (QQP) mechanism dynamically adjusts the human workload: "easy" images are auto-accepted, while human effort is strictly targeted at "hard" cases that provide the most learning value.
Simplified overview of the ARIDA-framework architecture.

Datasets

The framework was evaluated on two distinct data regimes:

  • RailSem19: A diverse, "disjoint" dataset where viewpoints and settings vary significantly. This tests the model's ability to generalize across different environments.
  • OSDaR23: A "sequential" dataset consisting of redundant imagery from a fixed camera setup (like a train's forward-facing camera). This highlights the risk of over-annotation and tests how well the system handles redundant information.

Key Insights and Ideas

  • The "Competence Threshold": In diverse datasets (RailSem19), Active Learning is most effective only after the model crosses a certain stability threshold (around 100 initial manual labels). Before this, the model's uncertainty estimates aren't reliable enough to guide the process effectively.
  • AL as a Safeguard Against Over-Annotation: In sequential data (OSDaR23), many frames look nearly identical. ARIDA identifies when the model has "learned enough" from a specific scene, effectively acting as a stopping mechanism that prevents humans from wasting time labeling redundant frames.
  • No-Ground-Truth Reliability: The CFS metric allows the system to be "self-aware." By checking if its own geometric reconstructions match its initial mask predictions, it can filter out low-quality auto-annotations without needing a human to check every single one.
Schematic of the polyline-pair annotation algorithm for railway tracks.

Key Results

  • Effort Reduction: ARIDA achieved a 67% reduction in manual labeling effort on the disjoint RailSem19 dataset. On the sequential OSDaR23 dataset, the reduction was a staggering 93%.
  • Sample Efficiency: The 100-initial-label configuration on RailSem19 reached the same performance as a fully supervised model trained on 200 images, but with significantly less human intervention.
  • Qualitative Success: The system successfully handles complex track geometries, including curvature-following and per-instance rail splitting, as shown in the qualitative outputs.
Qualitative ARIDA output on a RailSem19 image, showing high-quality polyline-pair generation.

Limitations and Future Work

While the results are promising, the authors note several areas for further exploration:

  • Simulated Oracle: The "human" in these experiments was a simulated oracle using hidden ground truth. Real-world human-in-the-loop testing with actual timing measurements is the next logical step.
  • Single-Class Focus: The evaluation focused primarily on the "rail/track" class. Future work needs to extend this to multi-class workloads (e.g., signals, switches, platforms).
  • Hyperparameter Sensitivity: The performance is sensitive to acceptance thresholds and QQP settings, which currently require empirical tuning.
  • Temporal Cues: For sequential data, incorporating temporal information (tracking objects across frames) could further improve selection strategies and annotation consistency.

Summary updated

Abstract

Modern railway perception requires large comprehensive image datasets, but creating them is slowed by manual labeling, especially for track polylines and other complex infrastructure geometry. This paper presents the Active Railway Infrastructure Dataset Annotation framework, a railway-specific semi-automated pipeline that converts instance predictions from a mask scoring instance segmentation network into bounding-boxes, polygons, polylines, and paired polylines. To keep automatic track geometry annotation reliable without ground truth, it applies confidence thresholds and an unique contour-agreement quality score as an acceptance gate. An active learning loop ranks remaining images by uncertainty and adapts the number of manual queries per cycle. This is achieved through a novel quantity control mechanism, in which easy images are auto-accepted and human effort targets hard cases. Experiments on disjoint, diverse imagery and time-sequential, redundant imagery quantify effort reduction and accuracy trade-offs, achieving about two thirds reduction in the disjoint regime and up to 93% in the sequential regime. The study also provides guidance for initial label budgets and stopping to avoid over-annotation.
BibTeX
@misc{dornik2026arida,
  author = {Maximilian Dornik and Christian Sieberichs and Simon Geerkens and Alexander Braun},
  title = {{ARIDA - Active railway infrastructure dataset annotation using neural network predictions}},
  year = {2026},
  eprint = {2611.05646},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05646},
}
Figure