Abstract
With deep learning approaches now out-performing traditional image processing techniques for image understanding, this paper accesses the potential of rapid generation of Convolutional Neural Networks (CNNs) for applied engineering purposes. Three CNNs are trained on 275 UAS-derived and freely available online images for object detection of 3m² segments of railway track. These includes two models based on the Faster RCNN object detection algorithm (Resnet and Incpetion-Resnet) as well as the novel one-stage Focal Loss network architecture (Retinanet). Model performance was assessed with respect to three accuracy metrics. The first two consisted of Intersection over Union (IoU) with thresholds 0.5 and 0.1. The last assesses accuracy based on the proportion of track covered by object detection proposals against total track length. In under six hours of training (and two hours of manual labelling) the models detected 91.3 %, 83.1 % and 75.6 % of track in the 500 test images acquired from the UAS survey Retinanet, Resnet and Inception-Resnet respectively. We then discuss the potential for such applications of such systems within the engineering field for a range of scenarios.
Introduction
Civil engineering infrastructure—such as roads, tunnels, and railway tracks—requires constant monitoring for defects and maintenance. Traditionally, this is a manual, time-consuming, and often dangerous task. While computer vision (CV) offers a solution, traditional CV techniques often struggle with the "inhomogeneity" and complexity of real-world environments.
Deep learning, specifically Convolutional Neural Networks (CNNs), has revolutionized image understanding, but a common perception remains that these systems require massive datasets and weeks of high-end computation to be effective. This paper addresses the gap between cutting-edge AI research and practical engineering application, proving that "rapid" deployment of site-specific models is possible with limited data and modest hardware.
Methodology
The authors established a workflow using Google’s TensorFlow framework to train and compare three distinct CNN architectures: Faster R-CNN with ResNet, Faster R-CNN with Inception-ResNet, and RetinaNet. The latter is a "one-stage" detector utilizing a novel Focal Loss function to address the extreme class imbalance common in aerial imagery.
The team utilized Transfer Learning, starting with models pre-trained on the ImageNet dataset. The study used a targeted dataset of 275 images: 150 acquired via a DJI Inspire 1 UAS and 125 sourced from the internet. Data augmentation tripled the dataset size, and ground-truth bounding boxes were created manually using LabelImg.
Key Insights: The Metric Mismatch
A significant finding was that the standard industry metric for object detection—Intersection over Union (IoU) at 0.5—is poorly suited for continuous linear features like railway tracks. Because tracks are homogeneous, a predicted box might be perfectly on-track but slightly shifted from the arbitrary ground-truth box, resulting in a "failure" at 0.5. To solve this, the authors proposed a "Track Proposal Coverage" metric, measuring the actual length of track covered by detections, which proved much more representative of real-world utility.
Results and Performance
The quantitative comparison highlighted RetinaNet as the clear winner for rapid deployment, being 11.7x faster than the Inception-ResNet model while providing significantly better coverage.
| Metric | ResNet (Faster R-CNN) | Inception-ResNet | RetinaNet |
|---|---|---|---|
| Track Coverage (%) | 83.1% | 75.6% | 91.3% |
| Accuracy @ IoU=0.1 (%) | 87.9% | 72.4% | 93.0% |
| Inference Time (sec) | 0.64s | 1.29s | 0.11s |
As shown in the performance graph, lowering the IoU threshold to 0.1 significantly improves the reported accuracy across all models, aligning more closely with the visual success of the detections on the linear infrastructure.
Limitations and Future Work
While the models showed signs of overfitting, this is considered less critical for site-specific "rapid" systems where the environment is consistent. Future work aims to move from bounding boxes to semantic segmentation for pixel-perfect identification of defects like cracks. Additionally, high-speed detectors like RetinaNet enable "intelligent flight control," where drones can navigate along tracks autonomously while generating real-time GIS data.
Summary updated