Object Recognition from very few Training Examples for Enhancing Bicycle Maps

Christoph Reinders, Hanno Ackermann, Michael Ying Yang, Bodo Rosenhahn

Abstract

In recent years, data-driven methods have shown great success for extracting information about the infrastructure in urban areas. These algorithms are usually trained on large datasets consisting of thousands or millions of labeled training examples. While large datasets have been published regarding cars, for cyclists very few labeled data is available although appearance, point of view, and positioning of even relevant objects differ. Unfortunately, labeling data is costly and requires a huge amount of work. In this paper, we thus address the problem of learning with very few labels. The aim is to recognize particular traffic signs in crowdsourced data to collect information which is of interest to cyclists. We propose a system for object recognition that is trained with only 15 examples per class on average. To achieve this, we combine the advantages of convolutional neural networks and random forests to learn a patch-wise classifier. In the next step, we map the random forest to a neural network and transform the classifier to a fully convolutional network. Thereby, the processing of full images is significantly accelerated and bounding boxes can be predicted. Finally, we integrate data of the Global Positioning System (GPS) to localize the predictions on the map. In comparison to Faster R-CNN and other networks for object recognition or algorithms for transfer learning, we considerably reduce the required amount of labeled data. We demonstrate good performance on the recognition of traffic signs for cyclists as well as their localization in maps.

Introduction

While autonomous vehicle research has produced massive labeled datasets like KITTI or Cityscapes, cycling infrastructure remains under-represented. Cyclists face unique challenges, including different viewpoints, specific traffic signs like "bicycle boulevards," and routes prohibited to cars. Collecting raw image data via crowdsourcing is feasible, but labeling thousands of images is prohibitively expensive. Standard deep learning models typically require thousands of examples to generalize. This paper proposes a system to recognize niche signs using extremely little data—averaging just 15 training examples per class—to enhance digital bicycle maps.

Methodology

The proposed multi-stage hybrid approach bridges classical machine learning and deep learning to maximize performance on small datasets.

Hybrid Classifier Pipeline

  1. Feature Learning: A CNN is pre-trained on the large German Traffic Sign Recognition Benchmark (GTSRB) dataset to learn general traffic sign representations.
  2. Random Forest (RF) Classification: The pre-trained CNN acts as a feature extractor. These features are fed into a Random Forest, which is significantly more robust than deep neural networks when training data is scarce.
  3. RF-to-NN Mapping: To enable efficient full-image inference, the trained RF is mapped into a two-hidden-layer neural network. The first layer evaluates split rules, and the second determines leaf membership.
  4. Fully Convolutional Transformation: The entire pipeline is converted into a Fully Convolutional Network (FCN). This allows the model to process images of any size and produce a probability map in a single forward pass, rather than using a computationally expensive sliding window.

Optimization Strategies

  • Part-Based Boosting: The system identifies specific "parts" of signs (e.g., a bicycle symbol). If a part is detected within a larger bounding box, the detection score is boosted.
  • Adaptive Filtering: To manage crowdsourced data volume, a filter retains only images where the cyclist is accelerating or decelerating, reducing data by 80% before processing.
  • Localization: GPS and heading data from smartphone sensors are used to project detected signs onto a global map using the haversine formula and pinhole camera geometry.

Experimental Setup

The system was evaluated using:

  • GTSRB: For initial feature learning (39,209 examples).
  • Crowdsourced Cyclist Dataset: Over 40,000 images with only 297 manually annotated bounding boxes across 10 classes (averaging ~15-30 samples per class).

Results and Discussion

The RF classifier achieved 99.3% accuracy on image patches. The overall system reached a mean Average Precision (mAP) of 0.686. Notably, for the "Bicycle Road" class, the AP was as high as 0.994.

Examples of detected traffic signs and their projected locations on a map.

The transformation to an FCN significantly improved inference speed, reducing processing time from over 10 hours per image (patch-based) to just 6.08 seconds for 8 different scales. The median localization error was 6.76 meters, which is sufficient for map metadata enhancement in services like OpenStreetMap.

Conclusion

By combining CNN features with Random Forests and mapping them back to an FCN architecture, this work demonstrates that high-performance object detection is possible with very few training examples. This approach effectively addresses the data scarcity in cycling infrastructure mapping and provides a scalable solution for enhancing bicycle maps.

Summary updated

Abstract

In recent years, data-driven methods have shown great success for extracting information about the infrastructure in urban areas. These algorithms are usually trained on large datasets consisting of thousands or millions of labeled training examples. While large datasets have been published regarding cars, for cyclists very few labeled data is available although appearance, point of view, and positioning of even relevant objects differ. Unfortunately, labeling data is costly and requires a huge amount of work. In this paper, we thus address the problem of learning with very few labels. The aim is to recognize particular traffic signs in crowdsourced data to collect information which is of interest to cyclists. We propose a system for object recognition that is trained with only 15 examples per class on average. To achieve this, we combine the advantages of convolutional neural networks and random forests to learn a patch-wise classifier. In the next step, we map the random forest to a neural network and transform the classifier to a fully convolutional network. Thereby, the processing of full images is significantly accelerated and bounding boxes can be predicted. Finally, we integrate data of the Global Positioning System (GPS) to localize the predictions on the map. In comparison to Faster R-CNN and other networks for object recognition or algorithms for transfer learning, we considerably reduce the required amount of labeled data. We demonstrate good performance on the recognition of traffic signs for cyclists as well as their localization in maps.
BibTeX
@inproceedings{reinders2018object,
  author = {Christoph Reinders and Hanno Ackermann and Michael Ying Yang and Bodo Rosenhahn},
  title = {{Object Recognition from very few Training Examples for Enhancing Bicycle Maps}},
  booktitle = {Intelligent Vehicles Symposium},
  year = {2018},
  doi = {10.1109/ivs.2018.8500469},
}
Figure