Effortless Deep Training for Traffic Sign Detection Using Templates and Arbitrary Natural Images

Lucas Tabelini Torres, Thiago M. Paixão, Rodrigo Ferreira Berriel, Alberto F. De Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-SantosIJCNN 2019 · DOI · OpenReview

Abstract

Deep learning has revolutionized autonomous driving, yet its reliance on large, manually annotated real-world datasets remains a significant bottleneck. Traffic sign detection specifically suffers from expensive annotation, severe data imbalance, and the perceived need for domain-specific backgrounds. This paper proposes a novel database generation method that utilizes only arbitrary natural images (e.g., from MS COCO) and synthetic traffic sign templates. By blending these templates into non-driving backgrounds with appropriate augmentations, we train a Faster R-CNN detector that achieves a 95.66% mAP on the German Traffic Sign Detection Benchmark (GTSDB). Our results demonstrate that detectors can be effectively trained without any problem-domain data for backgrounds, challenging conventional wisdom in deep learning.

Introduction

The development of robust traffic sign detection systems is critical for autonomous vehicle safety. However, the traditional supervised learning paradigm requires thousands of real-world images captured from road environments, each requiring precise bounding box annotations. This process is not only costly and time-consuming but also fails to address the inherent scarcity of certain sign classes. Furthermore, the prevailing assumption in computer vision is that training data must closely mirror the target domain's background (i.e., roads and highways). This work explores whether "domain-agnostic" backgrounds can serve as a viable alternative for training high-performance detectors.

Methodology: Synthetic Data Generation

The proposed pipeline eliminates the need for real-world training images by synthesizing a dataset from two disparate sources: standardized traffic sign templates and arbitrary natural images.

Overview of the proposed training database generation and inference pipeline, illustrating how templates and natural images are combined to train a Faster R-CNN model.

The process involves three main stages:

  1. Template and Background Acquisition: Standardized graphic templates of traffic signs are collected. For backgrounds, the MS COCO dataset is used, specifically excluding categories related to driving (like cars or traffic lights) to ensure the model does not learn from the target domain during training.
  2. Blending and Augmentation: To bridge the gap between synthetic templates and natural backgrounds, signs are subjected to random geometric transformations, brightness/contrast adjustments, and local adaptation to match the lighting of the background patch.
  3. Smoothing: Gaussian noise and border fading are applied to prevent the network from over-fitting to the sharp edges typical of simple "copy-paste" operations.

Key Insights: Background Independence

A central finding of this research is that the specific context of the background is secondary to the appearance of the object of interest. By placing traffic signs in surreal or unrelated contexts—such as a baseball field or a kitchen—the model is forced to learn the invariant features of the signs themselves rather than relying on contextual cues like the presence of a road or a sky.

Example of a synthetic training image where traffic sign templates are blended into an arbitrary natural background (a baseball field) to force the model to focus on sign features.

This approach also naturally solves the data imbalance problem. Since the data is generated programmatically, researchers can ensure an equal distribution of all sign classes, including those rarely seen in the real world. Additionally, because the system knows the exact coordinates of the pasted templates, annotations are generated automatically with perfect precision.

Experimental Results

The effectiveness of the "effortless" training method was evaluated using the German Traffic Sign Detection Benchmark (GTSDB). Despite never seeing a real road during training, the model achieved:

  • Mean Average Precision (mAP): 95.66%.
  • F1-Score: 93%.
  • Precision and Recall: 94% and 91%, respectively.

The performance gap between this synthetic-only model and a baseline trained on real-world GTSDB data was a mere 3.39%, proving that domain-specific backgrounds are not a prerequisite for high-accuracy detection.

Conclusion

This work demonstrates a significant shift in how we can approach training data for specialized detection tasks. By leveraging arbitrary natural images and simple templates, we can bypass the most expensive and restrictive parts of the deep learning pipeline. Future work will focus on integrating these detection capabilities with classification and exploring the model's performance under more extreme environmental conditions.

Summary updated

Abstract

Deep learning has been successfully applied to several problems related to autonomous driving. Often, these solutions rely on large networks that require databases of real image samples of the problem (i.e., real world) for proper training. The acquisition of such real-world data sets is not always possible in the autonomous driving context, and sometimes their annotation is not feasible (e.g., takes too long or is too expensive). Moreover, in many tasks, there is an intrinsic data imbalance that most learning-based methods struggle to cope with. It turns out that traffic sign detection is a problem in which these three issues are seen altogether. In this work, we propose a novel database generation method that requires only (i) arbitrary natural images, i.e., requires no real image from the domain of interest, and (ii) templates of the traffic signs, i.e., templates synthetically created to illustrate the appearance of the category of a traffic sign. The effortlessly generated training database is shown to be effective for the training of a deep detector (such as Faster R-CNN) on German traffic signs, achieving 95.66% of mAP on average. In addition, the proposed method is able to detect traffic signs with an average precision, recall and F1-score of about 94%, 91% and 93%, respectively. The experiments surprisingly show that detectors can be trained with simple data generation methods and without problem domain data for the background, which is in the opposite direction of the common sense for deep learning.
BibTeX
@inproceedings{torres2019effortless,
  author = {Lucas Tabelini Torres and Thiago M. Paixão and Rodrigo Ferreira Berriel and Alberto F. De Souza and Claudine Badue and Nicu Sebe and Thiago Oliveira-Santos},
  title = {{Effortless Deep Training for Traffic Sign Detection Using Templates and Arbitrary Natural Images}},
  booktitle = {IJCNN},
  year = {2019},
  doi = {10.1109/ijcnn.2019.8852086},
}
Figure