From Pixels to Actions: Learning to Drive a Car with Deep Neural Networks

Jonas Heylen, Seppe Iven, Bert De Brabandere, José Oramas M., Luc Van Gool, Tinne Tuytelaars

Abstract

The promise of self-driving cars promotes several advantages, e.g. they have the ability to outperform human drivers while being safer. Here we take a deeper look into some aspects from algorithms aimed at making this promise a reality. More specifically, we analyze an end-to-end neural network to predict a car's steering actions on a highway based on images taken from a single car-mounted camera. We focus our analysis on several aspects which could have a significant impact on the performance of the system. These aspects are: the input data format, the temporal dependencies between consecutive inputs, and the origin of the data. We show that, for the task at hand, regression networks outperform their classifier counterparts. In addition, there seems to be a small difference between networks that use coloured images and ones that use grayscale images as input. For the second aspect, by feeding the network three concatenated images, we get a significant decrease of 30% in mean squared error. For the third aspect, by using simulation data we are able to train networks that have a performance comparable to networks trained on real-life datasets. We also qualitatively demonstrate that the standard metrics that are used to evaluate networks do not necessarily accurately reflect a system's driving behaviour. We show that a promising confusion matrix may result in poor driving behaviour while a very ill-looking confusion matrix may result in good driving behaviour.
BibTeX
@inproceedings{heylen2018from,
  author = {Jonas Heylen and Seppe Iven and Bert De Brabandere and José Oramas M. and Luc Van Gool and Tinne Tuytelaars},
  title = {{From Pixels to Actions: Learning to Drive a Car with Deep Neural Networks}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  doi = {10.1109/wacv.2018.00072},
}

Loading related content...

Figure