EyeCar: Modeling the Visual Attention Allocation of Drivers in Semi-Autonomous Vehicles

Sonia Baee, Erfan Pakdamanian, Vicente Ordonez, Inki Kim, Lu Feng, Laura E. Barnes

Abstract

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmarks: (DR(eye)VE, BDD-A, DADA-2000), and our EyeCar dataset. Results indicate that MEDIRL outperforms existing models for predicting attention and achieves state-of-the-art performance. We present extensive ablation studies to provide more insights into different features of our proposed model.
BibTeX
@inproceedings{baee2019eyecar,
  author = {Sonia Baee and Erfan Pakdamanian and Vicente Ordonez and Inki Kim and Lu Feng and Laura E. Barnes},
  title = {{EyeCar: Modeling the Visual Attention Allocation of Drivers in Semi-Autonomous Vehicles}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.48550/arxiv.1912.07773},
}

Loading related content...

Figure