Courteous Autonomous Cars

Liting Sun, Wei Zhan, Masayoshi Tomizuka, Anca D. Dragan

Abstract

Autonomous vehicles (AVs) are transitioning from conservative to aggressive game-theoretic planning, often at the expense of human drivers' convenience. This paper formalizes "courtesy" as a mathematical objective: minimizing the additional cost or inconvenience an AV imposes on other road users. By integrating a courtesy term into motion planning and using Inverse Reinforcement Learning (IRL) on real-world datasets, the authors demonstrate that courtesy-aware models better predict human behavior and improve social coordination on the road.

Introduction

Current AV motion planning typically optimizes for "selfish" objectives such as safety, efficiency, and ego-vehicle comfort. While early AVs were criticized for being overly timid, newer models using game-theoretic approaches can become aggressive—cutting off human drivers or forcing right-of-way at intersections. Such behaviors, while efficient for the robot, impose significant "costs" on humans in the form of sudden braking or path deviations. This paper addresses the challenge of formalizing "courtesy" into a machine-readable objective function to balance robot goals with human needs.

Methodology

The researchers introduce a courtesy term ($C_{court}$) into the robot’s cost function: $$C_{Robot} = C_{selfish} + \lambda_c C_{court}$$

$C_{court}$ is defined as the difference between the cost a human driver actually incurs due to the robot's presence and the cost they would have incurred in a "best-case" alternative scenario. Three baseline "alternative worlds" were explored:

  1. The Ghost World: The robot is absent.
  2. The Altruistic World: The robot acts solely to minimize human cost.
  3. The Status Quo World: The robot maintains its current velocity.

By minimizing this difference, the robot naturally avoids actions that force humans to deviate significantly from their preferred trajectories. The weights for these features were learned from real-world data using Inverse Reinforcement Learning (IRL).

Key Results and Insights

The courtesy-aware model significantly outperformed selfish baselines in reproducing real human trajectories from the NGSIM dataset, achieving a lower Mean Euclidean Distance (MED). Qualitative simulations showed that while a selfish robot might force a human to brake hard, a courteous robot adjusts its behavior to maintain the human's constant speed.

Comparison of robot behaviors ranging from selfish to most courteous, showing the reduction in human driver inconvenience as courtesy is prioritized.

A key insight is that courtesy acts as a form of loss aversion. Drawing on Prospect Theory, the authors argue that humans weigh inconvenience (losses) more heavily than gains. A courteous robot focuses on not making the human's drive worse rather than actively trying to improve it. In complex "blocking" scenarios, this leads to sophisticated social behaviors, such as the robot blocking an aggressive third party to protect the human driver.

Discussion and Limitations

The weight of courtesy ($\lambda_c$) is a critical meta-parameter. As shown in the relationship between weight and inconvenience, increasing $\lambda_c$ leads to a steady decrease in human inconvenience.

The relationship between the courtesy weight parameter and the resulting human driver's inconvenience.

However, setting it too high risks making the robot a "doormat" that fails to reach its own destination. Future work must address multi-agent scaling, as calculating "alternative world" costs for many actors simultaneously is computationally intensive. Additionally, while IRL allows robots to mimic human courtesy, there is potential to set courtesy levels higher than the human average to foster a safer and more pleasant driving environment.

Summary updated

Abstract

Typically, autonomous cars optimize for a combination of safety, efficiency, and driving quality. But as we get better at this optimization, we start seeing behavior go from too conservative to too aggressive. The car's behavior exposes the incentives we provide in its cost function. In this work, we argue for cars that are not optimizing a purely selfish cost, but also try to be courteous to other interactive drivers. We formalize courtesy as a term in the objective that measures the increase in another driver's cost induced by the autonomous car's behavior. Such a courtesy term enables the robot car to be aware of possible irrationality of the human behavior, and plan accordingly. We analyze the effect of courtesy in a variety of scenarios. We find, for example, that courteous robot cars leave more space when merging in front of a human driver. Moreover, we find that such a courtesy term can help explain real human driver behavior on the NGSIM dataset.
BibTeX
@inproceedings{sun2018courteous,
  author = {Liting Sun and Wei Zhan and Masayoshi Tomizuka and Anca D. Dragan},
  title = {{Courteous Autonomous Cars}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=F2ysaA0Led},
}
Figure