Can LLMs Understand Social Norms in Autonomous Driving Games?

Boxuan Wang, Haonan Duan, Yanhao Feng, Xu Chen, Yongjie Fu, Zhaobin Mo, Xuan DiOpenReview 2024 · DOI · OpenReview

TL;DR

This paper investigates whether Large Language Models (LLMs), specifically GPT-4.0, can understand and exhibit social norms when acting as agents in autonomous driving simulations. By framing driving scenarios as Markov games and using LLMs to generate agent decisions based on textual prompts, the researchers found that LLM agents can indeed develop and adhere to emergent social norms, such as yielding at intersections and forming platoons on highways.

What Problem Does This Paper Address?

The safe and efficient deployment of autonomous vehicles (AVs) hinges on their ability to navigate complex social environments, which are governed by unwritten rules or "social norms." While AVs can be programmed with explicit rules, understanding and adapting to emergent social behaviors is crucial for seamless integration with human drivers and other road users. Traditional methods like Reinforcement Learning (RL) can be complex to design and analyze for social norm emergence. This paper explores a novel approach: using LLMs as agents to see if they can spontaneously develop and follow social norms in simulated driving scenarios, offering a potentially more interpretable and flexible way to model these behaviors.

Methods

The core of the methodology involves setting up Markov games within a multi-agent system (MAS). In these games, LLMs act as intelligent agents. The LLMs are prompted with textual descriptions of the game's environment, rules, and their current observations (e.g., car locations, rewards). Based on these prompts, the LLM agent decides on an action (e.g., "Go," "Stop," "Lane Change"). The game state is then updated, and the process repeats.

Key aspects of their approach include:

  • LLM Agents: LLMs (specifically GPT-4.0 via the OpenAI Chat API) are used to generate agent decisions. This is a departure from traditional RL agents.
  • Prompt Engineering: Carefully crafted "system" and "user" prompts are used to define the game's general settings, the agent's perspective, and its observations. The "system" prompt sets up the game rules, while the "user" prompt provides the agent with its specific context and asks for its next action.
  • Turn-Based Simulation: To prevent LLM agents from having access to concurrent decisions of other agents (which would break the social norm emergence aspect), the simulation proceeds turn-by-turn. The game state is updated only after all agents have made their decisions for a given time step.
  • Two Scenarios: The framework is tested on two distinct driving scenarios: an unsignalized intersection and a highway platoon. These scenarios are designed to elicit specific social norms.

Datasets

The paper does not use pre-existing datasets for training LLMs. Instead, it generates its own simulation data by running LLM agents in custom-designed Markov game environments. The "datasets" are the logs of these simulations, recording agent decisions, rewards, and states over multiple runs (50 simulations per scenario).

Key Insights and Ideas

  • LLMs as Socially Aware Agents: The most significant insight is that LLMs, when prompted appropriately, can indeed exhibit emergent social behaviors that resemble social norms in simulated driving environments. This suggests LLMs possess a latent understanding of cooperative or rule-following behavior.
  • Conservative Driving at Intersections: In the unsignalized intersection scenario, LLM agents tend to adopt a conservative driving policy, particularly the "red car," which frequently exhibits "early stopping" behavior to avoid potential crashes. This highlights a learned norm of prioritizing safety.
  • Platoon Formation on Highways: In the highway platoon scenario, LLM agents demonstrate a tendency to form platoons, indicating a learned norm of efficient, coordinated driving for potential reward benefits.
  • Operability and Analyzability: The paper emphasizes that using LLMs as agents offers strong operability and analyzability, making it easier to design experiments and interpret the emergence of social norms compared to more opaque RL methods.

Key Results

  • Unsignalized Intersection: In simulations without background vehicles, the "red car" achieved a high success rate (0.96) in adhering to a "yielding" social norm, with an average of 1.42 early stops. When background vehicles were introduced, the success rate of adhering to this norm increased to 1.0, suggesting that the presence of other agents can reinforce social norms.
  • Highway Platoon: The success rate of forming platoons was consistently high across different numbers of background vehicles, ranging from 0.80 to 0.98. This indicates a strong emergence of the platoon formation norm. The study also observed that agents tend to change lanes early in the simulation to facilitate platoon formation.

Limitations and Future Work

  • Simulated Environment: The experiments are conducted in simplified grid-world simulations, which may not fully capture the complexity and nuances of real-world driving.
  • Single LLM Model: The study primarily uses GPT-4.0. The behavior might differ with other LLMs or different versions.
  • Limited Context: Agents are designed to not have access to historical context beyond the current observation, which simplifies the problem but might limit the depth of learned social reasoning.
  • Future Work: The authors propose extending the research to more complex real-world scenarios, developing a unified framework for simulating strategic interactions, and comparing LLM agent behavior with human players to assess the extent of strategic reasoning.

Summary updated

Abstract

Social norm is defined as a shared standard of acceptable behavior in a society. The emergence of social norms fosters coordination among agents without any hard-coded rules, which is crucial for the large-scale deployment of AVs in an intelligent transportation system. This paper explores the application of LLMs in understanding and modeling social norms in autonomous driving games. We introduce LLMs into autonomous driving games as intelligent agents who make decisions according to text prompts. These agents are referred to as LLM-based agents. Our framework involves LLM-based agents playing Markov games in a multi-agent system (MAS), allowing us to investigate the emergence of social norms among individual agents. We aim to identify social norms by designing prompts and utilizing LLMs on textual information related to the environment setup and the observations of LLM-based agents. Using the OpenAI Chat API powered by GPT-4.0, we conduct experiments to simulate interactions and evaluate the performance of LLM-based agents in two driving scenarios: unsignalized intersection and highway platoon. The results show that LLM-based agents can handle dynamically changing environments in Markov games, and social norms evolve among LLM-based agents in both scenarios. In the intersection game, LLM-based agents tend to adopt a conservative driving policy when facing a potential car crash. The advantage of LLM-based agents in games lies in their strong operability and analyzability, which facilitate experimental design.
BibTeX
@misc{wang2024can,
  author = {Boxuan Wang and Haonan Duan and Yanhao Feng and Xu Chen and Yongjie Fu and Zhaobin Mo and Xuan Di},
  title = {{Can LLMs Understand Social Norms in Autonomous Driving Games?}},
  year = {2024},
  eprint = {2408.12680},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.12680},
  howpublished = {OpenReview},
}
Figure