Abstract
Recently, LLM-powered driver agents have demonstrated considerable potential in the field of autonomous driving, showcasing human-like reasoning and decision-making abilities. However, current research on aligning driver agent behaviors with human driving styles remains limited, partly due to the scarcity of high-quality natural language data from human driving behaviors. To address this research gap, we propose a multi-alignment framework designed to align driver agents with human driving styles through demonstrations and feedback. Notably, we construct a natural language dataset of human driver behaviors through naturalistic driving experiments and post-driving interviews, offering high-quality human demonstrations for LLM alignment. The framework's effectiveness is validated through simulation experiments in the CARLA urban traffic simulator and further corroborated by human evaluations. Our research offers valuable insights into designing driving agents with diverse driving styles.
Introduction
While Large Language Models (LLMs) have shown great promise in planning and reasoning for autonomous vehicles (AVs), they often lack a specific "driving style." Most current AV systems are tuned for a generic version of safety that doesn't reflect the diversity of human behavior.
The primary challenge is a data gap: most driving datasets consist of non-linguistic data (like CAN-Bus signals or video) which are difficult for LLMs to "read." To align an agent with a human style, the agent needs to understand the why behind a maneuver, not just the what. This research addresses the scarcity of high-quality natural language data regarding human driving reasoning and the lack of frameworks to efficiently synchronize LLM agents with these styles.
The Multi-Alignment Framework
The proposed Multi-Alignment Framework consists of three main components designed to bridge the gap between raw sensor data and nuanced human-like decision-making:
- The Driver Agent: Powered by GPT-4, this agent perceives the environment and maintains a "short-term memory" of recent situations, reasonings, and actions. It uses Chain-of-Thought (CoT) prompting to deduce the best driving action (e.g., "Given the speed of the vehicle ahead... I should match speed by gentle acceleration").
- The Coach Agent: This agent acts as a mentor. It reviews the Driver Agent’s recent performance against "Good" human examples and provides feedback. If the Driver Agent makes a mistake or deviates from the intended style, the Coach issues a "Bad" rating and formulates a new driving guideline.
- Multi-Alignment Strategy: The framework uses an ablation-tested combination of Demonstrations (D) (few-shot prompts showing how a human drives) and Feedback (F) (guidelines generated by the Coach).
To account for the latency of GPT-4 API calls, the team implemented a "slowed-down" simulation clock in the CARLA simulator, ensuring the agent has sufficient time to process information before executing actions.
The Driving-Thinking-Dataset
A core contribution of this work is the Driving-Thinking-Dataset, a novel collection of human driving behaviors translated into natural language.
- Data Collection: 24 drivers performed tasks in a real-world urban environment.
- Post-Driving Interviews: Drivers participated in 1.5–2 hour "think-aloud" interviews where they watched their own driving footage and explained their decision-making process.
- Categorization: Using the Multidimensional Driving Style Inventory (MDSI) and objective CAN-Bus data (speed, throttle %), the researchers distilled these behaviors into two primary styles: Risky and Cautious.
Experimental Evaluation and Insights
The framework was validated in the CARLA simulator and through human surveys involving 270 participants. The study compared various alignment methods—demonstrations, feedback, and the combined multi-alignment approach—across different driving styles.
Key Results
- Safety and Distinctiveness: Cautious agents achieved the lowest collision rates under the full Multi-Alignment (D+F) condition. Conversely, Risky agents exhibited significantly higher average speeds and throttle percentages.
- The Necessity of the "Coach": The study found that simply providing demonstrations is insufficient. The Coach Agent is vital for translating examples into high-level guidelines that the Driver Agent can follow consistently.
- Human Perception: In double-blind tests, humans could clearly distinguish between the styles. Interestingly, participants often associated "risky" driving with higher human-likeness, even if they rated it as less "intelligent" or safe.
- Language as an Interface: By converting driving into a "Situation-Reasoning-Action" format, the LLM leverages its pre-trained linguistic reasoning to solve physical navigation problems.
Limitations and Future Work
While promising, several hurdles remain for real-world deployment. The current system's reliance on GPT-4 introduces significant latency, requiring simulations to be slowed down. Future work will focus on improving real-time performance and exploring a wider variety of driving environments beyond the CARLA Town10 map. Additionally, researchers aim to expand the framework to include more nuanced personas, such as "Eco-friendly" or "Novice" driving styles, while addressing the high cost of current LLM API usage.
Summary updated