Online Personalized Preference Learning Method Based on In-Formative Query for Lane Centering Control Trajectory

Wei Ran, Hui Chen, Taokai Xia, Yosuke Nishimura, Chaopeng Guo, Youyu Yin2023 · DOI

TL;DR

This paper introduces an "Online Personalized Preference Learning Method" (OPPLM) for autonomous vehicles to learn individual driver preferences for lane centering control trajectories. By using pairwise comparisons and a Bayesian approach, the system efficiently learns a driver's ideal path with minimal queries, aiming to improve both safety and comfort.

What Problem Does This Paper Address?

Advanced Driver Assistance Systems (ADAS) and autonomous vehicles are becoming increasingly sophisticated, but their effectiveness hinges on matching individual driver preferences. Current personalization methods often rely on implicit assumptions about driver behavior or require extensive manual input. This paper tackles the challenge of personalizing ADAS, specifically for lane centering control (LCC) trajectories, by developing a method that can accurately learn a driver's unique preferences online, with minimal user interaction. The core problem is that drivers have diverse preferences for how a vehicle should navigate, and a one-size-fits-all approach is insufficient for optimal safety and comfort.

Methods

The proposed Online Personalized Preference Learning Method (OPPLM) utilizes a novel approach combining:

  • Pairwise Comparison Group Preference Queries: Instead of asking drivers to rate trajectories, the system presents pairs of trajectories and asks which one they prefer. This is a more intuitive and less demanding form of feedback.
  • Bayesian Approach for Parameter Estimation: The system uses a Bayesian framework to update its understanding of the driver's preferences. This allows for incorporating prior knowledge and updating beliefs as new data (query answers) become available.
  • Two-Layer Hierarchical Utility Theory Model: Driver preferences are modeled using utility theory, broken down into three perception models: Safety (SPM), Comfort (CPM), and Efficiency (EPM). These models, in turn, use specific trajectory indicators to estimate overall utility. Crucially, the model accounts for the uncertainty in driver responses.
  • Informative and Greedy Query Selection: To speed up learning, the system intelligently selects which trajectory pairs to present to the driver. "Greedy" selection focuses on the most likely preferred options, while "informative" selection aims to reduce uncertainty and improve the model's accuracy.
  • Convergence Criterion: A mechanism is implemented to determine when the system has learned the driver's preferences sufficiently, signaling the end of the learning process.

Datasets

The paper does not describe the use of pre-existing datasets for training. Instead, it relies on a user study where participants provided preference data through interaction with a driving simulator.

Key Insights and Ideas

  • Uncertainty Matters: Explicitly modeling the uncertainty in a driver's response to a query is crucial for accurate preference learning, especially when drivers find it difficult to discern subtle differences between trajectories.
  • Pairwise Comparison is Efficient: Asking drivers to choose between two options is significantly more efficient and less burdensome than other feedback methods, leading to faster learning.
  • Greedy Exploration Balances Speed and Accuracy: The combination of greedy and informative query selection strategies effectively balances the need for quick learning with the necessity of thorough exploration to avoid suboptimal solutions.
  • Utility Theory Provides a Robust Framework: Using utility theory to model preferences, broken down into safety, comfort, and efficiency, provides a structured and interpretable way to understand driver choices.

Key Results

  • Rapid Convergence: The OPPLM converged quickly, requiring an average of only 11 queries per subject. This is a significant improvement over existing methods that can require up to 100 queries.
  • High Accuracy: The learned driver preference model (specifically the Utility Evaluation Model - UEM) demonstrated high accuracy, with a mean Goodness-of-Fit (GOF) of 0.85 and a mean Score-Utility Consistency (SUC) of 0.74. This indicates the model accurately predicts driver preferences.
  • Model Performance: The UEM model significantly outperformed the individual perception models (SPM, CPM, EPM) in terms of GOF and SUC, suggesting that the integrated utility model is more effective.
  • Driver Experience: While experienced drivers showed a slightly higher mean GOF and SUC, the difference was not statistically significant, suggesting the method is effective across different driver experience levels.

Limitations and Future Work

  • Conflicting Answers: Some subjects provided conflicting answers, which can delay convergence and lead to inaccurate estimations. Future work could focus on more robust methods to handle such inconsistencies.
  • Early Convergence: In some cases, the system converged too quickly, potentially leading to sub-optimal solutions. More exploration might be needed to guarantee a truly preferred trajectory.
  • Perception Model Accuracy: The accuracy of the individual perception models (SPM, CPM, EPM) was lower than the overall UEM. Future research could focus on improving the selection of trajectory indicators or the estimation methods for these models.
  • Generalizability: The current study focused on LCC in a simple curve. Extending the OPPLM to more complex driving scenarios (e.g., with other vehicles, complex intersections) and other ADAS functions is a key area for future work.
  • Indicator Optimization: The trajectory indicators used were based on experience. Further research could explore optimizing these indicators or adapting them to individual drivers.

Summary updated

BibTeX
@misc{ran2023online,
  author = {Wei Ran and Hui Chen and Taokai Xia and Yosuke Nishimura and Chaopeng Guo and Youyu Yin},
  title = {{Online Personalized Preference Learning Method Based on In-Formative Query for Lane Centering Control Trajectory}},
  year = {2023},
  doi = {10.3390/s23115246},
}
Figure