Empirical Performance Evaluation of Lane Keeping Assist on Modern Production Vehicles

Yuhang Wang, Abdulaziz Alhuraish, Shuyi Wang, Zhou HaoarXiv 2025 · DOI · arXiv

TL;DR

This paper provides the first comprehensive empirical analysis of real-world Lane Keeping Assist (LKA) system failures using a novel open dataset. It categorizes LKA failures into perception, planning, and control errors, revealing that faded lane markings, low contrast, and sharp curves are major contributors, and proposes models to inform infrastructure design for safer LKA deployment.

What Problem Does This Paper Address?

Lane Keeping Assist (LKA) systems are becoming ubiquitous in modern vehicles, promising to enhance safety by preventing lane departures. However, despite their widespread adoption and the projected societal benefits, LKA systems still exhibit operational deficiencies, particularly in challenging real-world conditions. Existing research often relies on limited or proprietary datasets, controlled environments, or single vehicle models, hindering a comprehensive understanding of LKA's real-world performance and failure modes. This gap is critical because drivers might over-rely on LKA, leading to complacency and potential hazards. The paper aims to bridge this gap by providing a data-driven, empirical evaluation of LKA performance across a diverse range of production vehicles and driving scenarios.

Methods

The study employs a multi-faceted approach to analyze LKA performance:

  • Failure Categorization: LKA failures are systematically categorized into three root causes: perception (issues with detecting lane lines), planning (problems in deciding the vehicle's trajectory), and control (limitations in executing steering commands). This categorization is supported by detailed analysis of CAN signals.
  • Quantitative Performance Metrics: Key metrics are defined and utilized, including:
  • LKA Disengagement Status: A binary indicator of system engagement/disengagement.
  • Lane Deviation (Lane Centering Error): The lateral distance of the vehicle from the lane center.
  • Laneline Detection Result: Analysis of lane line detection performance from both vehicle CAN data and the Openpilot model's confidence score.
  • Statistical Analysis: The paper conducts statistical summaries and distribution analyses of environmental and roadway factors associated with LKA failures.
  • Modeling for Infrastructure Design:
    • A dynamic model is developed to link vehicle speed, curvature, and LKA steering constraints, aiming to inform curvature-aware geometric design and speed posting.
  • A machine learning-based predictive model (using Random Forest and XGBoost) is created to assess roadway readiness for LKA deployment by capturing the relationship between external conditions and LKA performance.

Datasets

The primary dataset used is the OpenLKA dataset, a newly released, comprehensive resource for evaluating LKA systems. It comprises:

  • Over 200 hours of data collected in Florida from various production car models.
  • Over 80 hours of data from the Comma Community open-source automated driving platform, contributed by over 80 drivers across more than 30 regions.

This dataset covers a wide array of driving scenarios, including mountainous terrain, heavy snow, urban environments, and rural roads, and includes over 60 distinct car models. Vision Language Models (VLMs) with Chain-of-Thought prompting were used to extract labels for weather, traffic, and other contextual information not directly available from CAN bus data or video. A subset, OpenLKA-Normal dataset, was created for comparison, representing normal LKA operation.

Key Insights and Ideas

  • Systematic Failure Modes: LKA failures aren't random; they can be reliably categorized into perception, planning, and control issues, providing a structured way to understand and address system weaknesses.
  • LKA vs. Human Driving on Curves: LKA systems tend to rigidly follow the lane center, leading to outward drift on curves. In contrast, human drivers often proactively steer slightly inward on curves, demonstrating a more adaptive and potentially safer strategy.
  • Dominant Failure Factors: Faded lane markings, low pavement-laneline contrast, and sharp road curvature are statistically significant individual factors that increase LKA failure likelihood. Crucially, combinations of these factors dramatically exacerbate the problem.
  • Infrastructure is Key: The paper strongly emphasizes that road infrastructure is not just a backdrop but a critical factor influencing LKA performance. Poorly maintained markings or challenging geometries can render even advanced LKA systems unreliable.
  • Rural Roads are a Challenge: The unique characteristics of rural roads (less consistent markings, lighting, and geometry) pose significant challenges for LKA, potentially widening the gap in ADAS benefits between urban and rural areas.

Key Results

  • Under normal conditions (clear visibility, flat roads, low curvature), LKA systems generally maintain lane deviation within 0.2 meters, demonstrating good performance.
  • LKA systems exhibit a strong linear relationship between lane deviation and road curvature, with deviation increasing as curvature increases. The equation d_lane(m) = -8.327 * curvature + 0.214 quantifies this, showing a tendency to drift away from the lane center on curves.
  • Statistical analysis of the OpenLKA-Failure dataset identified:
  • Faded lane markings (40.5%)

    • Pavement-laneline contrast (22.6%)
    • Sharp curves (18.8%) as the most frequent individual failure-inducing features.
  • Combinations like faded lane markings with poor lighting or sharp curves significantly increase failure likelihood.

  • The machine learning models predict LKA performance with high accuracy and recall, confirming that road curvature and lane markings are the most influential factors, followed by weather and lighting.
  • The models suggest that LKA deviation risk rises significantly when curvature exceeds 0.006 m⁻¹, and reliable operation typically degrades beyond 60.7 mph.

Limitations and Future Work

Limitations:

  • While the OpenLKA dataset is extensive, it's still a snapshot of real-world driving. The authors acknowledge that "long-tail events" (rare but critical scenarios) might not be fully captured.
  • The study focuses on LKA performance and doesn't delve deeply into driver behavior or the interaction between LKA and driver attention beyond noting the potential for complacency.

Future Work:

  • Infrastructure Design: The proposed theoretical and ML models can inform agencies in assessing LKA suitability on road segments and designing infrastructure that is more compatible with LKA systems. This includes improving lane marking durability, refining geometric design standards, and deploying intelligent speed warnings.
  • LKA System Evolution: The authors suggest that LKA systems need to evolve towards more adaptive control strategies, human-like trajectory anticipation, and multimodal perception augmented by infrastructure-integrated sensing (V2I).
  • Addressing Rural Road Safety: The findings highlight the need for targeted infrastructure improvements to ensure rural areas benefit from ADAS capabilities.
  • Broader ADAS Integration: The work provides a foundation for understanding LKA limitations, which is crucial for the development of more advanced and integrated ADAS features.

Summary updated

Abstract

Leveraging a newly released open dataset of Lane Keeping Assist (LKA) systems from production vehicles, this paper presents the first comprehensive empirical analysis of real-world LKA performance. Our study yields three key findings: (i) LKA failures can be systematically categorized into perception, planning, and control errors. We present representative examples of each failure mode through in-depth analysis of LKA-related CAN signals, enabling both justification of the failure mechanisms and diagnosis of when and where each module begins to degrade; (ii) LKA systems tend to follow a fixed lane-centering strategy, often resulting in outward drift that increases linearly with road curvature, whereas human drivers proactively steer slightly inward on similar curved segments; (iii) We provide the first statistical summary and distribution analysis of environmental and road conditions under LKA failures, identifying with statistical significance that faded lane markings, low pavement laneline contrast, and sharp curvature are the most dominant individual factors, along with critical combinations that substantially increase failure likelihood. Building on these insights, we propose a theoretical model that integrates road geometry, speed limits, and LKA steering capability to inform infrastructure design. Additionally, we develop a machine learning-based model to assess roadway readiness for LKA deployment, offering practical tools for safer infrastructure planning, especially in rural areas. This work highlights key limitations of current LKA systems and supports the advancement of safer and more reliable autonomous driving technologies.
BibTeX
@misc{wang2025empirical,
  author = {Yuhang Wang and Abdulaziz Alhuraish and Shuyi Wang and Zhou Hao},
  title = {{Empirical Performance Evaluation of Lane Keeping Assist on Modern Production Vehicles}},
  year = {2025},
  eprint = {2505.11534},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.11534},
}
Figure