What is the appropriate speed for an autonomous vehicle? Designing a Pedestrian Aware Contextual Speed Controller

Daniel Jiang, Stewart Worrall, Mao Shan

Abstract

Social acceptance is a major hurdle for autonomous vehicle technology, central to which is ensuring both passengers and nearby pedestrians feel safe. This idea of 'feeling safe' and perceived safety is highly subjective and rooted in human intuition. As such, traditional analytical approaches to autonomous navigation often fail to cater for the social expectations of individuals. Therefore, this paper proposes an approach to capture the complexity of social expectations and integrate this complexity into a 3-layered Contextual Speed Controller. The layers were; the legal road speed limit, the socially acceptable speed given the number of nearby pedestrians, and the socially acceptable speed based on proximity to nearby pedestrians. An implementation of this layered approach was tested in areas of both low and high vehicle-pedestrian interactions. From the experiments conducted, the lower two layers were seen working in tandem to modulate the vehicle speed to appropriate levels that mimicked conservative human driver behaviour. In summary, this work quantified the relationship between pedestrian context and socially acceptable vehicle speeds, allowing for more perceivably safe autonomous driving. Furthermore, the need for different driving schemes for navigating different road environments was identified.

Introduction: The Challenge of Social Acceptance

The "uncanny valley" of autonomous driving isn't just about avoiding collisions; it’s about social comfort. While current autonomous vehicle (AV) path planners are excellent at calculating collision-free trajectories, they often treat speed as a secondary byproduct of the path. However, perceived safety is highly subjective. For example, driving 60 KPH next to a sidewalk is normal on a major road but feels dangerous in a shared pedestrian plaza. Traditional analytical models often fail to capture these social nuances. By decoupling speed modulation from path planning, the authors aim to reduce computational complexity while making AVs behave more like intuitive human drivers.

Proposed Methodology: A 3-Layered Controller

The researchers proposed a Contextual Speed Controller operating in three distinct layers to manage vehicle velocity based on environmental cues.

System architecture of the 3-layered Contextual Speed Controller.
  1. Legal Layer: Adheres to the posted road speed limits.
  2. Context Layer: Determines an appropriate speed based on the number of pedestrians in the scene. The authors created a "Speed Profile" lookup table by analyzing how human drivers naturally slow down as pedestrian density increases.
  3. Proximity Layer: Adjusts speed based on how close pedestrians are to the vehicle's intended path.

Technical Implementation

To achieve real-time performance, the system fuses 2D RGB data (using YOLOv3) with 3D LiDAR point clouds. By projecting 3D clusters onto 2D image planes for validation, they achieved a fusion runtime of under 10ms. A key innovation is the Lateral Scaling Factor, a "buffer" mechanism where pedestrians near the path are projected forward onto the path using a tunable multiplier. For instance, if a person is 3m to the side, a scaling factor of 3 treats them as if they are 9m directly ahead, forcing the car to slow down preemptively.

Experimental Setup and Datasets

The study utilized the USyd Campus Dataset, which contains over a year of weekly naturalistic driving data from the University of Sydney. The data was categorized into three distinct environments to test the controller's versatility.

Comparison of a regular road (left) and a shared pedestrian plaza (right) from the USyd Campus Dataset.
  • Shared Spaces: High pedestrian density, no distinct segregation between cars and people.
  • Semi-shared Roads: Heavy interaction but with some road markings.
  • Regular Roads: Vehicle-dominant roads with pedestrians mostly on sidewalks.

Key Insights and Results

The data confirmed that human drivers do not have a linear relationship with speed. In shared spaces, the correlation between pedestrian count and speed reduction is much stronger than on regular roads. By making the speed controller a modular "plug-in" rather than part of the core path planner, the system reduces the dimensionality of the planning problem, allowing the vehicle to respond faster to dynamic environments.

Quantified Social Speeds: The researchers produced a lookup table for "socially acceptable" speeds. In shared spaces, the average human speed dropped from 14.7 KPH (0-2 pedestrians) to 8.5 KPH (9+ pedestrians). Real-Time Performance: The 3D pedestrian detection pipeline maintained low inference times (<60ms for YOLOv3 and <10ms for fusion), making it viable for live deployment. Behavioral Matching: Qualitative analysis showed that the final output of the 3-layered controller closely tracked the "Real Driver" baseline, effectively modulating speed even when pedestrians were detected in 2D but not yet confirmed in 3D.

Limitations and Future Work

The system currently relies heavily on the initial 2D bounding boxes from YOLOv3; if the 2D detector misses a person due to poor lighting or occlusion, the 3D fusion stage fails to account for them. Additionally, the proximity layer approximates the vehicle's path based on the current wheel angle. Integrating the planned future trajectory would likely improve precision. Finally, while the system mimics human drivers, future work should include surveys to determine if pedestrians actually feel more comfortable around an AV equipped with this controller.

Summary updated

Abstract

Social acceptance is a major hurdle for autonomous vehicle technology, central to which is ensuring both passengers and nearby pedestrians feel safe. This idea of `feeling safe' and perceived safety is highly subjective and rooted in human intuition. As such, traditional analytical approaches to autonomous navigation often fail to cater for the social expectations of individuals. Therefore, this paper proposes an approach to capture the complexity of social expectations and integrate this complexity into a 3-layered Contextual Speed Controller. The layers were; the legal road speed limit, the socially acceptable speed given the number of nearby pedestrians, and the socially acceptable speed based on proximity to nearby pedestrians. An implementation of this layered approach was tested in areas of both low and high vehicle-pedestrian interactions. From the experiments conducted, the lower two layers were seen working in tandem to modulate the vehicle speed to appropriate levels that mimicked conservative human driver behaviour. In summary, this work quantified the relationship between pedestrian context and socially acceptable vehicle speeds, allowing for more perceivably safe autonomous driving. Furthermore, the need for different driving schemes for navigating different road environments was identified.
BibTeX
@misc{jiang2021what,
  author = {Daniel Jiang and Stewart Worrall and Mao Shan},
  title = {{What is the appropriate speed for an autonomous vehicle? Designing a Pedestrian Aware Contextual Speed Controller}},
  year = {2021},
  eprint = {2104.06147},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.06147},
}
Figure