Vehicle Community Strategies

Cinjon Resnick, Ilya Kulikov, Kyunghyun Cho, Jason Weston

Abstract

Interest in emergent communication has recently surged in Machine Learning. The focus of this interest has largely been either on investigating the properties of the learned protocol or on utilizing emergent communication to better solve problems that already have a viable solution. Here, we consider self-driving cars coordinating with each other and focus on how communication influences the agents' collective behavior. Our main result is that communication helps (most) with adverse conditions.

Introduction

While individual vehicle intelligence has advanced significantly through driver-assistance systems, highway safety remains a critical global challenge. The United States Department of Transportation has explored mandates for vehicle-to-vehicle (V2V) communication, yet the specific utility of shared data in complex, multi-agent coordination remains under-explored. Most research in emergent communication focuses on abstract puzzles; this work shifts the focus to a high-stakes physical task: highway driving. We investigate whether shared protocols can compensate for environmental hazards, such as fog, that limit individual sensing capabilities.

Methodology

The study utilizes a custom 2D highway simulation built on the Box2D physics engine and integrated with OpenAI Gym. The environment features 12 autonomous vehicles navigating a highway with randomized geometry, exit points, and car dimensions.

The multi-agent highway simulation environment showing vehicle interactions and sensor ranges.

Agent Control and Communication

Vehicles are controlled via a shared policy trained using Proximal Policy Optimization (PPO). Three communication regimes were evaluated:

  1. No Communication (Baseline): Agents rely on a 40-dimensional local observation vector, including LiDAR data and internal state.
  2. V2V Protocol: Agents receive a 21-dimensional message from neighbors containing explicit state information such as acceleration and path history.
  3. Emergent Communication: Agents attempt to learn a protocol from scratch, though this proved difficult to stabilize in complex environments.

To ensure robustness, the researchers implemented a 10% message dropout during training, preventing the agents from becoming overly reliant on a brittle communication channel.

Experimental Setup

The simulation is procedurally generated to ensure diversity, varying highway angles and exit locations across 15 distinct configurations. A key experimental variable is "fog," which uniformly reduces the range of a vehicle's LiDAR sensors. This creates a scenario where individual "vision" is impaired, potentially increasing the value of communicated information from other agents.

Results and Discussion

The primary metric for success is "Episode Success," defined as all 12 vehicles reaching their respective exits without a collision.

Quantitative results comparing the Baseline and V2V models across sunny and foggy conditions.

As shown in the results, communication provides a marginal benefit in "sunny" (ideal) conditions. However, in "foggy" conditions, the V2V protocol acts as a critical sensor redundancy. While the baseline model's episode success drops to 54.56% in fog, the V2V-enabled agents maintain a 61.04% success rate. When models are specifically trained in adverse conditions, the performance gap widens, with V2V models significantly outperforming the baseline.

The Safety-Speed Tradeoff

An interesting behavioral observation was that V2V-enabled vehicles tended to drive approximately 10% slower than baseline agents. This suggests that the communication protocol facilitates a more cautious, coordinated collective behavior that prioritizes safety over individual speed, leading to higher overall success rates in hazardous environments.

Challenges in Emergent Communication

Despite the success of the engineered V2V protocol, agents struggled to "invent" effective emergent languages. Many training runs converged to degenerate behaviors, such as vehicles spinning in place. This indicates that for complex physical coordination, human-engineered data structures currently provide more stability than purely learned protocols.

Conclusion

This research demonstrates that V2V communication is most valuable not in ideal scenarios, but as a robust fallback when individual sensors fail. Future work should aim to stabilize emergent communication and compare these reinforcement learning policies against traditional heuristic-based safety systems to further refine collective autonomous driving strategies.

Summary updated

Abstract

Interest in emergent communication has recently surged in Machine Learning. The focus of this interest has largely been either on investigating the properties of the learned protocol or on utilizing emergent communication to better solve problems that already have a viable solution. Here, we consider self-driving cars coordinating with each other and focus on how communication influences the agents' collective behavior. Our main result is that communication helps (most) with adverse conditions.
BibTeX
@misc{resnick2018vehicle,
  author = {Cinjon Resnick and Ilya Kulikov and Kyunghyun Cho and Jason Weston},
  title = {{Vehicle Community Strategies}},
  year = {2018},
  eprint = {1804.07178},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.07178},
}
Figure