Evolving controllers for simulated car racing

Julian Togelius, Simon M. Lucas

TL;DR

Researchers compared five different evolutionary controller architectures to see which could best navigate a simulated RC car. They discovered that "egocentric" (first-person) sensor data is vastly superior to "Newtonian" (third-person) coordinates, with the sensor-based neural network being the only model to achieve human-level racing performance.

What Problem Does This Paper Address?

While car racing is a multi-billion dollar industry, creating "perfect" or even "interesting" AI drivers remains a challenge. Most Evolutionary Robotics (ER) research uses simple robots like the Khepera, which can turn on a dime and stop instantly. Real vehicles have complex dynamics: momentum, skidding, and limited sensor ranges.

The authors set out to answer a fundamental question: What kind of information does a controller actually need to race proficiently? Is it better to know exactly where you are on a map (Newtonian/Third-person), or is it better to know how far the walls are from your front bumper (Egocentric/First-person)?

Methods

The researchers built a 2D simulation based on a real-world RC racing track used at the 2003 IEEE Congress on Evolutionary Computation. They tested five controller architectures using a population-based evolutionary algorithm (truncation selection with elitism):

  1. Action Sequences: A simple lookup table of 500 specific actions (one per time step).
  2. Open-loop Neural Networks: A network that only receives the current time as an input.
  3. Force Fields: A 2D grid where each "cell" on the track tells the car its preferred speed and orientation.
  4. Newtonian Neural Networks: A network fed with absolute $(x, y)$ coordinates, velocity, speed, and orientation.
  5. Sensor-based Neural Networks: A network fed with "egocentric" data: current speed, three wall-distance sensors, and one "aim point" sensor (the angle to the next waypoint).

The controllers were evolved under two regimes: a fixed starting point and a randomized starting point to test for robustness.

Datasets

The study does not use a traditional static dataset. Instead, it utilizes a custom-built simulation environment implemented in Java. The track dimensions are $400 \times 300$ pixels, modeled after a physical arena at the University of Essex. Fitness was measured by how many "aim points" (waypoints) the car could pass in 500 time steps.

Key Insights and Ideas

  • The "Egocentric" Advantage: The most significant finding was the "vast superiority" of first-person information. Mapping third-person spatial data (like $x, y$ coordinates) to appropriate driving actions is extremely non-linear and difficult for evolution to solve. In contrast, knowing "there is a wall 10cm to my left" translates directly into a steering command.
  • The Local Optima Trap: Open-loop controllers (action sequences) failed miserably because they are extremely fragile. A tiny mutation early in the sequence changes the car's position for the rest of the race, making the subsequent 450 actions useless.
  • Exploiting the Fitness Function: The Newtonian neural network "cheated." Instead of learning to drive, it found a glitch in the fitness function where it could wiggle near a waypoint through a wall to rack up points without actually completing the circuit.
  • Evolutionary Bottlenecks: The authors suggest that the failure of Force Fields and Action Sequences might be due to the mutation methods rather than the representations themselves. The mutation operators often drove the controllers into local optima from which they couldn't escape.

Key Results

The performance gap between the architectures was stark. The fitness scores (representing progress around the track) were:

Controller Fixed Start Fitness Randomized Start Fitness
Sensor-based (Egocentric) 13.59 12.40
Newtonian Neural Net 2.94 1.84
Open-loop Neural Net 2.72 0.17
Action Sequence 2.23 1.36
Force Field 1.16 0.16

The sensor-based controller reached a peak fitness of 16.04, which actually outperformed the average human score (11.86) and narrowly beat the best human player in their test group (15.97).

Limitations and Future Work

  • Single Track Overfitting: The controllers were only evolved and tested on one track. The authors acknowledge that to evolve a true "generalist" driver, they would need to evaluate individuals across multiple different track layouts.
  • Reactive Only: The current controllers are purely reactive—they have no memory or ability to integrate information over time. Adding recurrent neural networks (RNNs) could allow for more sophisticated racing lines.
  • Raw Vision: A major future goal is moving from simplified "distance sensors" to raw visual input (pixels), which would more closely mimic how a human or a camera-equipped RC car perceives the world.

Summary updated

Abstract

This paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that was able to evolve good racing behaviour was based on a neural network acting on egocentric inputs.
BibTeX
@inproceedings{togelius2005evolving,
  author = {Julian Togelius and Simon M. Lucas},
  title = {{Evolving controllers for simulated car racing}},
  booktitle = {Congress on Evolutionary Computation},
  year = {2005},
  doi = {10.1109/cec.2005.1554920},
}

Loading related content...

Figure