Mechanisms for Open-Ended Evolution in Artificial Systems

This direction is split into other directions: open-ended evolution. Its last version stays readable and citable here.

Version 1 of 131 papers38 conceptsfrozen 2026-10-04

In biological history, evolution does not "finish." It does not reach a single optimal organism and stop; instead, it produces an ever-expanding tree of complexity, from single cells to social civilizations. In contrast, most artificial intelligence and artificial life simulations suffer from a "complexity plateau." Whether they are optimizing a neural network's weights or simulating a digital ecosystem, these systems typically reach an equilibrium where innovation ceases and the system stagnates.

The research direction of Open-Ended Evolution (OEE) seeks to bridge this gap. It aims to design systems that continuously generate novel, complex, and "interesting" forms without human intervention.

The Problem of Stagnation

As noted in the proceedings of ALIFE XI, a primary challenge in artificial life is that models eventually reach a state of equilibrium. Vito Trianni and colleagues argue that this lack of open-endedness often stems from a reliance on centralized selection mechanisms or fixed environments that offer a finite number of "niches." Once an agent solves the environment's challenges, there is no further pressure to innovate.

To overcome this, researchers have identified several key mechanisms that allow a system to keep moving.

Complexification: Beyond Parameter Tuning

Early neuroevolution focused on optimizing the weights of a fixed neural architecture. However, Kenneth Stanley and Risto Miikkulainen argued in their work on NEAT that true innovation requires complexification. Instead of just changing how a brain works, evolution must be able to change the structure of the brain itself—adding new neurons and connections over time.

By starting with simple models and incrementally adding complexity, evolution can maintain an "arms race." This principle is extended in ATEP by Michael Beukman, which allows agents to evolve their neural topologies as they encounter increasingly difficult challenges, preventing the structural stagnation that occurs when an agent's "brain size" is fixed.

Co-evolution and the Environment

If the environment is static, evolution will eventually "solve" it. Open-endedness requires a dynamic environment that evolves alongside the agents.

  1. Emergent Curricula: Systems like PINSKY co-evolve game levels and agent policies. Aaron Dharna and Lisa B. Soros found that the most critical moments in these systems are "stepping stones"—rare transfers where an agent learns a skill in one environment that unexpectedly unlocks progress in a much more difficult one.
  2. Automated World-Building: OMNI-EPIC, developed by Maxence Faldor and Jeff Clune, uses Large Language Models to autonomously code new environments. By using "interestingness" as a metric, the system generates an infinite stream of novel challenges, from dodgeball to restaurant service, ensuring the agents never run out of things to learn.

Self-Organization in Continuous Substrates

A different approach to OEE focuses on Self-Organization (SO) within cellular automata. Rather than explicitly programming agents, researchers look for life-like structures that emerge from local rules.

Bert Wang-Chak Chan’s Lenia framework provides a continuous "chemical soup" where complex, moving patterns emerge. To drive these patterns toward OEE, Niko Lorantos introduced multi-objective ranking that rewards stability, novelty, and sparsity simultaneously.

The recent Flow-Lenia by Erwan Plantec and Pierre-Yves Oudeyer adds mass conservation to this system. By treating activations as "matter" that flows, the system naturally produces multi-species simulations where speciation and extinction emerge from internal dynamics rather than external fitness functions.

Quality-Diversity and Curiosity

In OEE, the goal is often not to find the best solution, but to find all possible interesting solutions. This is the domain of Quality-Diversity (QD) algorithms.

Maxence Faldor and Antoine Cully developed Leniabreeder, which uses QD to discover a vast ecosystem of patterns. Similarly, Gautier Hamon explores how curiosity-driven algorithms can act as a catalyst for OEE, pushing the search into unexplored regions of the state space to avoid the "diversity plateau" where a single dominant species takes over.

The Biological Arrow of Time

Why does complexity increase at all? Fernando Rosas and Mikhail Prokopenko suggest a theoretical framework based on Turing jumps. They argue that major evolutionary transitions occur when organisms begin to model themselves, creating a self-referential computational loop. This "biological arrow of time" suggests that OEE is not just about adding more parts, but about the emergence of new levels of information hierarchy.

By shifting the focus from solving specific tasks to maintaining a perpetual process of innovation—through complexification, co-evolution, and self-organization—this field moves closer to creating artificial systems that possess the creative power of natural evolution.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_mechanisms_for_open_ended_evolution_in_a_1acf790f,
  title = {Mechanisms for Open-Ended Evolution in Artificial Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mechanisms-for-open-ended-evolution-in-artificial-systems/rdr_1acf790f15c3487aa1a4507a223d6f74},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_mechanisms_for_open_ended_evolution_in_a_1acf790f_v1,
  title = {Mechanisms for Open-Ended Evolution in Artificial Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mechanisms-for-open-ended-evolution-in-artificial-systems/rdr_1acf790f15c3487aa1a4507a223d6f74/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}