Automating the Search for Artificial Life with Foundation Models

Akarsh Kumar, Chris Lu, Louis Kirsch, Yujin Tang, Kenneth O. Stanley, Phillip Isola, David Ha

TL;DR

The paper introduces Automated Search for Artificial Life (ASAL), a novel framework that leverages vision-language foundation models (like CLIP) to automate the discovery of complex, emergent behaviors in Artificial Life (ALife) simulations. By using foundation models as a proxy for human perception, ASAL successfully guides searches for specific target phenomena, open-ended novelty, and diverse simulation atlases across multiple classic ALife environments.

What Problem Does This Paper Address?

A core goal of Artificial Life (ALife) is to understand "life as it could be" by studying computational simulations. However, discovering the exact rules and configurations that lead to interesting, lifelike emergent behaviors—such as self-replication or flocking—has historically been a bottleneck. Because simple local rules interact in highly non-linear ways, predicting a simulation's macro-behavior from its micro-parameters is incredibly difficult.

As a result, researchers have mostly relied on manual trial-and-error or rigid, handcrafted mathematical metrics of "complexity" and "interestingness." These traditional metrics often fail to capture the nuanced, subjective concepts that humans naturally recognize as "lifelike."

Methods

ASAL solves this by outsourcing the evaluation of simulations to vision-language foundation models (FMs). Since these models are trained on internet-scale data, their latent spaces inherently align well with human visual and conceptual representations.

The authors formulate ASAL around three distinct automated search mechanisms:

  1. Supervised Target Search: Finds simulation parameters ($\theta$) that generate visual states matching a specific natural language prompt (or a sequence of prompts over time) by maximizing the cosine similarity between the simulation's rendered frames and the text embedding.
  2. Open-Endedness Search: Discovers simulations that continuously generate novel patterns over time. It achieves this by minimizing the alignment between the current simulation frame and its historically closest neighbor in the FM embedding space.
  3. Illumination Search: Maps out an entire substrate to find a diverse set of behaviors by minimizing the embedding similarity between different simulations, effectively pushing them apart to fill the space.
Figure 2: ASAL: Our proposed framework, ASAL, uses vision-language foundation models to discover ALife simulations by formulating the processes as three search problems. Supervised Target: To find target simulations, ASAL searches for a simulation which produces a trajectory in the foundation model space that aligns with a given sequence of prompts. Open-Endedness: To find open-ended simulations, ASAL searches for a simulation which produces a trajectory that has high historical novelty during each timestep. Illumination: To illuminate the set of simulations, ASAL searches for a set of diverse simulations which are far from their nearest neighbor.

ASAL is entirely substrate-agnostic, meaning it can be applied to any simulation that can be rendered as an image. The authors demonstrate its utility across five diverse ALife substrates: Boids (flocking), Particle Life (particle interactions), Life-like Cellular Automata (discrete grids), Lenia (continuous cellular automata), and Neural Cellular Automata (NCA).

Datasets

This paper does not introduce or evaluate traditional static datasets. Instead, it operates directly on the generative parameter spaces of standard ALife substrates. For evaluation and embedding generation, the authors utilize pretrained foundation models, specifically CLIP (Contrastive Language-Image Pretraining) and DINOv2.

Key Insights and Ideas

  • FMs as Human Proxies: The most profound insight is that foundation models can act as automated, human-aligned judges for qualitative phenomena. They bridge the gap between raw pixel data and high-level human concepts.
  • Quantifying the Qualitative: Because the FM latent space is continuous and structured, it allows researchers to quantitatively measure previously purely qualitative concepts, such as parameter sensitivity, simulation plateaus, and the exact point where "more is different" (e.g., discovering that a certain pattern only emerges when the particle count exceeds 1,000).
  • The Edge of Chaos via Novelty: By maximizing historical novelty in the FM space, the system naturally uncovers simulations that exist on the "edge of chaos"—neither freezing into static patterns nor exploding into white noise.

Key Results

  • Target Discovery: ASAL successfully discovered highly complex configurations matching diverse prompts like "a biological cell under the microscope," "a network of neurons," and even "a pepperoni pizza" across Lenia, Boids, and Particle Life.
Figure 3: Discovered target simulations: Using Equation 2, ASAL discovered simulations that result in a final state which matches the specified prompt. Results are shown for three different substrates. More prompts can be seen in appendix Figures 12, 13, and 14.
  • Rediscovering Conway's Game of Life: In a brute-force search of 262,144 Life-like Cellular Automata, ASAL's open-endedness metric ranked the famous Conway's Game of Life in the top 5% of all open-ended simulations, validating the metric's alignment with human historical consensus.
  • Ablation Studies: Comparing CLIP, DINOv2, and raw pixel representations for illumination search revealed that both deep foundation models generated significantly more diverse and human-meaningful simulation atlases than low-level pixel metrics, with CLIP showing a slight edge due to its language alignment.

Limitations and Future Work

While powerful, ASAL currently evaluates simulations frame-by-frame using image-based foundation models. This means it can miss purely temporal dynamics (e.g., the direction of a rotation or a specific cadence of movement). The authors note that replacing image FMs with video-language foundation models is a natural next step to natively capture time-dependent behaviors.

Additionally, the search is fundamentally constrained by the expressivity of the underlying substrate; if a substrate's rules cannot physically support a "caterpillar" shape, the optimizer will fail. The authors envision future iterations where Large Language Models (LLMs) are used to write and mutate the code of the substrates themselves, potentially automating the search all the way from basic physics up to complex, open-ended alien ecosystems.

Summary updated

Abstract

With the recent Nobel Prize awarded for radical advances in protein discovery, foundation models (FMs) for exploring large combinatorial spaces promise to revolutionize many scientific fields. Artificial Life (ALife) has not yet integrated FMs, thus presenting a major opportunity for the field to alleviate the historical burden of relying chiefly on manual design and trial-and-error to discover the configurations of lifelike simulations. This paper presents, for the first time, a successful realization of this opportunity using vision-language FMs. The proposed approach, called Automated Search for Artificial Life (ASAL), (1) finds simulations that produce target phenomena, (2) discovers simulations that generate temporally open-ended novelty, and (3) illuminates an entire space of interestingly diverse simulations. Because of the generality of FMs, ASAL works effectively across a diverse range of ALife substrates including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. A major result highlighting the potential of this technique is the discovery of previously unseen Lenia and Boids lifeforms, as well as cellular automata that are open-ended like Conway's Game of Life. Additionally, the use of FMs allows for the quantification of previously qualitative phenomena in a human-aligned way. This new paradigm promises to accelerate ALife research beyond what is possible through human ingenuity alone.
BibTeX
@misc{kumar2024automating,
  author = {Akarsh Kumar and Chris Lu and Louis Kirsch and Yujin Tang and Kenneth O. Stanley and Phillip Isola and David Ha},
  title = {{Automating the Search for Artificial Life with Foundation Models}},
  year = {2024},
  eprint = {2412.17799},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.17799},
}
Figure