Visual Hide and Seek

Boyuan Chen, Shuran Song, Hod Lipson, Carl Vondrick

Abstract

We train embodied agents to play Visual Hide and Seek where a prey must navigate in a simulated environment in order to avoid capture from a predator. We place a variety of obstacles in the environment for the prey to hide behind, and we only give the agents partial observations of their environment using an egocentric perspective. Although we train the model to play this game from scratch, experiments and visualizations suggest that the agent learns to predict its own visibility in the environment. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, effect the learned policy. Our results suggest that, although agent weaknesses make the learning problem more challenging, they also cause more useful features to be learned.

Introduction

In reinforcement learning (RL), agents often develop "reactive" policies—simple mappings from pixels to actions that solve specific tasks without a deep understanding of scene dynamics. This research explores whether a complex, multi-agent task like Hide and Seek can trigger the emergence of higher-level concepts, such as object affordances and perspective-taking. Unlike previous studies using top-down views, this work utilizes a partially observable, 3D environment where agents rely solely on egocentric (first-person) visual input, mimicking real-world biological constraints.

Methods

The simulation environment, built in the Unity engine, consists of a 14x14 arena with various obstacles like cubes and cylinders. The game involves two agents: a Hider (the learning agent) and a Seeker (a programmed opponent).

The Hider's architecture uses a neural network trained via Proximal Policy Optimization (PPO), processing raw pixel input from a 120-degree field of view. The Seeker follows a deterministic logic: it moves toward the hider if visible, otherwise exploring waypoints or moving to the last known position. To analyze representation learning, the authors introduced systematic interventions, such as varying the hider's speed and adding auxiliary "visibility rewards."

Overview of the Visual Hide and Seek simulation environment, showing (a) the Seeker's observation, (b) the global simulation state, and (c) the Hider's egocentric observation.

Key Insights and Results

A central finding of this study is the Disadvantage Paradox: imposing "judicious weaknesses" on the agent, such as reduced speed, actually fosters superior internal representations. While a fast agent can survive through reactive evasion, a slower agent must learn to predict its own visibility and leverage obstacles strategically.

Probing the latent space of trained agents revealed emergent perspective-taking. Without explicit supervision, agents developed features capable of predicting whether they were visible to the seeker with high accuracy (up to 94.58% in some variants). This suggests the agents developed a rudimentary "Theory of Mind" regarding the predator's perspective.

t-SNE visualization of the agent's internal features, demonstrating a clear clustering based on whether the seeker is visible (S) or not (¬S).

Quantitatively, the basic hider achieved a 45% success rate in avoiding capture over 1,000 steps. Qualitative analysis showed that agents with visibility rewards or speed disadvantages developed cautious behaviors, such as "peeking" from behind obstacles and hugging walls to minimize exposure.

Conclusion and Future Work

This research demonstrates that task difficulty and agent constraints are key drivers for the emergence of sophisticated environmental representations in embodied AI. However, a significant limitation remains: the lack of temporal memory. Current agents occasionally fail because they "forget" the location of obstacles or the seeker once they leave the field of view. Future work will focus on integrating computational memory models, such as LSTMs, to enable long-term mapping and tracking, potentially leading to even more human-like strategic behaviors.

Summary updated

Abstract

We train embodied agents to play Visual Hide and Seek where a prey must navigate in a simulated environment in order to avoid capture from a predator. We place a variety of obstacles in the environment for the prey to hide behind, and we only give the agents partial observations of their environment using an egocentric perspective. Although we train the model to play this game from scratch, experiments and visualizations suggest that the agent learns to predict its own visibility in the environment. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, effect the learned policy. Our results suggest that, although agent weaknesses make the learning problem more challenging, they also cause more useful features to be learned. Our project website is available at: http://www.cs.columbia.edu/ ~bchen/visualhideseek/.
BibTeX
@inproceedings{chen2020visual,
  author = {Boyuan Chen and Shuran Song and Hod Lipson and Carl Vondrick},
  title = {{Visual Hide and Seek}},
  booktitle = {ALIFE},
  year = {2020},
  url = {https://openreview.net/forum?id=SeOh-kSPmUb},
}
Figure