Embodiment dictates learnability in neural controllers

Joshua P. Powers, Ryan Grindle, Sam Kriegman, Lapo Frati, Nick Cheney, Josh C. Bongard

TL;DR

This paper demonstrates that a robot's physical design, specifically the placement of its sensors, can dramatically influence its ability to learn new tasks without forgetting old ones (catastrophic forgetting). By optimizing sensor placement, researchers can create "loss landscapes" that make it easier for neural controllers to find solutions that generalize across multiple tasks.

What Problem Does This Paper Address?

Catastrophic forgetting is a major hurdle in machine learning, particularly for systems that need to learn and adapt continuously in dynamic environments. When a neural network learns a new task, it often overwrites the knowledge gained from previous tasks. While much research has focused on improving learning algorithms to mitigate this, this paper explores a different angle: how the robot's own physical embodiment can inherently make it more or less susceptible to forgetting. The "environment" for a robot includes its own body, and how its sensors and actuators are arranged on that body can fundamentally change the problem it's trying to solve.

Methods

The authors use a simplified simulated robot with two wheels and two infrared sensors. They investigate how the placement of these sensors on the robot's body affects the learnability of its neural controller.

  • Theoretical Model: They derive a system of differential equations to model the robot's movement based on sensor inputs and control parameters (weights).
  • Empirical Model: They simulate the robot in a physics engine (Open Dynamics Engine) to validate the theoretical findings and account for real-world complexities.
  • Task: The robot is trained to perform "phototaxis" – moving towards a light source – in four different environments, each with the light source at a different location.
  • Metrics:
    • ML (Learnability): Measures the proportion of controllers that are successful in all four environments for a given robot design. A higher ML means it's easier to find a generalist controller.
  • MCF (Resistance to Catastrophic Forgetting): Measures the ratio of generalist controllers (successful in all tasks) to specialist controllers (successful in at least one task). A higher MCF indicates better resistance to forgetting.
  • Search Space: They systematically explore a large design space by discretizing sensor positions (81x81 grid) and controller weights (121x121 grid), evaluating millions of combinations.

Datasets

Not discussed in the paper. The experiments are based on simulated environments and a custom robot model.

Key Insights and Ideas

  • Embodiment as a First-Class Citizen for Learning: The paper strongly suggests that physical design is not just about performance but also about learnability. A well-designed body can make learning significantly easier.
  • Counter-Intuitive Designs Excel: The "best" designs, both theoretically and empirically, were often asymmetrical and unintuitive. This challenges the common human bias towards symmetrical and "natural-looking" robot designs, which can inadvertently hinder learning.
  • Sensor Placement Shapes the Loss Landscape: By altering sensor positions, the authors show that the "weight manifold" (the set of controller parameters that solve a task) can be expanded or shrunk, and their overlap across tasks can be increased or decreased. This directly impacts the difficulty of finding generalist solutions.
  • "Canonical" Designs are Suboptimal for Learning: The symmetrical, anteriormost sensor placement, often considered a standard or "canonical" design, performed poorly in terms of both learnability and resistance to catastrophic forgetting compared to optimized asymmetrical designs.

Key Results

  • Theoretical Model: An asymmetrical sensor placement (l₁ = (-0.5, -0.25), l₂ = (0.5, 0.25)) achieved the highest controller learnability (ML = 0.286) and a high resistance to catastrophic forgetting (MCF = 0.636).
  • Empirical Model: A different asymmetrical design (Fig. 3C) performed best on both metrics, achieving ML = 0.0039 and MCF = 0.038. This design had significantly more generalist controllers (57 out of 14641) compared to the canonical design (1 out of 14641).
  • Canonical Design Performance: The canonical, symmetrical design had much lower learnability (ML = 0.049 theoretical, ML = 0.000068 empirical) and resistance to catastrophic forgetting (MCF = 0.24 theoretical, MCF = 0.00052 empirical).
  • Visualizations: Heatmaps clearly show that the weight manifolds for asymmetrical designs are more "open" and easier to navigate for learning algorithms, whereas the canonical design's manifold is like finding a "needle in a haystack."

Limitations and Future Work

  • Simplification: The theoretical model is highly abstracted, and the empirical model uses simplified physics (e.g., single-point wheel contact). Real-world factors like friction, motor limits, and collisions are not fully captured.
  • Discrete Search: The authors used discretized grids for sensor placement and weights, meaning they explored a subset of the continuous design space.
  • Limited Parameters: Only sensor placement and controller weights were optimized. Future work should investigate co-optimizing morphology and control parameters simultaneously.
  • Testing with Learning Algorithms: The paper's metrics are theoretical and empirical measures of the loss landscape's suitability for learning. They acknowledge that these metrics need to be tested against actual learning algorithms.
  • Generalizability: The study uses a simple robot and task. Future work should explore how morphology influences learnability in more complex robots, task environments, and behaviors.

Summary updated

Abstract

Catastrophic forgetting continues to severely restrict the learnability of controllers suitable for multiple task environments. Efforts to combat catastrophic forgetting reported in the literature to date have focused on how control systems can be updated more rapidly, hastening their adjustment from good initial settings to new environments, or more circumspectly, suppressing their ability to overfit to any one environment. When using robots, the environment includes the robot's own body, its shape and material properties, and how its actuators and sensors are distributed along its mechanical structure. Here we demonstrate for the first time how one such design decision (sensor placement) can alter the landscape of the loss function itself, either expanding or shrinking the weight manifolds containing suitable controllers for each individual task, thus increasing or decreasing their probability of overlap across tasks, and thus reducing or inducing the potential for catastrophic forgetting.
BibTeX
@inproceedings{powers2020embodiment,
  author = {Joshua P. Powers and Ryan Grindle and Sam Kriegman and Lapo Frati and Nick Cheney and Josh C. Bongard},
  title = {{Embodiment dictates learnability in neural controllers}},
  booktitle = {ALIFE},
  year = {2020},
  doi = {10.1162/isal\_a\_00243},
}
Figure