Sample-Efficient Behavior Cloning Using General Domain Knowledge

Feiyu Zhu, Jean Oh, Reid G. Simmons

Abstract

Standard Behavior Cloning (BC) often requires vast amounts of expert data to train effective robotic agents. Researchers from Carnegie Mellon University have introduced Knowledge Informed Models (KIM), a framework that leverages Large Language Models (LLMs) to incorporate human domain knowledge directly into neural network architectures. By translating natural language advice into structured, sparse policies, KIM achieves high performance with as few as five demonstration episodes, significantly outperforming traditional black-box models in both sample efficiency and robustness.

Introduction

Behavior Cloning (BC) is a fundamental approach in imitation learning where an agent learns to perform a task by mimicking an expert's actions. Despite its conceptual simplicity, BC is notoriously data-hungry and prone to failure in novel scenarios. Standard neural networks, such as Multi-Layer Perceptrons (MLPs), lack the inherent structure or inductive biases necessary to generalize from small datasets. While human experts can easily articulate the logic behind their actions—such as "slow down before a corner"—translating this qualitative advice into a quantitative model has traditionally required intensive manual feature engineering.

The KIM Framework

The Knowledge Informed Models (KIM) framework automates the integration of human logic into trainable models through a three-step pipeline:

  1. Knowledge Extraction: A human expert provides high-level strategic advice in natural language. For instance, in a racing task, the expert might specify: "Speed up when there is nothing ahead; slow down when there is a corner ahead."
  2. LLM-Guided Architecture Design: An LLM (specifically GPT-4o) interprets the natural language instructions to generate a structured policy in PyTorch. Unlike dense, fully-connected networks, KIM architectures are sparse and semantically grounded. They include specific nodes for latent variables like target_angle or distance_to_center, creating what the authors term a "Semantic Latent Space."
  3. Parameter Tuning and Warm Start: To overcome the optimization challenges inherent in highly structured models, the LLM suggests initial weights based on semantic correlations. The model is then fine-tuned on a minimal set of expert demonstrations using standard gradient descent to optimize the exact parameter values.
The KIM pipeline: translating human advice into structured neural architectures via LLMs.

Experimental Evaluation

The authors evaluated KIM on two classic benchmarks from the Gymnasium library: Lunar Lander (v2) and Car Racing (v2). Lunar Lander requires discrete actions to land a craft softly between two flags, while Car Racing involves continuous control. To test real-world applicability, the Car Racing demonstrations were provided by a human using a physical steering wheel controller, which introduced natural human error and noise into the training data.

Key Insights

  • Structure as a Regularizer: The sparse, logical flow of the KIM architecture prevents the model from overfitting to task-irrelevant features like background colors or random visual noise.
  • Efficiency through Sparsity: In the Lunar Lander task, the KIM model utilized only 15 parameters—orders of magnitude fewer than standard baselines—yet achieved superior performance.
  • Interpretability: The use of semantic latents allows researchers to inspect internal nodes (e.g., slow_down_adj) to understand the reasoning behind specific actions, moving away from the "black box" nature of traditional AI.

Results

KIM demonstrated remarkable sample efficiency across both tasks. In the Lunar Lander environment, KIM reached a ~90% success rate with only 5 to 10 demonstration episodes. In contrast, standard neural network baselines struggled to reach even a 40% success rate with the same amount of data.

Success rate comparison in Lunar Lander: KIM significantly outperforms standard small and large neural networks with minimal demonstration episodes.

Furthermore, KIM exhibited significant robustness to noise. In the Car Racing task, when Gaussian noise was added to the agent's actions, KIM retained approximately 65% of its reward even under high noise levels, whereas the performance of standard baseline models plummeted. This suggests that the underlying logical structure provides a stabilizing effect that raw data-driven models lack.

Conclusion and Future Work

KIM represents a significant step toward making robot training more accessible and efficient by utilizing the reasoning capabilities of LLMs to bridge the gap between human intuition and neural network design. However, the approach remains dependent on the quality of human instructions. Future work aims to develop an "interactive KIM" where humans can debug model logic in real-time and build libraries of action primitives to solve increasingly complex, multi-stage robotic tasks.

Summary updated

BibTeX
@inproceedings{zhu2025sample,
  author = {Feiyu Zhu and Jean Oh and Reid G. Simmons},
  title = {{Sample-Efficient Behavior Cloning Using General Domain Knowledge}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/807},
}
Figure