Sample-Efficient Behavior Cloning Using General Domain Knowledge

Feiyu Zhu, Jean Oh, Reid G. Simmons

Introduction

Behavior Cloning (BC) is a fundamental approach in imitation learning where an agent learns a policy by mimicking expert trajectories. Despite its conceptual simplicity, standard BC often suffers from extreme sample inefficiency and poor generalization. Generic architectures, such as Multi-Layer Perceptrons (MLPs), must learn task-relevant features and physical constraints from scratch, requiring thousands of demonstrations. Furthermore, these "black-box" models lack semantic structure, making them fragile when encountering distribution shifts.

This paper introduces Knowledge Informed Models (KIM), a framework that leverages Large Language Models (LLMs) to bridge the gap between human intuition and neural network architecture. By translating natural language domain knowledge into structured, sparse network topologies, KIM achieves high performance with a fraction of the data required by traditional methods.

The KIM Framework

The KIM pipeline: Human experts provide natural language strategies which an LLM translates into a structured neural network architecture for training.

The KIM methodology consists of a three-step pipeline designed to bake "common sense" directly into the model's structure:

  1. Knowledge Extraction: A human expert describes high-level strategies in natural language (e.g., "Slow down when there is a corner ahead").
  2. LLM-Guided Architecture Generation: An LLM (such as GPT-4o) parses these instructions to identify latent variables and their functional dependencies. It then generates a PyTorch implementation where the network's layers and connections reflect these semantic relationships rather than being fully connected.
  3. Parameter Tuning: The resulting model is highly sparse. Instead of training dense matrices, the agent only tunes a small set of parameters (weights, biases, and clipping bounds) using a combination of gradient descent and grid search.

Crucially, the LLM also assists in weight initialization by analyzing the described correlations (e.g., assigning positive initial weights for positively correlated variables), which significantly improves optimization stability.

Experimental Evaluation

The researchers evaluated KIM on two benchmarks from the Gymnasium library: Lunar Lander (discrete actions) and Car Racing (continuous actions). In both environments, KIM was compared against standard small and large MLP baselines across varying numbers of expert demonstrations.

Key Results

Success rate on the Lunar Lander task across different numbers of demonstration episodes. KIM (green) consistently outperforms standard NN baselines.
  • Sample Efficiency: In the Lunar Lander task, KIM achieved a success rate approximately 20% higher than standard MLPs when trained on the same limited data. Remarkably, KIM could solve the task reliably with as few as 5 demonstrations.
  • Robustness: In Car Racing, KIM achieved near-expert rewards with only 2 demonstrations. Qualitative analysis showed that KIM-driven agents followed the track center more accurately and navigated corners more smoothly than baseline models, which frequently drove off-track.
  • Noise Tolerance: When Gaussian noise was introduced to the action space, KIM's performance degraded gracefully. In Car Racing, KIM retained 65% of its reward under high noise levels that caused baseline MLPs to fail completely.

Discussion and Future Work

The primary advantage of KIM lies in its semantic sparsity. By replacing thousands of redundant parameters with a handful of structured connections (e.g., 15 parameters for Lunar Lander), the model becomes nearly impossible to overfit on small datasets. This structure also ensures that the agent's actions remain grounded in physical principles, preventing the catastrophic failures common in unconstrained neural networks.

Future work aims to develop interactive KIM, allowing models to be updated incrementally as new expert knowledge or data becomes available. Additionally, the authors see potential in applying this structured approach to learning transition dynamics (World Models) to further enhance autonomous system reliability.

Summary updated

BibTeX
@inproceedings{zhu2025sample,
  author = {Feiyu Zhu and Jean Oh and Reid G. Simmons},
  title = {{Sample-Efficient Behavior Cloning Using General Domain Knowledge}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/807},
}
Figure