Abstract
Behavioral cloning in multiagent systems often relies on fixed-point estimates like the mean to predict agent actions. However, in stochastic biological systems, behavior is frequently multimodal, making the mean an unrealistic predictor. This paper demonstrates that sampling from a learned empirical distribution significantly outperforms standard regression techniques in maintaining the realism of simulated behaviors. By evaluating models on both simulated boids and live fish data, we show that while traditional regression may minimize error, sampling-based approaches are essential for capturing the true underlying distribution of behavior.
Introduction
In Agent-Based Modeling (ABM), the goal is to create "executable models"—simulations that accurately replicate the behavior of real-world subjects such as schooling fish or insects. This is typically achieved through behavioral cloning, where a machine learning model learns to map an agent's environmental state to its subsequent action.
A significant challenge in this process is stochasticity. Most standard machine learning techniques, such as Linear Regression, are designed to find the "central tendency" or the mean of the data to minimize prediction error. However, biological behavior is often multimodal. Consider a fish swimming toward a wall: it must turn either left or right. A standard model calculating the average of these two actions would predict a path straight ahead—directly into the wall. This "Fish-Wall" example illustrates how minimizing error can lead to physically impossible or unnatural behaviors in a simulation.
Methods
This research compares three primary approaches to behavioral cloning:
- Linear Regression (Lin-Reg): A standard fixed-estimate predictor assuming a linear relationship and Gaussian noise.
- k-Nearest Neighbors Regression (kNN-Reg): A non-parametric method that averages the actions of the $k$ most similar states in the training set.
- kNN-Sample (Proposed): Instead of averaging, this method treats the $k$ nearest neighbors as a local empirical distribution and samples a single action at random.
To evaluate these methods, the authors use standard metrics like Root Mean Square Error (RMSE) alongside Kullback-Leibler (K-L) divergence. While RMSE measures point-by-point accuracy, K-L divergence measures how closely the simulated action distribution matches the observed biological distribution. A lower K-L divergence indicates higher realism.
Experimental Setup
The framework was tested on three datasets:
- Deterministic Boids: A baseline simulation using Reynolds' flocking rules.
- Stochastic Boids: Boids with added non-symmetric "exponential noise" to mimic unpredictable bursts of speed.
- Live Fish Data: Real-world tracking data of Notemigonus crysoleucas (golden shiners), comprising 1.4 million data points from high-resolution video tracking.
Key Results
The experiments revealed a critical trade-off between accuracy and realism. In deterministic cases, all methods performed similarly. However, in stochastic environments, the proposed kNN-Sample method significantly outperformed fixed-estimate models.
For stochastic boids, Linear Regression achieved a low RMSE but a massive K-L divergence (482.6), indicating a complete failure to capture the behavior's distribution. In contrast, kNN-Sample maintained a K-L divergence of 0.0465. In live fish experiments, kNN-Sample was the only method capable of reproducing the asymmetric, Poisson-like velocity distributions observed in real data. Fixed-estimate models produced "clumped" distributions around the mean that failed to resemble the training data qualitatively.
Discussion and Future Work
The findings suggest that for biological simulations, matching the distribution of behavior is often more important than minimizing point-by-point error. Non-parametric methods like kNN-sampling are particularly effective because they do not assume a specific shape for the noise (e.g., Gaussian).
Future work should focus on automating feature selection for complex behaviors and integrating sampling principles into more sophisticated models, such as Recurrent Neural Networks. Furthermore, the authors suggest that future behavioral cloning algorithms should incorporate distribution similarity metrics, like K-L divergence, directly into their cost functions during training.
Summary updated