From Real to Synthetic: A Fine-grained Dataset and High-fidelity Biomechanical Model for Animal Behavior Understanding

Qiqi Li, Junyi Wu, Hongyu Chen, Pengfei Wang, Nenggan ZhengOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

Abstract

Rat behavior research contributes to the exploration of human disease mechanisms. However, existing datasets are scarce and cover limited behavior types, hindering the analysis and modeling of complex behavior patterns. We constructed ActionRat, a new multi-view rat behavior dataset that, for the first time, captures diverse actions during free exploration and brain-computer interface (BCI) control. It combines real and synthetic sequences with fine-grained keypoint annotations and atomic action sequences, supporting broader behavior analysis tasks. To efficiently generate synthetic data for dataset expansion, we developed OpenRatEngine, a high-fidelity 3D virtual biomechanical model. This model integrates anatomical priors from computed tomography (CT) scans, kinematic constraints, and lifelike appearance, reducing the domain gap between synthetic and real data. Equipped with pose control, OpenRatEngine generates synthetic sequences with accurate 3D keypoint annotations. We evaluated behavioral uncertainty quantification and animal pose estimation tasks on the ActionRat dataset, and demonstrated the outstanding synthetic data generation capability and realism of OpenRatEngine. Extensive experiments across deep learning models confirmed the effectiveness and value of both real and synthetic data.
BibTeX
@misc{li2026from,
  author = {Qiqi Li and Junyi Wu and Hongyu Chen and Pengfei Wang and Nenggan Zheng},
  title = {{From Real to Synthetic: A Fine-grained Dataset and High-fidelity Biomechanical Model for Animal Behavior Understanding}},
  year = {2026},
  url = {https://openreview.net/forum?id=tDdeW2puHW},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure