RealLife-Bench: A Benchmark for VLMs on Open-Ended Daily Activities with Holistic Evaluation
Lists
Only you see this
Abstract
We introduce \textbf{RealLife-Bench}, a holistic evaluation framework designed to rigorously assess Vision Language Models in open ended household tasks. First to simulate realistic daily activities we manually annotate scene agnostic task skeletons and employ Hierarchical Topological Projection to map these high level semantic instructions such as ``I am hungry'' and their corresponding multi-solution directed-acyclic-graphs onto diverse simulation environments to generate executable task completion trajectories. Second we establish a comprehensive evaluation framework comprising an end to end online evaluation and five distinct process evaluations to diagnose specific capability bottlenecks. RealLife Bench comprises 1,021 online evaluation episodes and over 10,000 offline process evaluation samples. Our extensive experiments reveal the correlation between distinct capabilities and the completion of embodied tasks identifying perception grounding as the most critical success factor for current VLMs. We will make the code and dataset publicly available.
BibTeX
@misc{zhao2026reallife,
author = {Youze Zhao and FengJiao Chen and Xiaoyu Li and Jun Kuang and Xuezhi Cao and Xu Yang},
title = {{RealLife-Bench: A Benchmark for VLMs on Open-Ended Daily Activities with Holistic Evaluation}},
year = {2026},
url = {https://openreview.net/forum?id=lyWU5VdD35},
howpublished = {OpenReview},
note = {Submitted to ICML 2026},
}Loading related content...