Knowledge-Gated Action Unit Integration for Fine-Grained Contextual Emotion Recognition in Vision-Language Frameworks
Lists
Only you see this
Abstract
Large Vision-Language Models (LVLMs) have demonstrated exceptional zero-shot potential. However, their application in Context-Based Emotion Recognition (CBER) is often limited by a tendency to prioritize global scene semantics over fine-grained facial dynamics. This reliance on global context frequently results in the omission of subtle Action Units (AU) or the generation of hallucinations. To bridge this gap, we introduce the action-unit aware understanding and integration framework. Our methodology synergizes a systematic search to map the dependency profile of specific micro-expressions, allowing for the determination of an optimal AU filtering threshold tailored to each affective category. During the inference phase, we leverages a coarse-to-fine mechanism: the LVLM first formulates a preliminary hypothesis, which then facilitates the adaptive injection of filtered expert AU descriptions to refine the final emotional prediction. Comprehensive experimental evaluations show that our approach significantly surpasses fully supervised baselines in a zero-shot setting, achieving a new state-of-the-art mAP in contextual affective computing.
BibTeX
@misc{wan2026knowledge,
author = {Hai Wan},
title = {{Knowledge-Gated Action Unit Integration for Fine-Grained Contextual Emotion Recognition in Vision-Language Frameworks}},
year = {2026},
url = {https://openreview.net/forum?id=QsdmmApjbE},
howpublished = {OpenReview},
}Loading related content...