Progressive Learning of Instance-Level Proxy Semantics for Few-Shot Action Recognition
Lists
Only you see this
Abstract
Few-shot action recognition is a crucial task for mitigating the challenges of data scarcity in video understanding. Recent advancements in large-scale pre-trained models have introduced the potential of incorporating semantic knowledge from multi-modal pre-trained models, such as CLIP, to alleviate these challenges. Although some progress have been made, existing methods still rely on class-level text embeddings that are inherently low in diversity, limiting their ability to generalize to unseen actions. To overcome this limitation, we propose a novel framework called Progressive Learning of Instance-Level Proxy Semantics (ProLIPS). ProLIPS integrates Proxy Semantic Diffusion (PSD) to generate rich, instance-level proxy semantic features with diverse semantic contents and temporal dynamics, utilizing a multi-step CLIP-guidance mechanism and a time-conditioned reverse diffusion process. Our approach preserves the diversity of semantic-aligned visual features, significantly improving the generalization and robustness of few-shot action recognition. Extensive experiments on five challenging benchmarks demonstrate the effectiveness of ProLIPS.
BibTeX
@article{peng2026progressive,
author = {Fang Peng and Xiaoshan Yang and Yaowei Wang and Changsheng Xu},
title = {{Progressive Learning of Instance-Level Proxy Semantics for Few-Shot Action Recognition}},
journal = {IEEE Transactions on Multimedia},
year = {2026},
url = {https://openreview.net/forum?id=PX1I3mGvGO},
}Loading related content...