Starting Point Selection and Multiple-Standard Matching for Video Object Segmentation With Language Annotation
Lists
Only you see this
Abstract
In this study, we investigate language-level video object segmentation, where first-frame language annotation is used to describe the target object. Because a language label is typically compatible with all frames in a video, the proposed method can choose the most suitable starting frame to mitigate initialization failure. Apart from extracting the visual feature from a static video frame, a motion-language score based on optical flow is also proposed to describe moving objects more accurately. Scores of multiple standards are then aggregated using an attention-based mechanism to predict the final result. The proposed method is evaluated on four widely-used video object segmentation datasets, including the DAVIS 2017, DAVIS 2016, SegTrack V2 and YouTubeObject datasets, and a novel accuracy measured as mean region similarity is obtained on both the DAVIS 2017 (67.2%) and DAVIS 2016 (83.5%) datasets. The code will be published.
BibTeX
@article{sun2023starting,
author = {Mingjie Sun and Jimin Xiao and Eng Gee Lim and Yao Zhao},
title = {{Starting Point Selection and Multiple-Standard Matching for Video Object Segmentation With Language Annotation}},
journal = {IEEE Trans. Multimedia},
year = {2023},
doi = {10.1109/tmm.2022.3159403},
}Loading related content...