Abstract
Unsupervised video object segmentation (UVOS) requires identifying and tracking salient objects without initial human-provided masks. This paper presents a "Key Instance Selection" strategy that analyzes video saliency and appearance frequency in early frames to select a fixed number of primary targets. This approach mitigates the "ID explosion" problem common in unsupervised methods, where the number of tracked objects grows uncontrollably. The system achieved a 3rd place ranking in the DAVIS 2019 Unsupervised Challenge.
Introduction
In semi-supervised video object segmentation (SVOS), a human provides a mask for the first frame, defining the tracking targets. In contrast, unsupervised video object segmentation (UVOS) requires the system to autonomously determine which objects are significant.
A naive UVOS approach—tracking every object proposal—faces two major hurdles:
- Complexity Explosion: The number of tracked IDs increases indefinitely, leading to excessive memory and processing demands.
- Noise: Many proposals are background clutter or transient artifacts that do not represent meaningful objects.
The authors address these by identifying a fixed number ($K$) of "key" instances early in the video, focusing resources on the most salient targets.
Methodology
The proposed pipeline follows a "propose, link, and select" architecture.
Candidate Generation and Linking
The system utilizes Mask R-CNN and DeepLab to generate object proposals. To maintain continuity during motion blur or occlusions, masks are propagated from previous frames using RGMP. An online tracker links new proposals to existing IDs based on:
- IoU (Intersection over Union): Spatial overlap between consecutive frames.
- Trajectory: Predicted bounding box locations.
- Re-identification (ReID): Comparison of appearance features against a historical "object pool."
- Semantic Co-segmentation: Category-level feature consistency.
Key Instance Selection
This core component operates during the first $M$ frames. The system collects all potential IDs and, at frame $M$, calculates a Selection Score for each based on:
- Video Saliency: How much the object stands out or moves relative to the background.
- Frequency: The consistency of the ID's appearance across the initial frames.
Only the top $K$ IDs are retained for tracking throughout the remainder of the video. Hyperparameters for this selection were optimized using a Meta AI search on a 144-GPU cluster.
Experimental Results
The method was evaluated on the UVOS DAVIS challenge dataset (30 videos), which lacks initial masks.
The system ranked 3rd overall with a Global Mean ($J\&F$) score of 0.516. Key findings include:
- Selection Efficiency: The "Key Instance" method significantly outperformed random selection. For $K=5$, the method achieved a score of 0.576, compared to 0.474 for random selection.
- Robustness: Qualitative analysis showed the model successfully handling fast motion (e.g., "mbike-trick") and multiple similar objects (e.g., "lab-coat").
Discussion and Key Insights
The primary insight is that "importance" is as critical as "objectness." By leveraging video saliency, the model distinguishes primary actors (e.g., a scooter rider) from background elements (e.g., trees). Furthermore, the use of an Object Pool—comparing candidates to a history of ReID descriptors rather than just the previous frame—enhances robustness against occlusions.
Limitations
- Temporal Bottleneck: Objects entering after the initial $M$ frames may be ignored.
- Fixed $K$: A static number of instances may not suit all videos; an adaptive $K$ could be more effective.
- Initial Computational Load: The selection phase requires running multiple heavy models (Mask R-CNN, Saliency, etc.) simultaneously.
Summary updated