Abstract
High-resolution salient object detection (HRSOD) remains challenging due to the limitations of standard models on 4K/8K imagery, which often result in incomplete object detection or blurred boundaries. We present the Recurrent Multi-scale Transformer (RMFormer), which utilizes a recurrent architecture with shared weights to refine predictions across scales consistently. To support this research, we introduce HRS10K, the largest high-resolution SOD dataset to date, containing 10,500 meticulously annotated images at 2K-8K resolution. RMFormer achieves state-of-the-art performance across multiple benchmarks, particularly in boundary precision.
Introduction
Salient Object Detection (SOD) is fundamental for visual tracking and object recognition. While effective at low resolutions, standard models struggle with high-resolution (4K/8K) inputs, producing artifacts like irregular boundaries. Existing high-resolution strategies, such as global-local fusion or multi-stage refinement, often suffer from error accumulation or high parameter counts. Furthermore, the scarcity of large-scale high-resolution datasets has hindered the development of robust models.
RMFormer: Recurrent Multi-scale Architecture
RMFormer addresses these challenges by employing a recurrent refinement strategy rather than a standard feed-forward approach. By sharing weights across scales, the model learns a more robust representation of saliency that is not tied to a specific resolution, preventing the error accumulation common in non-recurrent models.
The architecture consists of several key components:
- Coarse Prediction Stage (CPS): Generates an initial saliency map from a low-resolution image using a Swin Transformer backbone.
- Recurrent Refinement Stages (RRS): Progressively refines the map at higher scales using the same shared transformer backbone, ensuring consistency and parameter efficiency.
- Dual-flow Guided Decoder (DGD): Integrates inter-stage guidance (from lower resolutions) and inner-stage details (from the current scale) to maintain sharp boundaries while filling in object centers.
- Pixel-wise Refiner (PR): Targets the most uncertain pixels, typically along edges, for high-precision re-prediction using local and global context.
The HRS10K Dataset
To overcome data limitations, we introduce HRS10K, featuring 10,500 images at 2K-8K resolution. It significantly expands upon previous benchmarks like HRSOD (1,610 images) and UHRSD (4,932 images).
HRS10K is characterized by:
- High Complexity: As shown in the edge pixel distribution, it contains objects with significantly higher edge complexity than older datasets.
- Diversity: It spans 105 categories across 7 major subjects (Stuff, Plant, Person, Object, Mobile, Food, Animal), reducing the performance bias observed in models trained on smaller, less diverse datasets.
Results and Discussion
RMFormer was evaluated against 13 state-of-the-art methods. It consistently ranked at the top across metrics including MAE, F-measure, and mean Boundary Accuracy (mBA). On the HRS10K-TE benchmark, RMFormer achieved an mBA of 0.766, surpassing the previous best (PGNet) at 0.745. Qualitative results demonstrate its superior ability to capture fine details, such as fluffy fur or thin structures like Ferris wheel wires.
While highly accurate, the model is computationally intensive, requiring 563.14 GFLOPs and running at approximately 16.1 FPS on an RTX 3090. This makes it slower than some low-resolution-only models, though significantly more precise for high-resolution applications.
Conclusion
RMFormer provides a robust solution for high-resolution SOD through recurrent refinement and targeted pixel-wise processing. By introducing the HRS10K dataset, we provide a more challenging and diverse benchmark for the community. Future work will focus on optimizing the architecture to reduce computational overhead and improve inference speed for real-time high-resolution applications.
Summary updated