Salient Object Detection (SOD)—the task of identifying the most "interesting" parts of an image—is a cornerstone of modern computer vision, powering features like portrait mode and smart video cropping. However, most state-of-the-art (SOTA) models are "GPU-heavy," designed for massive desktop cards and performing poorly when ported to mobile CPUs or embedded devices.
In the paper "iNAS: Integral NAS for Device-Aware Salient Object Detection," researchers from Nankai University and Huawei Technologies introduce a framework that doesn't just design a better model, but designs the right model for specific hardware targets.
The Challenge of Device-Specific SOD
The field of SOD currently suffers from a "silo" problem. Researchers typically focus on either designing a powerful backbone (the feature extractor) or a complex saliency head (the part that fuses features into a mask), often ignoring the synergy between the two. Furthermore, a model optimized for an NVIDIA GPU might be sluggish on a mobile phone because different hardware architectures have different bottlenecks, such as memory bandwidth versus raw computation.
Currently, achieving low-latency SOD for a specific device requires labor-intensive hand-design. iNAS automates this by making the entire architecture searchable and "device-aware."
Methodology: Integral Neural Architecture Search
The authors propose three key innovations to make automated, hardware-aware design feasible:
1. Integral Search Space
Unlike previous Neural Architecture Search (NAS) methods that only optimize the backbone, iNAS searches the Backbone, the Transport layers (which connect features), and the Decoder (which produces the final output) simultaneously. This holistic approach allows the saliency head to compensate for backbone choices, leading to more efficient overall designs.
2. Searchable Multi-Scale Unit (SMSU)
To capture objects of different sizes, models usually require complex multi-branch structures. iNAS utilizes SMSUs, which employ multiple branches during training but use structural reparameterization to collapse those branches into a single, high-speed convolution for deployment.
3. Latency-Group Sampling (LGS)
Standard NAS sampling tends to find "average" models. LGS divides the search space into specific latency "buckets." This ensures the search explores the entire spectrum, from ultra-fast mobile models to high-performance desktop models, creating a diverse Pareto frontier of options.
Key Results and Insights
The models discovered by iNAS consistently outperformed hand-crafted SOTA models in speed-to-accuracy trade-offs across various hardware platforms:
- Significant Speedups: Compared to previous SOTA methods, iNAS reduced latency by 3.8× on the Huawei Nova6 SE, 3.3× on Intel CPUs, and 2.6× on the NVIDIA Jetson Nano.
- Efficiency: The
iNAS(GPU)-Lmodel matches top-tier accuracy (Max F-measure of 0.947 on ECSSD) while being significantly leaner in terms of FLOPs and wall-clock time. - Zero-Retraining Deployment: Using a "one-shot" supernet approach, the resulting models can be deployed immediately after the search without needing to be trained from scratch.
Conclusion
iNAS demonstrates that searching the backbone and head together while incorporating device-specific latency leads to superior real-world performance. While the search phase still requires significant compute (0.8 GPU-days), the resulting models bridge the gap between high-accuracy research and practical mobile deployment. Future work may extend this "integral" approach to video salient object detection and memory-constrained environments.
Summary updated