Introduction
Neural Architecture Search (NAS) has revolutionized mobile model design, yet most frameworks remain tethered to a "CPU-first" mentality. While models like MobileNetV3 are optimized for CPU latency, real-world deployment frequently targets mobile GPUs for their superior throughput. MoGA (Mobile GPU-Aware) is the first NAS framework specifically designed to optimize for mobile GPUs, identifying a family of models that outperform state-of-the-art architectures in both accuracy and GPU efficiency.
The Mobile GPU Latency Gap
A central motivation for MoGA is the observation that CPU latency is a poor proxy for GPU performance. In mobile environments, a model that appears efficient on a CPU may suffer from significant overhead or under-utilization when executed on a GPU-based inference engine like MACE or SNPE.
As illustrated in the scatter plot above, there is no strong linear correlation between CPU and GPU latencies across different architectures. This discrepancy necessitates a hardware-aware search strategy that directly benchmarks operations on the target GPU hardware.
Methodology: GPU-Aware Evolution
MoGA utilizes a multi-objective evolution strategy based on NSGA-II to navigate the trade-offs between accuracy, latency, and model size. Key technical components include:
- GPU-Aware Search: The framework employs a layer-wise GPU latency lookup table. By benchmarking individual blocks on target hardware, the total GPU latency of a candidate model can be accurately predicted by summing the costs of its components.
- One-Shot Supernet Training: To minimize search costs, MoGA adopts a "FairNAS" style approach. A single large "supernet" is trained once, allowing sub-networks to be evaluated instantly without further training.
- Weighted Fitness Strategy: Instead of treating all objectives equally, MoGA uses customized weights to prioritize accuracy and latency, ensuring the search converges on the most efficient high-performing models.
The Parameter Paradox and Latency Allocation
Traditional mobile design often focuses on minimizing parameter counts to save memory. However, MoGA reveals that at the mobile scale, models are more prone to underfitting than overfitting. Increasing parameters can improve representational power without necessarily increasing latency, provided those parameters are placed in GPU-friendly operations.
By analyzing the latency distribution, it is evident that MoGA models allocate their "latency budget" differently than MobileNetV3. MoGA variants tend to invest more heavily in depthwise convolutions and larger kernels (e.g., 7x7) to maximize receptive fields—operations that mobile GPUs handle more efficiently than CPUs.
Key Results
The MoGA family (MoGA-A, B, and C) consistently matches or exceeds the performance of existing state-of-the-art mobile models:
- MoGA-A: Achieves 75.9% Top-1 accuracy on ImageNet, surpassing MnasNet-A2 while utilizing fewer FLOPs.
- MoGA-C: Specifically optimized for GPU efficiency, it reaches 75.3% accuracy and is faster on mobile GPUs than MobileNetV3, despite being slower on CPUs.
- Search Efficiency: The entire pipeline requires only 12 GPU days, making it roughly 200x faster than the original MnasNet search.
Conclusion
MoGA demonstrates that optimizing for the specific characteristics of mobile GPUs is essential for the next generation of mobile vision models. By addressing the CPU-GPU latency gap and leveraging a multi-objective evolution strategy, MoGA provides a scalable and efficient path for deploying high-performance neural networks on mobile hardware.
Summary updated