URNet: User-Resizable Residual Networks with Conditional Gating Module

Sang-Ho Lee, Simyung Chang, Nojun Kwak

Abstract

URNet (User-Resizable Residual Network) introduces a flexible deep learning architecture that enables runtime adjustment of computational costs without retraining. By incorporating a lightweight Conditional Gating Module (CGM), URNet learns to skip specific residual blocks based on input features and a user-defined resource budget. This approach allows for direct control over the accuracy-efficiency trade-off, making it ideal for environments with fluctuating service demands. Evaluated on ImageNet and CIFAR benchmarks, URNet demonstrates superior performance compared to static compression and existing dynamic networks, maintaining high accuracy even under significant computational constraints.

Introduction

Standard Convolutional Neural Networks (CNNs) are typically designed with a fixed computational cost. However, real-world applications often face fluctuating resource availability and service demands. For instance, a cloud-based inference server may need to process a sudden surge of requests by reducing the per-task precision to maintain overall throughput.

Existing solutions, such as model compression or standard dynamic networks, lack the necessary flexibility. Compression creates a static, smaller model, while dynamic networks decide their own execution paths without allowing for external user control. URNet addresses these limitations by providing a "resizable" architecture that gives users direct control over the trade-off between speed and accuracy via a simple scale parameter.

Methodology

The core innovation of URNet is the Conditional Gating Module (CGM), which is integrated into a standard ResNet backbone.

The Gating Mechanism

A CGM is placed before each residual block. It processes features from the previous layer alongside a user-specified scale parameter $S$ (ranging from 0 to 1). The module outputs a binary decision to either execute or skip the associated residual block. These modules are extremely efficient, adding less than 0.1% computational overhead to the base model.

The URNet architecture featuring Conditional Gating Modules (CGM) and the dual-loss training objective.

Training Strategy

To overcome the non-differentiability of binary decisions, URNet employs a probabilistic training approach. During training, the gate operates as a sigmoid function with probability $p$ and a binary step function with probability $1-p$, allowing for standard backpropagation.

To ensure the network adheres to the user-defined budget, the authors introduce a Scale Loss ($L_s$). This loss function penalizes the model if the average number of active blocks deviates from the target scale $S$. The final objective is a weighted sum of the classification loss and the scale loss.

Experiments and Results

URNet was evaluated on CIFAR-10, CIFAR-100, and ImageNet (ILSVRC2012). The results demonstrate that URNet consistently achieves a better accuracy-vs-FLOPs trade-off than other state-of-the-art dynamic methods like BlockDrop.

Quantitative evaluation of URNet across different scales, showing accuracy, block usage, and FLOPs for various hyperparameter settings.

Key Findings

  • Efficiency on ImageNet: Using a ResNet-101 backbone, URNet maintains baseline accuracy (76.4%) while utilizing only 80% of the original computation. At 65% computation, the accuracy drop is limited to approximately 1%.
  • Smart Regularization: At scales around 70-80%, URNet occasionally outperforms the baseline ResNet. This suggests the gating mechanism acts as a form of intelligent regularization, similar to stochastic depth.
  • Input-Aware Adaptability: Qualitative analysis reveals that the model intelligently skips blocks for "easy" images while activating more blocks for complex or noisy scenes, even when a lower overall scale is requested.

Conclusion and Future Work

URNet provides a robust framework for deploying deep learning models in dynamic environments. By allowing users to "dial" computational costs at runtime, it bridges the gap between static model design and real-world operational requirements. While training for a wide range of scales can slightly impact peak performance, the flexibility and stability of the model across $S \in [0.2, 1.0]$ make it a highly practical solution. Future research may explore hardware-specific optimizations to further translate theoretical FLOP savings into wall-clock speedups.

Summary updated

Abstract

Convolutional Neural Networks are widely used to process spatial scenes, but their computational cost is fixed and depends on the structure of the network used. There are methods to reduce the cost by compressing networks or varying its computational path dynamically according to the input image. However, since a user can not control the size of the learned model, it is difficult to respond dynamically if the amount of service requests suddenly increases. We propose User-Resizable Residual Networks (URNet), which allows users to adjust the scale of the network as needed during evaluation. URNet includes Conditional Gating Module (CGM) that determines the use of each residual block according to the input image and the desired scale. CGM is trained in a supervised manner using the newly proposed scale loss and its corresponding training methods. URNet can control the amount of computation according to user's demand without degrading the accuracy significantly. It can also be used as a general compression method by fixing the scale size during training. In the experiments on ImageNet, URNet based on ResNet-101 maintains the accuracy of the baseline even when resizing it to approximately 80% of the original network, and demonstrates only about 1% accuracy degradation when using about 65% of the computation.
BibTeX
@inproceedings{lee2020urnet,
  author = {Sang-Ho Lee and Simyung Chang and Nojun Kwak},
  title = {{URNet: User-Resizable Residual Networks with Conditional Gating Module}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.5886},
}
Figure