E2GC: Energy-efficient Group Convolution in Deep Neural Networks

Nandan Kumar Jha, Rajat Saini, Subhrajit Nag, Sparsh Mittal

TL;DR

While Group Convolutions (GConv) are a staple for reducing the computational footprint of deep networks, a naive choice of the number of groups can actually destroy energy efficiency. This paper introduces E2GC, a design strategy that keeps the group size (channels per group) constant across layers, balancing the trade-off between reduced computation and the high cost of data movement.

What Problem Does This Paper Address?

As deep learning moves toward battery-constrained edge devices (drones, wearables, self-driving cars), optimizing for "FLOPs" or "Parameters" is no longer enough—we must optimize for energy.

Standard Group Convolutions (GConv) and Depthwise Convolutions (DWConv) are popular because they drastically reduce Multiply-Accumulate (MAC) operations. However, the authors point out a hidden cost: as you increase the number of groups ($g$) to save on computation, you simultaneously decrease data reuse. When data reuse drops, the system spends more energy moving data from memory to the processor than it does actually performing the math. Most existing architectures (like ResNeXt or ShuffleNet) fix the number of groups ($g$) across the whole network, which the authors argue leads to suboptimal energy usage and fragmented memory access.

Methods

The authors propose a shift in perspective: instead of fixing the number of groups ($g$), we should fix the group size ($G$)—the number of channels within each group.

  1. Optimum Group Size Modeling: They derive a mathematical model to find the "sweet spot" for energy efficiency. Their model accounts for the disparity between the energy cost of an arithmetic operation versus a memory access.
  2. E2GC Module: Based on their model, they propose the Energy-efficient Group Convolution (E2GC) module. In this setup, as the network gets deeper and the number of channels ($m$) increases, the number of groups ($g$) also increases to keep $G$ (where $G = m/g$) constant.
  3. Comparison with FgGC: They compare their approach against "Fixed group Group Convolution" (FgGC), which is the standard industry practice of keeping $g$ constant (e.g., $g=32$ in ResNeXt-50).
  4. Pointwise Blending: Like other efficient architectures, they follow the E2GC layer with a $1 \times 1$ pointwise convolution to ensure information flows between the disjoint groups.

Datasets

The researchers evaluated their approach on two major image classification benchmarks:

  • ImageNet-1K: The gold standard for large-scale object recognition (1.2M images).
  • Food-101: A challenging fine-grained dataset with 101 categories, known for having noisy images and high local variation.

Key Insights and Ideas

  • The Energy "U-Curve": The authors observed that as $g$ increases, energy consumption initially drops (due to fewer MACs) but eventually starts increasing again. This happens because the overhead of low data reuse and fragmented memory access begins to outweigh the computational savings.
  • Arithmetic Intensity (AI) is King: The paper highlights that DWConv ($G=1$) has the lowest AI, leading to poor resource utilization on GPUs. E2GC aims to keep AI at a level that keeps the hardware busy.
  • Implicit Regularization: Group convolutions act as a form of structured "DropConnect." By keeping $G$ constant, E2GC provides a more consistent regularization effect across the network compared to FgGC, where the regularization strength effectively changes as layers get deeper.
  • Generalization vs. Power: There is a fundamental trade-off between a network's "representational power" (ability to learn complex features) and its "generalization ability" (ability to perform on new data). E2GC allows designers to tune $G$ to find the best balance for a specific dataset.

Key Results

The authors tested their modules by replacing standard layers in MobileNet-V1 and ResNeXt-50 on NVIDIA P100 and P4000 GPUs:

  • Energy Savings: On the P100 GPU, E2GC increased the energy efficiency of MobileNet-V1 by 10.8% and ResNeXt-50 by 4.73% compared to standard fixed-group implementations.
  • Efficiency vs. Baselines: MobileNet-V1 with E2GC ($G=32$) was found to be up to 65.4% more energy-efficient than the baseline on the P4000 GPU.
  • Accuracy Gains: Interestingly, E2GC often improved accuracy. For ResNeXt-50 on ImageNet, the E2GC version reached 77.04% Top-1 accuracy, significantly higher than the baseline's 74.43% at comparable computational complexity.

Limitations and Future Work

  • Hardware Sensitivity: The "optimal" group size $G$ is platform-dependent. A value that works for an NVIDIA P100 might not be the same for a mobile DSP or a TPU, as it depends on the specific memory hierarchy and cache sizes.
  • Manual Tuning: While the paper provides a model, finding the exact constants ($\alpha, \beta$) for a new hardware architecture still requires some empirical measurement.
  • Scope: The study focuses on image classification; future work could explore if these energy savings translate as cleanly to object detection or segmentation tasks where spatial dimensions and memory patterns differ.

Summary updated

Abstract

The number of groups (g) in group convolution (GConv) is selected to boost the predictive performance of deep neural networks (DNNs) in a compute and parameter efficient manner. However, we show that naive selection of g in GConv creates an imbalance between the computational complexity and degree of data reuse, which leads to suboptimal energy efficiency in DNNs. We devise an optimum group size model, which enables a balance between computational cost and data movement cost, thus, optimize the energy-efficiency of DNNs. Based on the insights from this model, we propose an “energyefficient group convolution” (E2GC) module where, unlike the previous implementations of GConv, the group size (G) remains constant. Further, to demonstrate the efficacy of the E2GC module, we incorporate this module in the design of MobileNet-V1 and ResNeXt-50 and perform experiments on two GPUs, P100 and P4000. We show that, at comparable computational complexity, DNNs with constant group size (E2GC) are more energy-efficient than DNNs with a fixed number of groups (Fg GC). For example, on P100 GPU, the energy-efficiency of MobileNet-V1 and ResNeXt-50 is increased by 10.8% and 4.73% (respectively) when E2GC modules substitute the Fg GC modules in both the DNNs. Furthermore, through our extensive experimentation with ImageNet-1K and Food-101 image classification datasets, we show that the E2GC module enables a trade-off between generalization ability and representational power of DNN. Thus, the predictive performance of DNNs can be optimized by selecting an appropriate G. The code and trained models are available at https://github.com/iithcandle/E2GC-re1ease.
BibTeX
@inproceedings{jha2020e,
  author = {Nandan Kumar Jha and Rajat Saini and Subhrajit Nag and Sparsh Mittal},
  title = {{E2GC: Energy-efficient Group Convolution in Deep Neural Networks}},
  booktitle = {VLSID},
  year = {2020},
  url = {https://openreview.net/forum?id=tBFaAn_lpR},
}

Loading related content...

Figure