Condensation-Net: Memory-Efficient Network Architecture With Cross-Channel Pooling Layers and Virtual Feature Maps

Tse-Wei Chen, Motoki Yoshinaga, Hongxing Gao, Wei Tao, Dongchao Wen, Junjie Liu, Kinya Osa, Masami KatoWorkshops @ CVPR 2019 · DOI

Introduction

Deep learning on resource-constrained embedded systems, such as smart cameras and IoT sensors, involves a constant trade-off between model accuracy and hardware limitations. While techniques like quantization and pruning are common, they often reach a point where further compression severely impacts performance. A primary bottleneck in these systems is memory bandwidth—the energy and time required to move large intermediate data, known as feature maps, between the processor and main memory.

Condensation-Net addresses this by increasing model capacity (adding more filter weights) without expanding the memory footprint of stored feature maps. This is achieved through the introduction of "virtual feature maps" and cross-channel pooling layers.

Methodology: Virtual Feature Maps and Cross-Channel Pooling

The core innovation of Condensation-Net lies in decoupling a network's learning capacity from its physical memory requirements.

Cross-Channel Pooling (Condensation)

Unlike standard pooling operations (e.g., Max Pooling) that typically operate across spatial dimensions (height and width), Condensation-Net performs pooling across the channel dimension. This allows the network to generate a significantly larger number of intermediate channels, which are then "condensed" into a smaller, manageable set of output channels.

Illustration of the cross-channel pooling mechanism where multiple channel pixels are condensed into one.

Virtual Feature Maps

The architectural breakthrough is the use of "virtual feature maps." In a standard architecture, every layer's output is written to main memory. In Condensation-Net, the hardware processes a larger set of intermediate channels sequentially. These intermediate results are fed directly into the cross-channel pooling unit without ever being stored in physical memory.

Comparison of standard network architecture versus the proposed architecture using virtual feature maps and cross-channel pooling.

By utilizing an expansion factor ($\alpha$), the network can employ $\alpha$ times more filter weights. For instance, with $\alpha=2$, the hardware computes twice the usual number of channels but pools them immediately, ensuring the memory system only handles the standard channel count.

Experimental Results

The researchers evaluated Condensation-Net on face detection tasks using the Wider Face dataset for training and the FDDB (Face Detection Data Set and Benchmark) for testing.

Performance and Efficiency

When compared against the popular Tiny-YOLOv2 benchmark:

  • Accuracy: Condensation-Net achieved a 2.0% accuracy boost for quantized networks (1-bit weights, 2-bit activations) and a 1.5% improvement for full-precision models.
  • Memory Bandwidth: The virtual feature map approach resulted in a 26.5% reduction in memory bandwidth, a critical metric for battery-powered edge devices.
  • Hardware Overhead: The additional logic required for the pooling unit is minimal, accounting for only 1.0% of the total gate count and 0.2% of the total memory cost of the accelerator.

The $\alpha$ Trade-off

While increasing the expansion factor ($\alpha$) adds capacity, the authors noted diminishing returns. Accuracy improved at $\alpha=2$, but slightly declined at $\alpha=4$, suggesting that overly aggressive condensation can discard valuable information. Additionally, while memory bandwidth is saved, computational latency increases; Condensation-Net ($\alpha=2$) takes approximately 1.3x longer to process an image compared to Tiny-YOLOv2.

Conclusion

Condensation-Net demonstrates that it is possible to break the "memory wall" in embedded AI by intelligently managing how intermediate data is processed and stored. By leveraging virtual feature maps, the architecture allows for higher-capacity models that remain compatible with the strict bandwidth limits of edge hardware. Future work will explore the application of this architecture to broader tasks such as scene segmentation and general image classification.

Summary updated

BibTeX
@inproceedings{chen2019condensation,
  author = {Tse-Wei Chen and Motoki Yoshinaga and Hongxing Gao and Wei Tao and Dongchao Wen and Junjie Liu and Kinya Osa and Masami Kato},
  title = {{Condensation-Net: Memory-Efficient Network Architecture With Cross-Channel Pooling Layers and Virtual Feature Maps}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00024},
}
Figure