Cross-channel Communication Networks

Jianwei Yang, Zhile Ren, Chuang Gan, Hongyuan Zhu, Devi ParikhNeurIPS 2019 · OpenReview

Standard Convolutional Neural Networks (CNNs) treat filters within a layer as independent agents, leading to redundant information and inefficient feature representation. To address this, researchers have introduced the Cross-channel Communication (C3) block, a graph-inspired module that allows filters to "talk" to one another. This intra-layer communication results in more diverse features and better performance across classification, detection, and segmentation tasks.

The Problem: Isolated Filters

In a traditional CNN, each filter at a specific layer looks at the input and generates a response map independently. While these responses are combined in the next layer, they do not interact within the same layer. This lack of communication often leads to "redundancy"—multiple filters capturing nearly identical information because they have no way to coordinate.

Previous attempts to fix this, like the popular Squeeze-and-Excitation (SE) blocks, compress each channel into a single number (a scalar) to weight them. The authors argue this is like trying to have a complex conversation through a tiny straw; the "bandwidth" for communication is simply too limited.

The Solution: Cross-Channel Communication (C3)

The C3 block aims to provide a richer, high-bandwidth channel for filters to exchange information and calibrate their responses based on what their "neighbors" are seeing. It acts as a "micro neural network" inserted into existing architectures (like ResNet).

Comparison of Squeeze-and-Excitation vs. Cross-Channel Communication blocks.

It operates in three distinct phases:

  1. Feature Encoding: Instead of squeezing a channel into a single number, C3 flattens the response map and passes it through a small multi-layer perceptron (MLP) with a bottleneck. This extracts a high-level "summary" of what that specific filter detected.
  2. Message Passing: This is the core innovation. The module treats each channel as a node in a complete graph. It calculates the similarity between channels and uses a soft-attention mechanism to pass "messages" between them. Essentially, each channel updates its own information by looking at what all other channels have found.
  3. Feature Decoding: The updated messages are passed through another MLP to "decode" them back into the original feature space, which are then added back to the original input via a residual connection.

By using this encoder-message-decoder structure, the network learns which channels should group together and which should differentiate, all without losing the spatial information that simple "squeezing" discards.

Key Insights and Results

The authors validated the C3 block across several industry-standard benchmarks, including ImageNet, CIFAR-100, MS COCO, and Pascal VOC 2012. The results highlighted several major advantages:

  • Reducing Redundancy: By calculating the correlation between channels, the authors proved that C3 blocks actually make filters more unique. Lower correlation means the network is learning a more diverse set of features.
  • Shallow is the New Deep: One of the most striking findings was that a shallower network equipped with C3 blocks could match or outperform a much deeper baseline. For instance, a ResNet-74 equipped with C3 blocks outperformed a standard ResNet-110 baseline, while requiring significantly fewer parameters (1.22M vs 1.74M).
Classification accuracy showing ResNet-74 with C3 outperforming a deeper ResNet-110 baseline.
  • Placement Matters: The authors found that adding C3 blocks to the later layers of a network is more effective than adding them to the early layers. Early layers capture simple textures, while later layers capture complex semantics where "coordination" between parts (like eyes, ears, and fur for a "cat" label) is more valuable.
  • Superior Performance: The C3 block consistently outperformed both the baseline models and the SE-block counterparts. On ImageNet, a ResNet-101 with C3 achieved a 21.93% top-1 error, beating both the baseline (22.48%) and the SE-enhanced version (22.14%). Similar jumps were seen in COCO Detection (34.8 mAP vs 33.9 mAP).
  • Holistic Object Focus: Using Class Activation Maps (CAM), the authors showed that C3-equipped models focus on the entire object more effectively, whereas baseline models often fixate on small, highly discriminative patches.

Limitations and Future Work

While the C3 block is efficient and independent of the number of channels, it still introduces more parameters than the ultra-lightweight SE block. Furthermore, the authors only added C3 blocks to a few separate layers to keep complexity down; finding the absolute "optimal" placement for these blocks remains an open question. Future work could explore more sophisticated graph architectures or adaptive message-passing schemes that dynamically change based on the input image.

Summary updated

Abstract

Convolutional neural networks process input data by sending channel-wise feature response maps to subsequent layers. While a lot of progress has been made by making networks deeper, information from each channel can only be propagated from lower levels to higher levels in a hierarchical feed-forward manner. When viewing each filter in the convolutional layer as a neuron, those neurons are not communicating explicitly within each layer in CNNs. We introduce a novel network unit called Cross-channel Communication (C3) block, a simple yet effective module to encourage the neuron communication within the same layer. The C3 block enables neurons to exchange information through a micro neural network, which consists of a feature encoder, a message communicator, and a feature decoder, before sending the information to the next layer. With C3 block, each neuron accounts for the channel-wise responses from other neurons at the same layer and learns more discriminative and complementary representations. Extensive experiments for multiple computer vision tasks show that our proposed mechanism allows shallower networks to aggregate useful information within each layer, and performances outperform baseline deep networks and other competitive methods.
BibTeX
@inproceedings{yang2019cross,
  author = {Jianwei Yang and Zhile Ren and Chuang Gan and Hongyuan Zhu and Devi Parikh},
  title = {{Cross-channel Communication Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=1UzAYTO4Nt},
}

Loading related content...

Figure