Abstract
Graph-based convolutional models, such as non-local blocks, are effective for strengthening context modeling in convolutional neural networks (CNNs). However, their pixel-wise computational overhead is prohibitive for high-resolution imagery. We propose Squeeze Reasoning (SR), a framework that shifts reasoning from the spatial map to a channel-wise global vector. By building a node graph where each node represents an abstract semantic concept, we significantly reduce computational costs while maintaining consistency within semantic categories. Our approach is modular, end-to-end trainable, and achieves state-of-the-art results across semantic segmentation, object detection, and panoptic segmentation tasks with minimal overhead.
Introduction: The Context Problem
In computer vision, "context" is essential for scene understanding—knowing how a car relates to the road or how a sidewalk relates to a building. Standard Convolutional Neural Networks (CNNs) have a limited receptive field, often failing to capture long-range dependencies. While "Non-local" blocks or self-attention mechanisms address this by allowing every pixel to interact, they suffer from a computational cost that grows quadratically ($O(N^2)$) with the number of pixels. For high-resolution images common in autonomous driving, this overhead is prohibitive. The authors propose Squeeze Reasoning to capture global context without the pixel-wise price tag.
Methodology: Squeeze Reasoning
The Squeeze Reasoning (SR) module consists of three main steps designed to abstract spatial information into semantic nodes:
- Node Squeezing: Instead of looking at every pixel, the model "squeezes" the entire spatial feature map into a single global vector. This can be achieved via standard Global Average Pooling (GAP) or Global Hadamard Pooling (GHP), which captures second-order statistics to better preserve complex relations.
- Node Graph Reasoning: The squeezed vector is divided into $K$ groups, where each group acts as a "node" representing an abstract semantic concept (e.g., "vegetation" or "vehicles"). The model then uses a Graph Convolutional Network (GCN) or a self-attention mechanism to perform reasoning between these nodes. Because there are only a few nodes (e.g., 16 or 32) compared to thousands of pixels, this step is incredibly efficient.
- Feature Reconstruction: The refined, "reasoned" nodes are projected back and used to re-weight the original feature map, effectively broadcasting global insights back to the spatial pixels.
Key Insight: Semantic Sparsity
A key discovery in this research is that different semantic classes (like cars vs. vegetation) naturally activate different groups of channels sparsely. This confirms that reasoning in the channel/node space is a valid and efficient proxy for reasoning in the spatial space. By treating groups of channels as "semantic nodes," the model performs reasoning at a higher level of abstraction, bypassing the expensive $O(N^2)$ pixel-affinity matrices.
Experimental Results
The SR module was evaluated across several diverse benchmarks, including Cityscapes, ADE20K, and MS COCO:
- Accuracy vs. Efficiency: On Cityscapes, adding the SR head to a dilated FCN baseline improved mIoU from 74.8% to 79.9%, while only adding 1.5% to the total computational cost.
- Real-time Performance: SRNet achieved 77.5% mIoU on Cityscapes while running at 65 FPS on a single 1080-Ti GPU, outperforming many specialized real-time models.
- Generalization: On MS COCO, the SR module consistently outperformed Non-local blocks across object detection, instance segmentation, and panoptic segmentation tasks, all while using significantly less memory and computation.
Conclusion
Squeeze Reasoning offers a lightweight, modular framework for efficient scene understanding. By shifting the reasoning process to a compressed semantic space, it enables high-performance context modeling at a fraction of the cost of traditional methods. Its plug-and-play nature makes it easily adaptable to existing architectures for a wide range of downstream vision tasks.
Summary updated