Efficient Spatial Reasoning via Asymmetric Convolutional Factorization

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 277 papers102 conceptsfrozen 2026-10-04

In the standard toolkit of computer vision, the square $N \times N$ convolutional kernel is the undisputed workhorse. From the early days of AlexNet to modern ResNets, the $3 \times 3$ filter has been treated as the optimal balance between local feature extraction and computational cost. However, as we push toward higher resolutions and real-time edge deployment, the quadratic growth of square kernels ($N^2$) becomes a bottleneck.

A recurring insight across recent research is that square kernels are often redundant. By factorizing a single 2D convolution into a sequence or parallel combination of 1D asymmetric kernels—such as $1 \times N$ and $N \times 1$—we can maintain the same receptive field while drastically reducing parameters and operations.

The Geometry of Factorization

The intuition behind asymmetric convolution is rooted in linear algebra: many 2D filters are "separable." Just as a matrix can sometimes be decomposed into the outer product of two vectors, a square convolutional operation can be approximated by two orthogonal 1D passes.

Christian Szegedy and colleagues famously demonstrated this in Rethinking the Inception Architecture for Computer Vision. They showed that a $3 \times 3$ convolution (9 parameters) could be replaced by a $1 \times 3$ followed by a $3 \times 1$ convolution (6 parameters). This 33% reduction in cost comes with a hidden benefit: by placing a non-linear activation (like ReLU) between the two 1D kernels, the network gains additional expressive power without increasing the parameter count.

This structural refinement is not just about saving memory; it changes how the network "sees." In MEA-Net, Huilin Liu and others use these asymmetric blocks to sharpen boundaries in medical images. Because edges are inherently directional, $1 \times N$ strips are naturally suited to capturing the linear characteristics of object boundaries more efficiently than isotropic square filters.

Scaling the Receptive Field

One of the most significant challenges in modern vision tasks like semantic segmentation is the need for a "global" view. To understand a scene, a pixel needs to know about other pixels far away. Traditionally, this required stacking dozens of layers or using computationally expensive Transformers.

Asymmetric factorization offers a "cheat code" for expanding the receptive field. In SegNeXt, Meng-Hao Guo and Qibin Hou introduce Multi-Scale Convolutional Attention (MSCA), which decomposes large kernels into pairs of orthogonal strips. This allows the network to capture long-range spatial dependencies with linear rather than quadratic complexity.

This idea reaches its extreme in SLaK, where Shiwei Liu and Tianlong Chen 0001 scale kernels up to $51 \times 51$. A square $51 \times 51$ kernel would have 2,601 weights per channel—a computational nightmare. By summing parallel $51 \times 5$ and $5 \times 51$ kernels, SLaK achieves the same massive "view" with a fraction of the overhead, allowing pure ConvNets to compete with the global attention mechanisms of Vision Transformers.

Aligning Kernels with Data Structure

While square kernels assume that horizontal and vertical information are equally important, many data types are inherently asymmetric.

  • Scene Text: In SVTR, Yongkun Du and colleagues leverage the fact that text is usually horizontal. They progressively reduce the vertical dimension of feature maps while preserving horizontal resolution, using asymmetric strides to balance speed and character recognition.
  • Audio and Speech: Tianchi Liu argues in Golden Gemini that spectrograms should not be treated like photos. Time and frequency are different physical dimensions; using asymmetric strides and kernels that respect these differences leads to more accurate speaker verification.
  • Remote Sensing: For UAV and satellite imagery, Tao Zhang shows that parallel horizontal and vertical branches prevent "order bias" and make models more robust to image rotations, which is critical when the "up" direction is arbitrary.

The Implementation Gap

If asymmetric convolutions are so efficient, why aren't they the universal default? The answer lies in hardware optimization. Most deep learning libraries (like cuDNN) are heavily optimized for $3 \times 3$ and $1 \times 1$ square kernels.

Wei Wen and Shuai Zhang point out in their work on Trained Rank Pruning that while asymmetric kernels reduce the theoretical number of operations (FLOPs), they often suffer from poor library support. On some hardware, a "heavier" $3 \times 3$ convolution might actually run faster than a "lighter" $1 \times 7$ kernel because the hardware is specifically tuned for square memory access patterns.

Despite these engineering hurdles, the trend toward Inception-style decomposition—seen in InceptionNeXt—suggests that the future of efficient vision lies in breaking the square. By treating the horizontal and vertical axes as distinct but complementary paths, researchers are building models that are not only smaller but more attuned to the structural realities of the visual world.

Papers in this version

Showing 60 of 77 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 36 papers
  2. Version 1 · 2026-10-04 · 77 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_asymmetric_convolutional_decomposition_b904e9c8,
  title = {asymmetric convolutional decomposition},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-convolutional-decomposition/rdr_b904e9c82d7d4eb693cb1ce11437b790},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_efficient_spatial_reasoning_via_asymmetr_b904e9c8_v1,
  title = {Efficient Spatial Reasoning via Asymmetric Convolutional Factorization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/efficient-spatial-reasoning-via-asymmetric-convolutional-factorization/rdr_b904e9c82d7d4eb693cb1ce11437b790/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}