Exploiting Feature Map Redundancy with Cheap Linear Transformations

34 concepts24 papers

When visualizing the feature maps of a trained deep convolutional neural network, an interesting pattern emerges: many channels look remarkably similar to one another. They are not identical, but they are highly correlated—acting almost as "ghosts" or variations of the same underlying feature.

Standard architectures generate every single one of these feature maps using expensive, full-scale convolutional filters or dense linear projections. This raises a fundamental question: if many feature maps are inherently redundant, do we really need to spend massive computational budgets calculating all of them from scratch?

A powerful line of research demonstrates that we do not. Instead of computing a massive block of features uniformly, networks can generate a small set of core, intrinsic feature maps and synthesize the remaining redundant maps using computationally cheap linear operations. This shift dramatically slashes floating-point operations (FLOPs) and parameter counts without sacrificing the network's representational capacity.

The Intrinsic-Ghost Split

The core mechanism behind this paradigm replaces a standard, heavy layer with a two-stage generation process.

  1. Intrinsic Feature Extraction: The network applies a standard, highly expressive operation (such as a conventional convolution or linear projection) to generate a limited subset of primary feature maps.
  2. Linear Augmentation: The network applies computationally inexpensive linear transformations—such as depthwise separable convolutions, shifts, or scales—directly to the intrinsic maps to generate the remaining "ghost" maps.

Finally, the intrinsic and ghost feature maps are concatenated to form the complete output volume expected by the next layer.

Input ---> [ Standard Convolution ] ---> Intrinsic Maps ---\
                                                            ---> [ Concatenate ] ---> Output
                                     ---> [ Cheap Linear ] ---> Ghost Maps -----/

As Kai Han et al. demonstrated in GhostNet: More Features From Cheap Operations, the mean squared error when reconstructing secondary feature maps from intrinsic maps using small linear kernels is extremely low. This proves that the complex, non-linear mappings typically learned by full convolutions are largely unnecessary for a significant portion of the network's channels.

This two-step formulation has been successfully adapted across diverse domains. For instance, Qihang Yang et al. leveraged this strategy in EADNet: Efficient Asymmetric Dilated Network for Semantic Segmentation to reduce the computational workload required for high-dimensional semantic representations. Similarly, Bing Liu utilized linear operations to derive redundant feature maps in the SLTM Network: Efficient Application of Lightweight Image Segmentation Technology in Detecting Drivable Areas for Unmanned Line-Marking Machines, enabling real-time deployment on low-power embedded hardware.

Why Linear Transformations Suffice

To understand why this approach works, it helps to contrast it with traditional network compression. Standard channel pruning treats redundancy as a defect, identifying less important channels and discarding them entirely. However, completely removing channels can severely degrade the network's downstream representational capacity.

Linear feature generation treats redundancy not as dead weight to be discarded, but as a structured relationship to be exploited. Instead of dropping channels, the network learns a compressed basis.

In Exploiting the Redundancy in Convolutional Filters for Parameter Reduction, Kumara Kahatapitiya et al. showed that a network can maintain high accuracy by learning a small set of independent primary filters and generating the rest as linear combinations of them. Rather than altering the network architecture, this method preserves the full dimensional space while shrinking the model's footprint by nearly 50%.

From a mathematical perspective, this works because the underlying variance of the feature space can be captured in a lower-dimensional projection. Breton L. Minnehan et al. formalized this in Cascaded Projection: End-To-End Network Compression and Acceleration, calculating optimal linear combinations across input channels to preserve maximum variance. By maintaining the full width of the feature maps through low-cost linear reconstructions, the network retains its expressive power far better than discrete pruning ever could.

Generalizing Beyond Convolutions

While initially popularized in convolutional networks for computer vision—such as the object detection frameworks optimized by Cheng Huang et al. in Application of YOLOv5 for mask detection on IoT—the principle of linear redundancy exploitation generalizes to other architectures and dynamic execution paradigms.

Transformers and Linear Projections

The massive parameter footprints of Transformer architectures are also susceptible to feature redundancy. In GhostBERT: Generate More Features with Cheap Operations for BERT, Zhiqi Huang et al. successfully integrated ghost modules into self-attention layers. By using standard linear projections for a core set of features and applying cheap 1D depthwise separable convolutions to synthesize the rest, they maintained the performance of large language models at a fraction of the computational cost.

Going a step further, Yuxin Fang et al. demonstrated in What Makes for Hierarchical Vision Transformer? that entire complex self-attention mechanisms within local windows can sometimes be replaced by simple linear mapping layers (LINMAPPER), provided the macro-architecture is preserved.

Dynamic and Input-Dependent Reconstruction

Rather than fixing the ratio of intrinsic to linear-generated features statically during training, redundancy can also be exploited dynamically at runtime. Bowen Pan et al. introduced an input-dependent policy in VA-RED2: Video Adaptive Redundancy Reduction. For each incoming video frame, the network dynamically decides which features are essential to compute fully and which can be safely skipped and reconstructed via low-cost linear operations, cutting FLOPs by up to 40% on the fly.

Ultimately, whether applied statically to a lightweight CNN or dynamically to a massive Transformer, the underlying intuition remains identical: deep neural networks naturally generate highly correlated representations, and we can bypass the heavy tax of non-linear operations by letting cheap linear transformations fill in the blanks.

Key Foundational Work

GhostNet: More Features From Cheap Operations (2020) — contributed 4 concept elements to this direction.
LCSCNet: Linear Compressing Based Skip-Connecting Network for Image Super-Resolution (2019) — contributed 1 concept elements to this direction.
Cascaded Projection: End-To-End Network Compression and Acceleration (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_feature_map_redundancy_exploitation_d7a61181,
  title = {feature map redundancy exploitation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/feature-map-redundancy-exploitation/rdr_d7a61181cfd54c3291246610a796592d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_feature_map_redundancy_exploitation_d7a61181_v1,
  title = {feature map redundancy exploitation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/feature-map-redundancy-exploitation/rdr_d7a61181cfd54c3291246610a796592d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake