Robust Vision via Spatially Restricted Receptive Fields

This direction is split into other directions: receptive field restriction (receptive field size vulnerability). Its last version stays readable and citable here.

Version 1 of 119 papers26 conceptsfrozen 2026-10-04

In standard convolutional neural networks (CNNs), the "receptive field"—the area of the input image that influences a specific neuron—grows rapidly as we go deeper into the network. While this allows models to capture global context, it creates a significant security and reliability flaw: a small, localized corruption can "bleed" across the entire feature map, potentially ruining the final prediction.

Researchers have converged on a powerful architectural counter-measure: Receptive Field Restriction. By physically limiting how far information can propagate spatially, we can "quarantine" noise, occlusions, and adversarial attacks within a small subset of the network’s internal representations.

The Problem of Global Corruption

Standard vision models are surprisingly fragile. As Anurag Ranjan and colleagues demonstrated in Attacking Optical Flow, a tiny adversarial sticker—less than 1% of the image size—can cause a network to see chaotic, non-existent motion across the entire frame.

This happens because the network's layers are highly interconnected. A single "bad" pixel in the input eventually influences almost every neuron in the final layer. This "Achilles' heel" was identified early on by Yichuan Tang and Chris Eliasmith, who argued in Deep networks for robust visual recognition that restricting hidden unit connectivity to local patches is essential for maintaining the integrity of global representations when faced with noise.

Quarantining Features with Local Receptive Fields

The core intuition behind this research direction is spatial isolation. If we ensure that each feature in our final map only "sees" a small 5x5 or 10x10 window of the original image, then a localized adversarial patch can only corrupt a fixed, predictable number of features.

This mechanism is the backbone of several "certifiably robust" defenses:

  1. Bounding Influence: Arjun Bhagoji, Chong Xiang, and Prateek Mittal introduced PatchGuard (PatchGuard: Provable Defense against Adversarial Patches Using Masks on Small Receptive Fields), which uses small receptive fields to bound the spatial influence of a patch. Because the number of corrupted features is limited, they can apply a "robust mask" to suppress anomalous feature spikes.
  2. Voting and Aggregation: In Efficient Certified Defenses Against Patch Attacks on Image Classifiers, Jan Hendrik Metzen and Maksym Yatsura show that by restricting layers to local regions, the network effectively becomes a "bag" of local classifiers. Even if a patch tricks a few local "votes," the majority of the clean image regions will still provide the correct classification.
  3. Handling Occlusions: This isn't just for security. Markus Oberweger and Vincent Lepetit applied this to 3D pose estimation in Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation. By using independent image patches, they prevent the distortion caused by an occluding object (like a hand covering a tool) from propagating through the entire feature map.

Forcing a Focus on Low-Level Anomalies

Restricting the receptive field also changes what the model learns. By stripping away global context, the model is forced to focus on local textures and structural details.

This is particularly useful for deepfake detection. Bar Cavia and Yedid Hoshen developed LaDeDa (Real-Time Deepfake Detection in the Real-World), which looks only at tiny 9x9 patches. By ignoring high-level semantic features (like the overall "look" of a face), the model becomes much better at spotting the low-level artifacts and structural anomalies left behind by generative AI.

Similarly, in Sparling: End-to-End Spatial Concept Learning via Extremely Sparse Activations, Kavi Gupta and colleagues use locality constraints to ensure that learned "concepts" (like a specific motif in an image) are truly local and interpretable, preventing the model from using global "shortcuts" to solve a task.

The Cost of Robustness

While restricting the receptive field provides mathematical guarantees and robustness, it comes with a trade-off. Zeyu Gao and Amin Saremi highlight in their analysis of Certifiable Defenses against Adversarial Patch Attacks that these defenses often face an "Accuracy-Latency Pareto frontier."

Because these models often require specialized architectures or masking layers to maintain their "certified" status, they can be computationally expensive. However, as Chong Xiang notes in PatchGuard++, moving these operations from pixel space to feature space can significantly reduce overhead, making these "quarantined" networks viable for real-time use in autonomous systems.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_vision_via_spatially_restricted_r_d8753205,
  title = {Robust Vision via Spatially Restricted Receptive Fields},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-vision-via-spatially-restricted-receptive-fields/rdr_d8753205bb604030b213a008d4a91f5d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_vision_via_spatially_restricted_r_d8753205_v1,
  title = {Robust Vision via Spatially Restricted Receptive Fields},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-vision-via-spatially-restricted-receptive-fields/rdr_d8753205bb604030b213a008d4a91f5d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}