In the broader field of computer vision, the "Transformer takeover" is nearly complete. Vision Transformers (ViTs) have largely displaced Convolutional Neural Networks (CNNs) as the state-of-the-art for general-purpose image recognition. However, in specialized domains—ranging from medical diagnostics to microscopic grain inspection—the CNN remains remarkably resilient.
Research across these fields suggests that the "death of the convolution" has been greatly exaggerated. In fact, for tasks involving high-resolution spatial hierarchies, limited data, or fine-grained local features, the inherent inductive biases of convolutional architectures often make them superior to the global attention mechanisms of Transformers.
The Power of Locality
The fundamental difference between a CNN and a Transformer lies in their assumptions about the data. A Transformer’s self-attention mechanism is "global": it treats every pixel (or patch) as potentially related to every other pixel, regardless of distance. While powerful, this requires the model to learn the concept of spatial proximity from scratch.
In contrast, CNNs possess a local inductive bias. They assume that pixels near each other are more likely to be related than pixels far apart. As Michael F. Romano and colleagues demonstrate in Deep learning-driven risk-based subtyping of cognitively impaired individuals, this bias aligns perfectly with the spatial hierarchies of 3D anatomical data. When identifying brain atrophy patterns, the relationship between neighboring voxels is more informative than the relationship between distant regions of the brain.
This focus on local morphology is particularly critical in histology. Eric Q. Wu and the team behind ROSIE found that convolutional architectures excel at predicting molecular features from small tissue patches because they prioritize the local cellular structures that define disease states.
Data Efficiency and the "Baked-In" Prior
Transformers are notoriously "data-hungry." Because they lack inherent spatial priors, they require massive datasets (like ImageNet-21k or JFT-300M) to learn how images are structured. In specialized fields, such as medical imaging or agriculture, datasets are often small and expensive to curate.
Adrián Galdrán and Gustavo Carneiro highlight this in Convolutional Nets Versus Vision Transformers for Diabetic Foot Ulcer Classification. They argue that CNNs outperform ViTs on limited datasets because their spatial biases—local connectivity and translation invariance—allow them to learn relevant features more efficiently. Similarly, Maximilian Springenberg and Annika Frommholz found that while ViTs can be competitive, modern CNNs like ConvNeXt maintain a performance advantage when trained from scratch or on domain-specific data.
Precise Localization vs. Blurry Attention
A recurring problem with Transformers in clinical settings is their tendency to produce broad, "blurry" activation maps. For tasks like lesion detection or organ segmentation, precision is paramount.
In nnMobileNet: Rethinking CNN for Retinopathy Research, Peijie Qiu and Yalin Wang demonstrate that convolutional receptive fields enable much more precise localization of small-scale features than patch-based self-attention. This is echoed by Xiangde Luo in studies of abdominal organ segmentation and lymph node delineation, where CNNs consistently outperformed Transformers in capturing the specific anatomical boundaries necessary for radiotherapy planning.
The intuition is simple: if you are looking for a tiny micro-aneurysm in a retinal scan, a model that naturally focuses on local neighborhoods (CNN) is less likely to lose that detail than a model that averages information across large image patches (Transformer).
Beyond Pixels: Structural Priors in Other Domains
The superiority of convolutional biases isn't limited to human biology. In GrainSpace, Lei Fan shows that CNNs are better suited for detecting microscopic surface defects in cereal grains. Even in non-visual domains like network security, Xuying Meng and colleagues found that NetConv, a convolutional architecture, outperformed Transformers in encrypted traffic classification. This is because localized byte patterns in protocol fields contain the most critical information—a "spatial" relationship that convolutions capture more efficiently than global self-attention.
Hybridizing the Best of Both Worlds
The most recent research direction isn't necessarily about choosing one over the other, but rather bringing Transformer-style training to convolutional architectures. Xin Li and Peijie Qiu recently introduced a BERT-style self-supervised learning framework for CNNs. By using sparse convolutions to perform masked image modeling, they achieved state-of-the-art results in identifying Alzheimer’s from retinal images.
This suggests a new synthesis: using the powerful self-supervised training techniques that made Transformers famous, but applying them to the convolutional backbone to preserve the local inductive biases that specialized imaging tasks demand. As Ju-Hyeon Nam notes in MADGNet, these architectural assumptions help maintain structural consistency in clinical environments where visual boundaries are noisy or poorly defined.