Abstract
In medical imaging, particularly diabetic retinopathy screening, detecting tiny lesions in high-resolution fundus images is critical. Standard Convolutional Neural Networks (CNNs) often fail because their downsampling layers—typically Many-to-One Reassembly of Features (M2ORF)—cause small, clustered signals to vanish. We present Many-to-Many Reassembly of Features (M2MRF), a novel architectural component that maps large input regions to multiple output features simultaneously. This approach preserves subtle activations and captures long-range spatial dependencies, significantly improving the segmentation of microaneurysms and other minute pathologies.
Introduction: The Vanishing Signal Problem
The primary challenge in fundus image analysis is extreme scale variation. Lesions such as microaneurysms (MAs) can occupy less than 0.001% of the total pixels in a multi-megapixel image.
Traditional CNN architectures rely on downsampling operators like pooling or strided convolutions. These operators follow a Many-to-One paradigm, squashing local neighborhoods into single feature points. When multiple tiny lesions are clustered together, this many-to-one mapping often averages them into the background, leading to a permanent loss of information early in the encoding process. Once these signals "vanish," even sophisticated decoders cannot recover them.
Methodology: Many-to-Many Reassembly
To address the downsampling bottleneck, the authors propose M2MRF. Unlike standard operators that treat each target location independently, M2MRF processes a large predefined region to generate multiple target features at once.
Key Architectural Innovations
- Many-to-Many Projection: Instead of a standard convolution, M2MRF uses a linear projection mapping $N$ source features to $M$ target features. This allows clustered lesions to reinforce each other's signals through mutual activation enhancement.
- Matrix Decomposition: To prevent the parameter explosion associated with large-region projections, the weight matrix is decomposed into two smaller, successive linear projections. This bottleneck structure enables a large receptive field with high efficiency.
- Integration: M2MRF was integrated into the HRNetV2 backbone, which is designed for high-resolution representation maintenance. The authors explored both a "Cascade" version (gradual downsampling) and a "One-step" version (direct jump to target resolution).
Experimental Results
The model was evaluated on two major benchmarks: the DDR dataset and the high-resolution IDRiD dataset.
- Quantitative Performance: On the DDR dataset, the one-step M2MRF-A improved the mean Area Under Precision-Recall curve (mAUPR) from a 45.21% baseline to 49.94%.
- Comparison with Transformers: M2MRF outperformed modern vision transformers like Swin-Transformer and Twins-SVT on the IDRiD benchmark. It was particularly effective at detecting Microaneurysms (MAs), the smallest and most difficult lesions to segment.
- Efficiency: Despite the increased receptive field, the matrix decomposition strategy allowed M2MRF to remain more parameter-efficient than existing learning-based operators like CARAFE++.
Conclusion
M2MRF shifts the focus of medical image segmentation from decoder-side upsampling to preserving information during the initial downsampling stages. By adopting a Many-to-Many philosophy, the model prevents the "original sin" of signal washout. While currently optimized for fundus images, this approach holds significant potential for other domains characterized by tiny, clustered objects, such as satellite imagery and microscopy.
Summary updated