Adaptive Weighting Multi-Field-of-View CNN for Semantic Segmentation in Pathology

Hiroki Tokunaga, Yuki Teramoto, Akihiko Yoshizawa, Ryoma Bise

Abstract

Accurate identification of cancer subtypes in digital pathology is critical for diagnosis and treatment planning. Pathologists typically navigate between low magnification to assess tissue architecture and high magnification to examine cellular morphology. Standard convolutional neural networks (CNNs) often struggle with the inherent trade-off between a wide field-of-view and high resolution. This paper introduces the Adaptive-Weighting-Multi-Field-of-View CNN (AWMF-CNN), a novel architecture that adaptively fuses features from multiple magnifications (5x, 10x, and 20x). By mimicking the multi-scale diagnostic process of human experts, AWMF-CNN significantly improves semantic segmentation performance in complex lung cancer datasets.

Introduction

In digital pathology, the classification of lung adenocarcinoma subtypes—such as Lepidic, Acinar, Papillary, Micropapillary, and Solid—is a high-stakes task. A fundamental challenge in applying deep learning to this domain is the resolution-context trade-off. A wide field-of-view (FOV) provides essential structural context but lacks the fine-grained resolution needed to identify individual cell shapes. Conversely, high-resolution patches offer cellular detail but lose the surrounding tissue architecture.

The trade-off between field of view and resolution across different lung cancer subtypes at 5x, 10x, and 20x magnifications.

Existing multi-scale models often treat all magnifications with equal importance. However, the optimal magnification for identification varies by subtype; for instance, some patterns are distinguishable at 5x, while others require 20x zoom. There is a clear need for a model that can dynamically prioritize different scales based on the visual content of the tissue.

Methods

The proposed Adaptive-Weighting-Multi-Field-of-View CNN (AWMF-CNN) architecture consists of three integrated components designed to process and fuse multi-scale information effectively.

1. Expert CNNs

The model employs three separate U-Net-based "Expert" CNNs ($f_{E_1}, f_{E_2}, f_{E_3}$), each trained on a specific magnification: 5x, 10x, and 20x. Each expert generates a probability map for the target cancer subtypes based on its specific field of view.

2. Weighting CNN

To determine the relative importance of each expert for a given input, a Weighting CNN ($f_W$) based on a modified Xception network is utilized. It analyzes the input image and predicts a set of weights ($w_1, w_2, w_3$) that represent the model's confidence in each expert's scale for that specific patch.

3. Aggregating CNN

The final component, the Aggregating CNN ($f_A$), takes the weighted outputs from the experts and fuses them. This stage involves concatenation and a series of convolutional layers to produce a single, high-resolution semantic segmentation mask.

Architecture of the Adaptive-Weighting-Multi-Field-of-View CNN (AWMF-CNN) featuring Expert CNNs, a Weighting CNN, and an Aggregating CNN.

Training Strategy

The model is trained end-to-end. Initially, the Expert CNNs are pre-trained independently. Subsequently, the weighting network is trained using the Dice coefficient—a measure of spatial overlap—between each expert's prediction and the ground truth. This process teaches the weighting network to recognize which magnification is most reliable for different visual patterns and cancer subtypes.

Experimental Results

The study utilized 29 Whole-Slide Images (WSIs) of lung adenocarcinoma, extracting over 160,000 patches for training. The model was evaluated on its ability to segment five regions: Normal, Lepidic, Acinar/Papillary, Micropapillary, and Solid.

Performance Gains

AWMF-CNN outperformed several state-of-the-art models, including standard U-Net, SegNet, and DeepLabv3+. In the challenging multi-subtype segmentation task, AWMF-CNN achieved a Mean Intersection over Union (mIoU) of 0.536, a significant improvement over the best single-scale model (DeepLabv3+ at 5x, which scored 0.433).

Complementary Expertise

Analysis of the results showed that the "union" of correct identifications from the three experts was significantly larger than the performance of any single expert. This confirms that the training process successfully forces the experts to specialize in different features. For example, wide-field features (5x) were found to be more discriminative for Acinar/Papillary and Lepidic subtypes, while high-resolution features (20x) were superior for identifying Solid regions.

Discussion and Conclusion

The AWMF-CNN framework demonstrates that adaptive weighting of multi-scale features provides a significant edge over fixed-weighting or single-scale approaches. By dynamically adjusting to the visual characteristics of different cancer subtypes, the model achieves higher robustness and accuracy in semantic segmentation.

Future work may explore pixel-wise local weighting to handle patches containing multiple subtypes more effectively. Additionally, testing the framework on larger, more diverse multi-center datasets will be essential to ensure its generalizability across different laboratory settings and staining variations. Despite the increased computational cost of running multiple expert streams, the performance gains suggest that AWMF-CNN is a promising step toward automated, high-precision digital pathology.

Summary updated

BibTeX
@misc{tokunaga2019adaptive,
  author = {Hiroki Tokunaga and Yuki Teramoto and Akihiko Yoshizawa and Ryoma Bise},
  title = {{Adaptive Weighting Multi-Field-of-View CNN for Semantic Segmentation in Pathology}},
  year = {2019},
  eprint = {1904.06040},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1904.06040},
}
Figure