Sequential Learning on Liver Tumor Boundary Semantics and Prognostic Biomarker Mining

Jieneng Chen, Ke Yan, Yu-Dong Zhang, Youbao Tang, Xun Xu, Shuwen Sun, Qiuping Liu, Lingyun Huang, Jing Xiao, Alan L. Yuille, Ya Zhang, Le Lu

Abstract

The boundary of tumors (hepatocellular carcinoma, or HCC) contains rich semantics: capsular invasion, visibility, smoothness, folding and protuberance, etc. Capsular invasion on tumor boundary has proven to be clinically correlated with the prognostic indicator, microvascular invasion (MVI). Investigating tumor boundary semantics has tremendous clinical values. In this paper, we propose the first and novel computational framework that disentangles the task into two components: spatial vertex localization and sequential semantic classification. (1) A HCC tumor segmentor is built for tumor mask boundary extraction, followed by polar transform representing the boundary with radius and angle. Vertex generator is used to produce fixed-length boundary vertices where vertex features are sampled on the corresponding spatial locations. (2) The sampled deep vertex features with positional embedding are mapped into a sequential space and decoded by a multilayer perceptron (MLP) for semantic classification. Extensive experiments on tumor capsule semantics demonstrate the effectiveness of our framework. Mining the correlation between the boundary semantics and MVI status proves the feasibility to integrate this boundary semantics as a valid HCC prognostic biomarker.

Introduction

In patients with hepatocellular carcinoma (HCC), the presence of microvascular invasion (MVI)—cancer cells invading tiny blood vessels—is a major red flag for poor prognosis and high recurrence risk. However, MVI is microscopic; it cannot be seen directly on preoperative CT or MRI scans.

Radiologists look for "proxy" features on the tumor boundary, such as whether the tumor capsule is intact or if there are small protrusions (focal extensional nodules). The challenge is that these features are highly subtle and subjective. Different radiologists often disagree on what they see (inter-reader variability), making it difficult to use these boundary semantics as reliable clinical biomarkers. This paper seeks to automate and objectify this process.

Methodology

The authors propose a two-stage framework that disentangles spatial localization from semantic classification:

Tumor Boundary Spatial Localization

The system first segments the tumor using a ResNet-50 based U-Net architecture. It extracts the boundary and applies a polar coordinate transform. By calculating the tumor's centroid and mapping the boundary relative to radius and angle, the circular boundary is effectively "unwrapped" into a 1D representation. An efficient vertex generator then samples $N$ equidistant points (vertices) along this boundary.

Deep Sequential Learning

Instead of looking at the whole image, the model samples multi-scale "pyramid" features at each vertex. These features are concatenated with positional embeddings to maintain spatial context and fed into a sequential decoder (a Multi-Layer Perceptron). The model is trained using a combination of Cross-Entropy and a 1D Sequential Dice Loss to handle class imbalance, as "normal" boundaries are significantly more common than "invaded" ones.

Data and Experiments

The study utilized a dataset of 358 unique patients (4,049 axial CT slices) with pathologically confirmed HCC. Ground-truth labels for MVI status were obtained from post-surgical histopathologic examinations.

The model was evaluated on two primary semantic features:

  • CAP (Capsular Invasion): Labeled on a three-point scale: complete, indistinct disruption, or significant disruption.
  • FEN (Focal Extensional Nodule): Labeled based on the number of protrusions into the surrounding liver tissue.
Examples of tumor capsule semantics (CAP) used for training and evaluation: (a) Class 1: complete capsule; (b) Class 2: incomplete capsule with indistinct disruption; (c) Class 3: incomplete capsule with significant disruption; (d) Combination of 3 classes.

Key Results

  • Performance Boost: On the CAP dataset, the proposed method improved the F1 score by 23.26% over the 2D segmentation baseline. On the FEN dataset, it saw a 10.32% improvement.
  • MVI Prediction: Using the model's predicted boundary semantics to predict MVI resulted in an AUC of 70.25%, outperforming the "Upper Bound" (69.36%) calculated from manual ground-truth annotations.
  • Robustness: The model proved robust against slight shifts in boundary localization thanks to the use of multi-scale pyramid features, which capture both local textures and broader context.

Discussion

The core innovation of this work is treating the tumor boundary as a 1D sequence. Standard 2D segmentation often "smears" the fine details of an edge. By unwrapping the edge, the model can focus specifically on the transition of semantics along the perimeter. By separating the task of finding the boundary from the task of describing it, the model avoids the "entangled optimization" problem where a network struggles to do both simultaneously.

The study found that the model's performance was comparable to a radiologist with 10 years of experience. More importantly, the model's predicted biomarkers correlated better with actual MVI status than the manual annotations provided by humans, suggesting it can surpass human consistency in identifying these subtle features.

Conclusion and Future Work

While 358 patients is a solid start for medical imaging, larger multi-center studies are needed to validate the generalizability of the biomarker. The current approach processes CT scans slice-by-slice (2D). Future work could extend this to a true 3D surface sequential learning model to capture even more prognostic information. Additionally, combining these features with MRI or liquid biopsy data could further improve MVI prediction accuracy.

Summary updated

Abstract

The boundary of tumors (hepatocellular carcinoma, or HCC) contains rich semantics: capsular invasion, visibility, smoothness, folding and protuberance, etc. Capsular invasion on tumor boundary has proven to be clinically correlated with the prognostic indicator, microvascular invasion (MVI). Investigating tumor boundary semantics has tremendous clinical values. In this paper, we propose the first and novel computational framework that disentangles the task into two components: spatial vertex localization and sequential semantic classification. (1) A HCC tumor segmentor is built for tumor mask boundary extraction, followed by polar transform representing the boundary with radius and angle. Vertex generator is used to produce fixed-length boundary vertices where vertex features are sampled on the corresponding spatial locations. (2) The sampled deep vertex features with positional embedding are mapped into a sequential space and decoded by a multilayer perceptron (MLP) for semantic classification. Extensive experiments on tumor capsule semantics demonstrate the effectiveness of our framework. Mining the correlation between the boundary semantics and MVI status proves the feasibility to integrate this boundary semantics as a valid HCC prognostic biomarker.
BibTeX
@inproceedings{chen2021sequential,
  author = {Jieneng Chen and Ke Yan and Yu-Dong Zhang and Youbao Tang and Xun Xu and Shuwen Sun and Qiuping Liu and Lingyun Huang and Jing Xiao and Alan L. Yuille and Ya Zhang and Le Lu},
  title = {{Sequential Learning on Liver Tumor Boundary Semantics and Prognostic Biomarker Mining}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87234-2\_72},
}
Figure