Bland–Altman agreement analysis between CT predicted and surgical peritoneal cancer index in pseudomyxoma peritonei of appendiceal origin

Mingjian Bai, Jingliang Chen, Yueming Xu, Jing Feng, Ruiqing Ma, Hongmin Jia, Hongbin Xu, Guowei Liang, Hongjiang WeiScientific 2023 · DOI

TL;DR

While CT scans are the go-to tool for mapping the spread of rare abdominal cancers like Pseudomyxoma Peritonei (PMP), this study reveals a critical gap between imaging and reality. Despite a high statistical correlation, CT-predicted scores often fail the "agreement test" during surgery, frequently misidentifying tumor boundaries or missing small lesions in the intestines.

What Problem Does This Paper Address?

In the world of surgical oncology, the Peritoneal Cancer Index (PCI) is the gold standard for quantifying how far a cancer has spread throughout the abdominal cavity. For patients with Pseudomyxoma Peritonei (PMP)—a rare "jelly belly" cancer usually originating in the appendix—the PCI score determines whether a patient is a candidate for aggressive cytoreductive surgery (CRS) combined with heated chemotherapy (HIPEC).

The dilemma is that a true PCI can only be determined during an invasive laparotomy (opening the abdomen). Surgeons rely on preoperative CT scans to estimate this score, but how accurate are these estimates? Most previous research used simple correlation statistics, which can be misleading. This paper uses a more rigorous method—Bland–Altman agreement analysis—to see if CT scores are precise enough to guide surgical decision-making.

Methods

The researchers conducted a retrospective study of 167 PMP patients at a specialized center in China. They compared two sets of data:

  1. CT-PCI: Scores calculated by an experienced radiologist using enhanced CT scans (utilizing both oral and intravenous contrast).
  2. Surgical PCI: The "gold standard" scores recorded by surgeons during the actual operation.

The team didn't just look at the total score (0–39); they also analyzed a "selected PCI" focusing on the small intestine and hepatic hilum—areas notoriously difficult to image and surgically resect.

The statistical heavy lifting was done via Bland–Altman plots. Unlike standard correlation (which just shows if two variables move together), Bland–Altman analysis measures the agreement between two methods by plotting the difference between measurements against their average.

Datasets

The study utilized a clinical dataset from the EpiData database at the Aerospace Center Hospital, covering patients treated between 2016 and 2021.

  • Initial cohort: 371 PMP patients.
  • Final study group: 167 patients (after excluding those with non-appendiceal origins, prior systemic chemotherapy, or incomplete imaging).

Key Insights and Ideas

  • Correlation $\neq$ Agreement: This is the paper's most important "aha moment." The researchers found a Spearman correlation coefficient of over 0.9 (very strong). However, the Bland–Altman analysis showed that the "Limits of Agreement" were too wide for clinical comfort. In plain English: the scores were related, but they weren't the same.
  • The "Mucus Problem": PMP is characterized by massive amounts of mucinous ascites (jelly-like fluid). CT scans struggle to distinguish the borderline between actual tumor tissue and this mucus, often leading to an overestimation of the cancer's extent.
  • The "Small Bowel Blind Spot": Conversely, CT often underestimates cancer on the small intestine. Lesions smaller than 0.5 cm are incredibly difficult to spot on a scan but are easily identified by a surgeon’s touch and sight.
  • Clinical Significance: In practice, a discrepancy of $\pm$ 5–7 PCI points can completely change a surgical plan. The study found that CT-PCI often fell outside this acceptable range, meaning it shouldn't be the sole factor in deciding whether to operate.

Key Results

  • Total PCI Agreement: The mean bias was small (0.431), but the 95% Limits of Agreement (LoA) ranged from -5.459 to 6.321. This spread is wide enough to impact clinical decisions.
  • Outliers: About 5.39% of total PCI cases and 5.99% of selected PCI cases fell entirely outside the statistical limits of agreement.
  • Predictive Power: Despite the agreement issues, CT was still "excellent" at predicting overall resectability (whether the tumor could be fully removed), with an Area Under the Curve (AUC) of 0.952. This suggests CT is great for a "yes/no" surgical screen, even if it struggles with the exact "how much" score.

Limitations and Future Work

The authors acknowledge several hurdles:

  1. Single-Center Bias: The data comes from one specialized hospital and a single experienced radiologist. Results might vary in general hospitals with less PMP-specific expertise.
  2. Selection Bias: 87 patients were excluded because their CTs were performed elsewhere, potentially skewing the data toward "ideal" imaging conditions.
  3. The Path Forward: The authors suggest that CT alone isn't enough. Future work should focus on multiple linear regression models that combine CT-PCI with other biomarkers or clinical predictors to create a more accurate preoperative "map" for surgeons.

Summary updated

Abstract

Peritoneal cancer index (PCI) is the surgical variable most commonly used to quantify the extent of peritoneal metastases for pseudomyxoma peritonei (PMP) patients. The present study aimed to investigate the agreement between CT predicted and surgical PCI by the Bland-Altman method for PMP of appendiceal origin. A total of 167 PMP patients of appendiceal origin were included between 2016 and 2021. Bland-Altman analysis was performed for both total PCI and selected PCI (regions 2 + 9-12). After the Bland-Altman plot was drawn, the mean bias and its 95% limit of agreements (LoAs) was quantified. Besides, the correlation coefficients between CT-PCI and surgical PCI were also been calculated. The Bland-Altman plot showed the mean bias ± SD between total CT-PCI and surgical PCI as 0.431 ± 3.005, with the LoAs from - 5.459 to 6.321. There were nine points of difference in total PCI exceeded the 95% LoAs, with the rate of 5.39% (9/167). As for selected CT-PCI, Bland-Altman plot showed the mean bias ± SD between selected CT-PCI and surgical PCI as - 0.287 ± 1.955, with the LoAs from - 4.118 to 3.544. There were ten points of difference in selected PCI exceeded the 95% LoAs, with the rate of 5.99% (10/167). The Spearman's rank correlation coefficient between total CT-PCI and surgical PCI was 0.911, P < 0.001, as for selected CT-PCI and surgical PCI, the coefficient was 0.909, P < 0.001. Although there was a strong correlation for both total and selected CT-PCI with surgical PCI, however, the agreement is still not good in Bland-Altman analysis, which suggested that CT-PCI cannot predict surgical PCI accurately even in professional PMP treatment centers. In brief explanation, CT makes it difficult to distinguish the borderline between tumor tissue and mucus and to detect tumor lesions in the small intestine regions, which caused overestimation or underestimation by CT-PCI. In the future, a multiple linear regression model based on CT-PCI might accurately predict surgical PCI preoperatively.
BibTeX
@inproceedings{bai2023blandaltman,
  author = {Mingjian Bai and Jingliang Chen and Yueming Xu and Jing Feng and Ruiqing Ma and Hongmin Jia and Hongbin Xu and Guowei Liang and Hongjiang Wei},
  title = {{Bland–Altman agreement analysis between CT predicted and surgical peritoneal cancer index in pseudomyxoma peritonei of appendiceal origin}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-48975-9},
}
Figure