Abstract
More than 144,000 Australians were diagnosed with cancer in 2019. The majority will first present to their GP symptomatically, even for cancer for which screening programs exist. Diagnosing cancer in primary care is challenging due to the non-specific nature of cancer symptoms and its low prevalence. Understanding the epidemiology of cancer symptoms and patterns of presentation in patient's medical history from primary care data could be important to improve earlier detection and cancer outcomes. As past medical data about a patient can be incomplete, irregular or missing, this creates additional challenges when attempting to use the patient's history for any new diagnosis. Our research aims to investigate the opportunities in a patient's pathology history available to a GP, initially focused on the results within the frequently ordered full blood count to determine relevance to a future high-risk cancer prognosis, and treatment outcome. We investigated how past pathology test results can lead to deriving features that can be used to predict cancer outcomes, with emphasis on patients at risk of not surviving the cancer within 2-year period. This initial work focuses on patients with lung cancer, although the methodology can be applied to other types of cancer and other data within the medical record. Our findings indicate that even in cases of incomplete or obscure patient history, hematological measures can be useful in generating features relevant for predicting cancer risk and survival. The results strongly indicate to add the use of pathology test data for potential high-risk cancer diagnosis, and the utilize additional pathology metrics or other primary care datasets even more for similar purposes.
Introduction
General Practitioners (GPs) represent the front line of cancer detection, yet they face a significant "signal-to-noise" challenge. Cancer symptoms in primary care are often non-specific, and the disease itself has a low prevalence relative to other conditions. Furthermore, primary care data is inherently "messy." Unlike controlled clinical environments, real-world medical records are characterized by uncertainty: irregular testing intervals, missed appointments, and incomplete data.
This study addresses the challenge of utilizing this incomplete pathology history to identify high-risk lung cancer patients—specifically those unlikely to survive two years post-diagnosis. By extracting prognostic signals from routine blood tests, we aim to provide better decision support for immediate clinical action in primary care settings.
Methodology
The research focused on the Full Blood Count (FBC), specifically analyzing five metrics: Platelet count, MCV, MCH, MCHC, and RDW. To manage data uncertainty and maximize predictive value, several novel feature engineering techniques were employed:
- Soft Range Definition: To capture borderline cases, a "soft range" (a 2.5% buffer inside standard normal limits) was defined, recognizing that traditional binary "normal/abnormal" flags may overlook early pathological signals.
- Temporal Labeling: Results were categorized based on their proximity to the diagnosis date (e.g., within a 3-month window vs. earlier) to identify temporal patterns predictive of survival.
- Combinatory Features: Synthetic features were created by combining blood metrics and incorporating biological sex to explore interactions between different biomarkers.
- Machine Learning Models: Decision Trees, AdaBoost, and XGBoost were trained and validated using 10-fold cross-validation to predict 2-year survival.
Data and Feature Engineering
The study utilized a Victorian subset of the MedicineInsight dataset, a nationally representative primary care database in Australia. The researchers identified a specific cohort of 472 lung cancer patients who had a recorded history for all five targeted hematological measures, ensuring a robust training set without the need for missing data assumptions.
As shown in the distribution above, the presence and timing of out-of-range (OOR) measures correlate with different survival outcomes across age groups. These temporal labels proved critical in distinguishing high-risk profiles from standard presentations.
Results and Discussion
The models achieved a classification accuracy of approximately 65%. While seemingly modest, this represents a strong indicator of risk given the high uncertainty of primary care data and the absence of information regarding cancer stage or treatment type.
A key finding was the predictive power of "Out-of-Range" (OOR) results. As detailed in the table above, patients with OOR results within three months of diagnosis exhibited mortality rates significantly higher than those with normal results. For instance, abnormal platelet counts were associated with mortality rates exceeding 52%. Furthermore, the impact of abnormal results was gender-sensitive; an out-of-range platelet count in women was associated with a mortality increase from 71.25% to 89.29%.
The study also confirmed that features derived from the timing of abnormal tests were more relevant than the raw values themselves. The "soft range" approach further validated that values just inside the normal limit carry predictive weight, suggesting that traditional lab reporting may overlook early warning signs.
Conclusion and Future Work
This research demonstrates that routine pathology history contains significant prognostic signals that can help GPs prioritize high-risk patients, even when data is irregular or incomplete. The methodology effectively handles the inherent uncertainty of primary care records by focusing on temporal patterns and borderline values.
Future research will focus on:
- Expanding the methodology to other cancer types, such as colorectal cancer.
- Incorporating complex time-series analysis to track the trajectory of blood results over several years.
- Integrating additional primary care data points, such as prescriptions and symptomatic history, to further enhance predictive power.
Summary updated