Computer-aided prognosis of tuberculous meningitis combining imaging and non-imaging data

Liane S. Canas, Dong Huu Khanh Trinh, Daniel Beasley, Joseph Donovan, Jon O. Cleary, Richard Brown, Nguyễn Thụy Thương Thương, Phu Hoan Nguyen, Nguyen Thi Hoang Ha, Reza Razavi, Sébastien Ourselin, Guy Thwaites, Marc Modat, Dang Phuong Thao, Trung Kien Dang, Doan Bui Xuan Thy, Du Hong Duc, Ronald B. Geskus, Ho Bich Hai, Ho Quang Chanh, Ho Van Hien, Huynh Trung Trieu, Evelyne Kestelyn, Lam Minh Yen, Le Dinh Van Khoa, Le Thanh Phuong, Le Thuy Thuy Khanh, Luu Hoai Bao Tran, An Phuoc Luu, Angela McBride, Nguyen Lam Vuong, Nguyễn Quang Huy, Nguyen Than Ha Quyen, Nguyễn Thanh Ngọc, Nguyen Thi Giang, Nguyen Thi Diem Trinh, Nguyen Thi Le Thanh, Nguyễn Thị Phương Dung, Nguyễn Thị Phương Thảo, Ninh Thi Thanh Van, Pham Tieu Kieu, Phan Nguyen Quoc Khanh, Phung Khanh Lam, Phung Tran Huy Nhat, Louise Thwaites, Duc Minh Tran, Trinh Manh Hung, Hugo C. Turner, Jennifer Ilo Van Nuil, Vo Tan Hoang, Vu Ngo Thanh Huyen, Sophie Yacoub, Cao Thi Tam, Duong Bich Thuy, Ha Thi Hai Duong, Ho Dang Trung Nghia, Le Buu Chau, Le Mau Toan, Le Ngoc Minh Thu, Le Thi Mai Thao, Luong Thi Hue Tai, Nguyen Hoan Phu, Nguyễn Quốc Việt, Nguyen Thanh Nguyen, Nguyễn Thanh Phong, Nguyễn Thị Kim Anh, Nguyễn Văn Hảo, Nguyen Van Thanh Duoc, Pham Kieu Nguyet Oanh, Phan Thi Van, Phan Tu Qui, Phan Vinh Tho, Truong Thi Phuong Thao, Natasha Ali, David A. Clifton, Mike English, Jannis Hagenah, Ping Lü, Jacob McKnight, Chris Paton, Tingting Zhu, Pantelis Georgiou, Bernard Hernandez Perez, Kerri Hill-Cawthorne, Alison Holmes, Stefan Karolcik, Damien Ming, Nicolas Moser, Jesús Rodríguez-Manzano, Alberto Gómez, Hamideh Kerdegari, Andrew P. King, Miguel Xochicale, Walter KarlenScientific 2024 · DOI

Abstract

Tuberculous meningitis (TBM) is the most lethal form of tuberculosis, yet accurate prognosis remains challenging due to the qualitative nature of current clinical scales and the complexity of co-morbidities like HIV. This paper presents an end-to-end machine learning framework that integrates 3D brain MRI scans with longitudinal clinical data to predict disease progression. By combining a DenseNet architecture for automated imaging feature encoding with a bilateral LSTM for time-series forecasting, the model achieves a 60% balanced accuracy across six outcome classes. Notably, the model identifies brain lesions with 96% accuracy and demonstrates that while clinical data predicts mild outcomes, imaging is critical for identifying severe neurological risks. This approach offers a robust, quantitative tool for timely clinical intervention in TBM.

Introduction

Tuberculous meningitis (TBM) represents the most severe manifestation of tuberculosis, often resulting in permanent neurological damage or death. Despite its high mortality rate, particularly in regions with high HIV prevalence, prognostic tools have remained largely qualitative. Clinicians typically rely on the Medical Research Council (MRC) grade or the Modified Rankin Scale (mRS) to assess severity. However, these scales are subjective and often fail to capture the full extent of brain pathology visible on Magnetic Resonance Imaging (MRI).

The integration of MRI data into predictive models has been hindered by the "messiness" of clinical datasets, characterized by irregular follow-up intervals and missing data. This study addresses these challenges by developing a deep learning framework capable of fusing multi-modal, longitudinal data to provide an automated, quantitative prognosis for TBM patients.

An End-to-End Deep Learning Framework

The researchers implemented a two-stage architecture designed to handle both static imaging features and dynamic clinical variables.

The proposed end-to-end framework for TBM prognosis prediction, integrating 3D MRI feature extraction (A) with a bilateral LSTM for time-series clinical data (B).

1. Feature Extractor (DenseNet-169)

A 3D convolutional neural network (CNN) based on the DenseNet-169 architecture serves as the backbone for processing T1-weighted MRI scans. The network was pre-trained to predict the TBM MRC grade, allowing it to learn a 128-dimensional vector representation of relevant morphological brain features, such as tuberculomas and vasculitis.

2. Prognosis Predictor (Bilateral LSTM)

The encoded imaging features are combined with 15 longitudinal clinical variables—including the Glasgow Coma Scale, age, BMI, and cerebrospinal fluid markers. This fused data is fed into a bilateral Long Short-Term Memory (LSTM) network. Unlike standard LSTMs, the bilateral approach learns disease progression patterns by looking both forward and backward in time, enhancing its ability to predict future mRS scores.

To address the issue of missing data and irregular sampling, the team introduced a tailored optimization framework. A Gaussian kernel weights the relevance of MRI scans based on their temporal proximity to clinical assessments, allowing the model to generate predictions for any time point in the patient's journey.

Data and Methodology

The study utilized data from 216 adults with TBM participating in prospective longitudinal trials in Vietnam. The cohort included 73 HIV-positive patients, a group historically associated with higher mortality. Data points included 3D T1-weighted MRI scans at enrollment, two months, and twelve months, alongside frequent clinical assessments spanning over a year.

Key Insights and Results

The model's performance yielded several critical insights into the nature of TBM and the value of multi-modal data:

  • The Necessity of Imaging for Severe Outcomes: A key finding was that while clinical data alone can predict mild to moderate outcomes, imaging data is essential for detecting severe outcomes (mRS ≥ 4). Models lacking MRI features significantly underestimated the risk of severe disability, highlighting the importance of brain morphology in long-term prognosis.
  • Biological Consistency Across HIV Status: Despite the higher mortality risk in HIV-positive patients, the model's accuracy remained consistent across both groups. Occlusion sensitivity maps revealed that the AI identified similar pathological features—such as lesions and inflammation—in both populations, suggesting that TBM pathology follows a consistent biological path regardless of HIV-induced immunosuppression.
  • Temporal Decay of Predictive Power: The model achieved 80% accuracy in identifying status changes (improving, stable, or worsening) when using scans taken within 10 days of assessment. However, this accuracy declined as the time since the last scan increased, quantifying the need for contemporary imaging in clinical decision-making.

Overall, the integrated model achieved a 60% balanced accuracy across six mRS classes, significantly outperforming traditional multivariate baseline models. Furthermore, the feature extractor associated brain lesions with disease staging with an accuracy of 96%.

Discussion and Future Directions

This framework represents the first successful attempt to predict TBM prognosis by combining imaging and clinical data through deep learning. Its ability to handle real-world data irregularities makes it a promising candidate for clinical decision support.

However, limitations remain. The reliance on imputation techniques for missing labels may introduce bias, and the relatively small HIV-positive cohort suggests a need for larger, more diverse datasets. Future work should also incorporate treatment variables, such as dexamethasone administration, which were blinded in this study. Expanding the model's validation to cohorts outside of Vietnam will be essential to ensure global applicability across different demographic and environmental contexts.

Summary updated

Abstract

Tuberculous meningitis (TBM) is the most lethal form of tuberculosis. Clinical features, such as coma, can predict death, but they are insufficient for the accurate prognosis of other outcomes, especially when impacted by co-morbidities such as HIV infection. Brain magnetic resonance imaging (MRI) characterises the extent and severity of disease and may enable more accurate prediction of complications and poor outcomes. We analysed clinical and brain MRI data from a prospective longitudinal study of 216 adults with TBM; 73 (34%) were HIV-positive, a factor highly correlated with mortality. We implemented an end-to-end framework to model clinical and imaging features to predict disease progression. Our model used state-of-the-art machine learning models for automatic imaging feature encoding, and time-series models for forecasting, to predict TBM progression. The proposed approach is designed to be robust to missing data via a novel tailored model optimisation framework. Our model achieved a 60% balanced accuracy in predicting the prognosis of TBM patients over the six different classes. HIV status did not alter the performance of the models. Furthermore, our approach identified brain morphological lesions caused by TBM in both HIV and non-HIV-infected, associating lesions to the disease staging with an overall accuracy of 96%. These results suggest that the lesions caused by TBM are analogous in both populations, regardless of the severity of the disease. Lastly, our models correctly identified changes in disease symptomatology and severity in 80% of the cases. Our approach is the first attempt at predicting the prognosis of TBM by combining imaging and clinical data, via a machine learning model. The approach has the potential to accurately predict disease progression and enable timely clinical intervention.
BibTeX
@inproceedings{canas2024computer,
  author = {Liane S. Canas and Dong Huu Khanh Trinh and Daniel Beasley and Joseph Donovan and Jon O. Cleary and Richard Brown and Nguyễn Thụy Thương Thương and Phu Hoan Nguyen and Nguyen Thi Hoang Ha and Reza Razavi and Sébastien Ourselin and Guy Thwaites and Marc Modat and Dang Phuong Thao and Trung Kien Dang and Doan Bui Xuan Thy and Dong Huu Khanh Trinh and Du Hong Duc and Ronald B. Geskus and Ho Bich Hai and Ho Quang Chanh and Ho Van Hien and Huynh Trung Trieu and Evelyne Kestelyn and Lam Minh Yen and Le Dinh Van Khoa and Le Thanh Phuong and Le Thuy Thuy Khanh and Luu Hoai Bao Tran and An Phuoc Luu and Angela McBride and Nguyen Lam Vuong and Nguyễn Quang Huy and Nguyen Than Ha Quyen and Nguyễn Thanh Ngọc and Nguyen Thi Giang and Nguyen Thi Diem Trinh and Nguyen Thi Le Thanh and Nguyễn Thị Phương Dung and Nguyễn Thị Phương Thảo and Ninh Thi Thanh Van and Pham Tieu Kieu and Phan Nguyen Quoc Khanh and Phung Khanh Lam and Phung Tran Huy Nhat and Louise Thwaites and Louise Thwaites and Duc Minh Tran and Trinh Manh Hung and Hugo C. Turner and Jennifer Ilo Van Nuil and Vo Tan Hoang and Vu Ngo Thanh Huyen and Sophie Yacoub and Cao Thi Tam and Duong Bich Thuy and Ha Thi Hai Duong and Ho Dang Trung Nghia and Le Buu Chau and Le Mau Toan and Le Ngoc Minh Thu and Le Thi Mai Thao and Luong Thi Hue Tai and Nguyen Hoan Phu and Nguyễn Quốc Việt and Nguyen Thanh Dung and Nguyen Thanh Nguyen and Nguyễn Thanh Phong and Nguyễn Thị Kim Anh and Nguyễn Văn Hảo and Nguyen Van Thanh Duoc and Pham Kieu Nguyet Oanh and Phan Thi Van and Phan Tu Qui and Phan Vinh Tho and Truong Thi Phuong Thao and Natasha Ali and David A. Clifton and Mike English and Jannis Hagenah and Ping Lü and Jacob McKnight and Chris Paton and Tingting Zhu and Pantelis Georgiou and Bernard Hernandez Perez and Kerri Hill-Cawthorne and Alison Holmes and Stefan Karolcik and Damien Ming and Nicolas Moser and Jesús Rodríguez-Manzano and Liane S. Canas and Alberto Gómez and Hamideh Kerdegari and Andrew P. King and Marc Modat and Reza Razavi and Miguel Xochicale and Walter Karlen},
  title = {{Computer-aided prognosis of tuberculous meningitis combining imaging and non-imaging data}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-68308-8},
}
Figure