Introduction
In clinical practice, "closing the loop" between diagnostic imaging and pathology is essential for quality assurance. Radiologists and pathologists often evaluate the same patient, yet their findings are frequently siloed within Electronic Medical Records (EMRs). To ensure diagnostic accuracy, hospital staff must manually compare imaging reports (e.g., an ultrasound of a neck mass) with subsequent pathology reports (e.g., a biopsy of the parotid gland).
This manual process is hindered by linguistic variability; one report might refer to the "maxillofacial region" while another specifies the "parotid gland." Traditional keyword matching fails to recognize that the parotid gland is a subset of the maxillofacial region. This study addresses the challenge of semantic similarity—identifying when disparate text descriptions refer to the same anatomical site.
Methods
The researchers developed an end-to-end Convolutional Neural Network (CNN) architecture designed to capture local, hierarchical patterns in medical text. Unlike generic NLP models, this approach integrates domain-specific knowledge through Ontology-Based Graph Embedding. By utilizing CMeSH (Chinese Medical Subject Headings) and graph embedding techniques, the model captures anatomical hierarchies before training begins.
The model employs a dual-input CNN structure that processes imaging and pathology reports through parallel convolutional layers. These layers extract feature vectors which are then concatenated to predict the probability of a match. To address the "black box" nature of neural networks, the study utilized the LIME (Local Interpretable Model-agnostic Explanations) algorithm to identify specific keywords, such as "thyroid" or "fetus," that most influenced the model's decisions.
Dataset and Evaluation
The study utilized a real-world dataset from Shanghai Tongren Hospital, consisting of 16,354 imaging and pathology report pairs from 1,926 patients collected over a three-month period in 2017. Ground truth was established by three physicians who manually annotated the pairs for anatomical overlap, achieving high inter-rater reliability (Kappa 0.95–0.97). The data was split into 80% for training and 20% for testing.
Key Results
The CNN model initialized with medical concept vectors significantly outperformed traditional NLP methods and Siamese LSTMs. The hierarchical nature of CNN kernels proved particularly effective at mapping specific tissues to broader anatomical regions.
As shown in the ROC analysis, the CNN model with concept vectors achieved an AUROC of 0.951, representing a 3–7% improvement over Latent Dirichlet Allocation (LDA) and Latent Semantic Analysis (LSA). The model also achieved a Macro F1-score of 0.935, a substantial increase over the keyword mapping baseline of 0.833. Even the simplest CNN version with random initialization outperformed complex traditional models, validating the architecture's suitability for medical text matching.
Discussion and Limitations
While the model demonstrates high accuracy, several challenges remain. The reliance on the Jieba segmentation tool for Chinese text can occasionally lead to errors in specialized medical terminology. Furthermore, the current model acts as a binary classifier; future iterations could implement a ranking system to help clinicians prioritize the most relevant reports. Finally, the model occasionally struggles with complex 3D spatial relationships (e.g., "ventral side of the inferior pole"), suggesting that incorporating more sophisticated spatial reasoning remains a key area for future research.
Summary updated