Introduction
Content-Based Image Retrieval (CBIR) is a cornerstone of modern digital pathology, enabling clinicians to query vast archives for similar historical cases to assist in diagnosis. However, the sheer scale of Whole Slide Images (WSIs)—which often reach gigapixel dimensions—presents a significant "needle in a haystack" problem. Traditional search engines typically partition these slides into small patches for indexing, but without intelligent selection, critical malignant regions are often overlooked in favor of benign tissue. This paper proposes a novel framework that leverages Immunohistochemistry (IHC) biomarkers to guide the search process, effectively creating an automated "attention map" for diagnostic precision.
The Targeted Search Framework
The authors introduce a multi-stage "Targeted Search" pipeline designed to mimic the expert workflow of a pathologist. The process begins with Image Registration, where H&E-stained slides are aligned with IHC-stained cuts at the pixel level using non-rigid registration. This ensures that the diagnostic information from the IHC stain can be accurately mapped back to the standard H&E morphology.
To handle the inherent variability of biological staining, the system employs an Adaptive Neuro-Fuzzy Inference System (ANFIS). Unlike simple binary thresholding, ANFIS processes RGB values to distinguish between varying intensities of staining (e.g., light brown vs. dark brown), accommodating the "fuzzy" nature of protein expression in tissue.
Composite Biomarker Image (CBI)
Information from multiple IHC markers—specifically CD30 and PAX5 in this study—is synthesized into a single Composite Biomarker Image (CBI). This mask acts as a spatial guide, highlighting the most diagnostically relevant cellular activity. By performing Selective Patching based on this mask, the search engine focuses exclusively on regions of interest rather than indexing the entire slide. These targeted patches are then processed through KimiaNet, a specialized DenseNet-121 architecture, to generate 1,024-dimensional feature vectors. Search is then performed using a $k$-nearest neighbor ($k$-NN) approach based on Euclidean distance.
Experimental Setup and Results
The framework was validated using a dataset of 23 Hodgkin Lymphoma cases (15 Classical Hodgkin Lymphoma [CHL] and 8 Nodular Lymphocyte Predominant Hodgkin Lymphoma [NLPHL]) from Grand River Hospital. The study utilized 69 WSIs and 9,296 patches extracted at 20× magnification.
The "Targeted Search" consistently outperformed standard H&E-only search methods across all metrics:
- Precision: For CHL cases, the "Majority-7" precision—representing the accuracy of the majority vote among the top seven results—reached 100% with IHC guidance, compared to 91.67% for standard search.
- Subtype Accuracy: In the more challenging NLPHL subtype, IHC guidance improved the Majority-7 precision from 63.64% to 72.73%.
- F1-Score: The overall F1-score for CHL retrieval saw a significant boost from 81.48 to 88.89.
Discussion and Limitations
By treating IHC biomarkers as a high-fidelity attention mechanism, this framework provides a path toward more reliable and clinically relevant image retrieval. It essentially provides "ground truth" for where a machine learning network should focus without requiring manual, time-consuming annotations from pathologists.
However, several limitations remain. IHC stains are more expensive and time-consuming to produce than standard H&E slides, which may lead to diagnostic delays or limited availability in some clinical settings. Furthermore, while the results are promising, the study was conducted on a relatively small cohort of 23 cases. Future work will need to address how to handle cases where biomarker data is sparse and validate the framework on larger, more diverse datasets with a wider variety of cancer subtypes.
Summary updated