TL;DR
TeachMe is a web-based platform that streamlines the annotation of abdominal lymph nodes while simultaneously training junior doctors. By combining a three-level expert review workflow with an AI-assisted feedback loop, the system creates high-quality medical datasets and measurably improves clinical interpretation skills.
What Problem Does This Paper Address?
Detecting lymph nodes (LNs) in abdominal CT scans is critical for diagnosing and treating colorectal cancer (CRC) recurrence. However, manual interpretation is a "needle in a haystack" problem—it is labor-intensive, time-consuming, and prone to human error.
While AI models can help, they require massive, precisely labeled datasets that are difficult to produce. Furthermore, there is a significant gap in radiology education: junior doctors need extensive practice to reach expert-level proficiency, but senior doctors are often too "time-poor" to provide consistent, detailed feedback. Existing e-learning tools are often passive (theory-based) rather than active (practice-based).
Methods
The authors developed TeachMe, a Browser/Server architecture system (Node.js, MongoDB, Python) designed to create a virtuous cycle between data collection and education. The core of the system is its three-level annotation-review workflow:
- Primary Annotation: Two junior doctors independently annotate a CT study.
- AI Supplement: A pre-trained automated detection model analyzes the scan to identify potential lymph nodes that the human annotators might have missed.
- Superior Review: A senior expert reviews the merged human and AI results to confirm the "gold standard" expert knowledge.
For the educational component, the system features a feedback mechanism. When a junior doctor completes a practice case, TeachMe compares their work against the expert database and generates a report highlighting omitted nodes, incorrect locations, or misclassifications.
Datasets
The study introduced the PLNDataset (Pelvic Lymph Node Dataset):
- Scale: 236 CRC patients' contrast-enhanced CT (CE-CT) scans.
- Content: 6,880 individual abdominal lymph nodes.
- Categorization: Nodes are divided into seven anatomical regions (e.g., Colorectal mesenteries, Aortic bifurcation, Obturator cranial LNs).
- Uniqueness: Unlike standard datasets, this was "calibrated" through the three-level review process, ensuring a higher degree of ground-truth accuracy.
Key Insights and Ideas
- AI as a "Safety Net": Instead of replacing the human, the AI acts as a second set of eyes during the data collection phase, specifically looking for "candidates omitted" by the primary annotators.
- Closing the Loop: The system treats the expert database not just as a training set for machines, but as an automated "tutor" for humans. This allows junior doctors to practice repeatedly without requiring a senior doctor to be present for every session.
- Standardized Sorting: The authors implemented a specific sorting rule for annotations (ascending left-to-right and descending top-to-bottom). This seemingly small UI choice allows doctors to track specific nodes across different studies and consultations more reliably.
Key Results
- Improved AI Performance: Training a detection model on the "calibrated" expert data (reviewed by superior annotators) significantly outperformed training on raw primary annotations. Sensitivity increased from 79% to 83%, and the mean Free-response Receiver Operating Characteristic (mFROC) improved from 0.351 to 0.391.
- Human Skill Acquisition: Junior doctors showed marked improvement over three rounds of practice. For example, "User A" improved their detection accuracy from approximately 0.84 to 0.92.
- Efficiency: The web-based nature of the tool removed the need for doctors to be physically present at a hospital workstation (PACS), allowing for more flexible data collection.
Limitations and Future Work
- Image Format: Currently, TeachMe does not support raw DICOM files directly. Images are displayed with fixed window widths and levels, meaning users cannot adjust brightness or contrast to better see "tricky" regions.
- Annotation Tools: The system is currently limited to rectangular bounding boxes. The authors acknowledge that for tasks like segmentation, they need to implement free-hand or bidirectional cross-shaped tools.
- Label Flexibility: The current label set is hard-coded for gastrointestinal surgery. Future versions aim to allow experts to formulate customized labels for other medical specialties.
Summary updated