Abstract
Temporal medical image analysis is essential for clinical decision-making, yet existing methods either align images and text at a coarse level - causing potential semantic mismatches - or depend solely on visual information, lacking medical semantic integration. We present CheXLearner, the first end-to-end framework that unifies anatomical region detection, Riemannian manifold-based structure alignment, and fine-grained regional semantic guidance. Our proposed Med-Manifold Alignment Module (Med-MAM) leverages hyperbolic geometry to robustly align anatomical structures and capture pathologically meaningful discrepancies across temporal chest X-rays. By introducing regional progression descriptions as supervision, CheXLearner achieves enhanced cross-modal representation learning and supports dynamic low-level feature optimization. Experiments show that CheXLearner achieves 81.12% (+17.2%) average accuracy and 80.32% (+11.05%) F1-score on anatomical region progression detection - substantially outperforming state-of-the-art baselines, especially in structurally complex regions. Additionally, our model attains a 91.52% average AUC score in downstream disease classification, validating its superior feature representation.
Introduction: The Challenges of Temporal Radiology
Tracking how a disease evolves over time is a cornerstone of radiology. However, building AI that can effectively compare a "current" X-ray to a "prior" one is notoriously difficult due to three primary hurdles:
- Semantic Mismatch: Conventional models often align a global medical report with a global image. This coarse approach fails to capture localized changes, such as a new nodule in a specific lung lobe.
- The Registration Nightmare: Variations in patient positioning, breathing, and motion between scans introduce "noise." Simple feature subtraction often highlights these physiological differences rather than actual disease progression.
- The Two-Stage Bottleneck: Many existing tools rely on a pre-trained, frozen region detector followed by a separate classifier. This prevents the detector from learning task-specific visual cues essential for spotting subtle progression.
Methodology: The CheXLearner Framework
CheXLearner addresses these issues through an end-to-end architecture that unifies detection and alignment with semantic guidance.
End-to-End DETR Detection
Unlike previous models that use frozen detectors, CheXLearner utilizes an unfrozen DEtection TRansformer (DETR). This allows the region detection module to dynamically adjust its feature representations based on the specific requirements of progression detection, ensuring the model focuses on the most relevant visual cues.
Med-Manifold Alignment Module (Med-MAM)
To overcome registration challenges, CheXLearner moves beyond flat Euclidean space. The Med-MAM maps visual features onto a Riemannian hyperbolic manifold (specifically the Poincaré disk). Hyperbolic geometry is inherently better at representing hierarchical structures, such as the branching vessels and airways of the lungs. By using "parallel transport" to align vectors from different time points, the module effectively neutralizes distortions caused by varying X-ray angles or patient breathing.
Fine-Grained Text Guidance
The model integrates a pre-trained MedCLIP encoder to process regional text descriptions (e.g., "At right hilar structures, worsened"). Through a contrastive loss (InfoNCE), the visual encoder is trained to align its features with these specific medical semantics, ensuring that the model learns to ignore irrelevant visual changes and focus on pathologically meaningful ones.
Key Insights and Results
The study primarily utilizes the Chest-ImaGenome dataset, leveraging over 427,000 anatomical region pairs for training and a gold-standard subset of 500 cases manually annotated by radiologists for benchmarking across 12 anatomical regions and 9 disease categories.
Performance Breakthroughs
CheXLearner sets a new state-of-the-art for temporal medical image analysis:
- Progression Detection: The model achieved 81.12% average accuracy and an 80.32% F1-score, representing a massive 17.2% and 11.05% improvement over the previous best, CheXDetector.
- Robustness in Complex Regions: In anatomically "messy" areas like the hilar structures, where vessels and airways overlap, the model saw F1-score improvements exceeding 32%.
- Downstream Utility: In general disease classification, the model reached a 91.52% average AUC, confirming that its learned features are highly representative of actual clinical conditions.
Conclusion and Future Directions
CheXLearner demonstrates that non-Euclidean geometry and fine-grained text guidance are powerful tools for medical imaging. By treating lung structures as hierarchies in curved space, the model becomes significantly more robust to patient positioning noise.
Future work will aim to address current limitations, such as the resolution bottleneck in detecting the earliest signs of disease. Additionally, integrating Medical Knowledge Graphs could provide deeper prior knowledge to supplement clinical reports, which are often conclusion-oriented rather than descriptive. Expanding the model's scope from 12 to the full 29 anatomical regions available in the dataset remains a key objective for scaling this technology.
Summary updated