Regressive Vision Transformer for Dog Cardiomegaly Assessment

Jialu Li, Youshan ZhangOpenReview 2024 · OpenReview

Abstract

Cardiac disease is one of the leading causes of death in dogs. Automatic cardiomegaly detection has great significance in helping clinicians improve the accuracy of the diagnosis process. Deep learning methods show promising results in improving cardiomegaly classification accuracy, while they are still not widely applied in clinical trials due to the difficulty in mapping predicted results with input radiographs. To overcome these challenges, we first collect large-scale dog heart X-ray images. We then develop a dog heart labeling tool and apply a few-shot generalization strategy to accelerate the label speed. We also develop a regressive vision transformer model with an orthogonal layer to bridge traditional clinically used VHS metric with deep learning models. Extensive experimental results demonstrate that the proposed model achieves state-of-the-art performance.

Introduction

In veterinary medicine, diagnosing heart enlargement (cardiomegaly) relies heavily on the Vertebral Heart Scale (VHS). This metric requires clinicians to manually measure the long and short axes of the heart on a lateral radiograph and compare these lengths to the dog's thoracic vertebrae.

Despite the success of deep learning in image classification, clinical adoption remains low due to two primary factors: the "black-box" nature of standard models which lack explainability, and the manual effort required to ensure measurements are anatomically correct (e.g., ensuring axes are perfectly perpendicular). This research introduces a Regressive Vision Transformer (RVT) designed to automate this process while maintaining clinical rigor and transparency.

The Vertebral Heart Scale (VHS) measurement process on a canine lateral radiograph, showing the long axis (AB), short axis (CD), and vertebral reference (EF).

Proposed Method: Regressive Vision Transformer

The RVT architecture is designed to mimic the geometric steps used by veterinarians. It consists of three primary components that transform raw pixels into clinical measurements.

Architecture Overview

  1. Pyramid Vision Transformer (PVT) Encoder: The model uses a shrinking pyramid structure to capture multi-scale features. This is essential for identifying both large-scale heart silhouettes and fine-grained anatomical landmarks like individual vertebrae.
  2. Feature Fusion Module (FFM): This module integrates low-level textures and edges with high-level global shapes, ensuring the model accurately locates the precise endpoints of the heart axes.
  3. The Orthogonal Layer: A key innovation of this work is the integration of a mathematical constraint directly into the network. In clinical practice, the short axis of the heart must be perpendicular to the long axis. The orthogonal layer enforces this 90-degree rule during the regression of key points, preventing anatomically impossible predictions.
The Regressive Vision Transformer (RVT) pipeline, featuring the PVT Encoder, Feature Fusion Module, and the final regression of clinical key points.

The DogHeart Dataset

To train and validate the model, the researchers curated the DogHeart dataset, comprising 2,000 high-quality X-ray images from the Shanghai Aichong Pet Hospital. Each image was meticulously labeled by two experts with six key points defining the heart axes and vertebral reference. To overcome the bottleneck of manual labeling, a few-shot generalization strategy was employed, reducing the total labeling time from months to just two weeks.

Key Results and Insights

The RVT model was benchmarked against 12 state-of-the-art architectures, including ResNet and EfficientNet.

  • Superior Performance: The RVT achieved a regression-based diagnostic accuracy of 87.3%, significantly outperforming traditional classification-only models.
  • Geometry as a Regularizer: Ablation studies confirmed that the "Orthogonal Layer" improved accuracy by 0.5% to 2.2% across different backbones by guiding the AI with anatomical logic.
  • Explainability: Unlike standard classifiers, the RVT outputs the specific coordinates used for the VHS calculation. This allows veterinarians to visually verify the AI's logic, transforming the model from a "black box" into a reliable digital assistant.
  • Efficiency: The model maintains an inference time of approximately 0.92 seconds per image on an RTX A6000, making it suitable for real-time clinical integration.

Conclusion

By bridging the gap between deep learning and established clinical metrics, the Regressive Vision Transformer provides a transparent and accurate tool for veterinary diagnostics. Future work will focus on expanding the dataset to include a wider variety of dog breeds and exploring the application of this regressive approach to other medical imaging tasks, such as human cardiothoracic ratio (CTR) assessment.

Summary updated

Abstract

Cardiac disease is one of the leading causes of death in dogs. Automatic cardiomegaly detection has great significance in helping clinicians improve the accuracy of the diagnosis process. Deep learning methods show promising results in improving cardiomegaly classification accuracy, while they are still not widely applied in clinical trials due to the difficulty in mapping predicted results with input radiographs. To overcome these challenges, we first collect large-scale dog heart X-ray images. We then develop a dog heart labeling tool and apply a few-shot generalization strategy to accelerate the label speed. We also develop a regressive vision transformer model with an orthogonal layer to bridge traditional clinically used VHS metric with deep learning models. Extensive experimental results demonstrate that the proposed model achieves state-of-the-art performance.
BibTeX
@misc{li2024regressive,
  author = {Jialu Li and Youshan Zhang},
  title = {{Regressive Vision Transformer for Dog Cardiomegaly Assessment}},
  year = {2024},
  url = {https://openreview.net/forum?id=2eE9EBMdId},
  howpublished = {OpenReview},
}
Figure