Introduction
Automated chest radiography diagnosis has advanced significantly, primarily within English-speaking contexts. However, applying these technologies to the Japanese language introduces unique linguistic challenges. Unlike English, Japanese lacks explicit word boundaries (spaces), making standard tokenization prone to errors, particularly with specialized medical terminology. Furthermore, Japanese frequently employs multiple orthographic variants for the same concept (e.g., mixing Kanji and Katakana). Traditional word-level models treat these variants as distinct entities, which dilutes training data and reduces model robustness.
To address these issues, this research proposes an end-to-end deep learning model that generates Japanese medical findings directly from chest X-rays using a character-level approach. This method bypasses the need for complex segmentation and handles orthographic variations more effectively.
Methodology
The proposed system utilizes an Encoder-Decoder architecture integrated with a "soft-attention" mechanism.
Encoder and Decoder Design
The encoder uses a ResNet-151 CNN to extract visual features from the input radiographs. To ensure the encoder recognizes domain-specific medical features, it was pre-trained on a classification task using over 13,000 chest images before being integrated into the generation pipeline.
The decoder is a Long Short-Term Memory (LSTM) network. Unlike conventional models that predict the next word, this decoder predicts the next character. This character-level generation allows the model to construct technical terms and handle spelling variations without a predefined vocabulary dictionary.
Attention Mechanism
The attention layer enables the decoder to dynamically focus on specific regions of the X-ray image while generating each character. This visual grounding provides interpretability, allowing radiologists to see which parts of the image influenced the generated text.
Dataset and Experimental Setup
The study utilized the JSRT (Japanese Society of Radiological Technology) Dataset, comprising 18,004 pairs of frontal chest radiographs and Japanese findings. Two subsets were created based on frequency thresholds (findings appearing at least 5 times vs. 30 times). To mitigate the class imbalance where approximately 75% of images are "normal," the researchers employed oversampling and undersampling techniques.
Results and Discussion
The model's performance was evaluated using BLEU scores and qualitative visual inspection of attention maps.
Quantitative Performance
The model achieved a high BLEU-4 score of 0.8214 on the original distribution, largely due to its high accuracy in identifying "normal" cases. However, when tested exclusively on abnormal images, the BLEU-4 score dropped to approximately 0.1713. This discrepancy highlights the complexity of describing diverse pathologies compared to the relative consistency of normal reports.
Qualitative Analysis and Visual Grounding
Qualitative results demonstrate that the attention mechanism successfully "tracks" pathologies. For instance, when generating terms for a pacemaker or diaphragm elevation, the attention weights correctly localized to the relevant anatomical or device regions.
Key Insights
- Characters over Words: Operating at the character level eliminates the need for a dictionary and improves the handling of technical terms like "両肺" (both lungs) versus "両側肺".
- Domain-Specific Pre-training: Pre-training the encoder on medical images is essential; general ImageNet pre-training is insufficient for the nuances of chest radiography.
Conclusion and Future Work
While the model successfully generates interpretable Japanese findings, it faces limitations. The inherent "normal" bias in medical datasets remains a challenge, and the model currently lacks the ability to perform temporal comparisons with previous scans—a common clinical requirement. Future research will focus on developing evaluation metrics that prioritize clinical correctness over linguistic similarity, as even minor linguistic errors can have significant clinical implications.
Summary updated