Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generation

Zhongyi Han, Benzheng Wei, Yilong Yin, Shuo Li

TL;DR

The Neural-Symbolic Learning (NSL) framework automates the generation of spinal radiology reports by successfully marrying deep learning's perceptual strengths with symbolic logic's reasoning capabilities. By utilizing an adversarial graph network for precise anatomical segmentation and meta-interpretive learning for unsupervised causal analysis, NSL generates radiologist-level diagnostic reports from raw MRI scans without requiring a massive corpus of ground-truth text reports for training.

What Problem Does This Paper Address?

In spine radiology, generating diagnostic reports from magnetic resonance imaging (MRI) is a time-consuming, manual task prone to delays. While deep learning has excelled at computer-aided detection, automating full clinical report generation remains incredibly challenging due to two primary hurdles:

  1. Structural Complexity and Variability: A typical lumbar spine MRI contains numerous highly variable structures (vertebrae, intervertebral discs, and neural foramina) with ambiguous spatial correlations, making accurate pixel-level identification difficult.
  2. Inexact Supervision: Training end-to-end image captioning models requires massive datasets of annotated text reports. Furthermore, standard neural networks struggle to learn the high-level causal relationships (e.g., how a deformed disc causes nerve stenosis) that are vital for a correct clinical diagnosis.

The authors address these gaps by moving away from pure end-to-end deep learning, instead designing a hybrid system that mimics the human radiologist's process of visual perception followed by logical deduction.

Spine anatomy and target structures to be analyzed, including vertebrae (V), discs (D), and neural foramina (NF).

Methods

The NSL framework splits report generation into a two-stage pipeline: Neural Learning for visual perception and Logic Reasoning for clinical deduction.

+-----------------------------------------------------------------------+
|                         NSL FRAMEWORK PIPELINE                        |
|                                                                       |
|  [Raw MRI] ---> (Adversarial Graph Network) ---> [Segmented Organs]   |
|                                                          |            |
|  [Report]  <--- (If-Then Templates) <--- (Meta-Interpretive Learning) |
+-----------------------------------------------------------------------+

1. Neural Learning (Adversarial Graph Network)

To handle complex spinal structures, the authors insert a Symbolic Graph Reasoning module directly into the latent space of a Generative Adversarial Network (GAN).

  • The Generator uses an autoencoder structure with adaptive dilated convolutions to capture multi-scale features. The graph reasoning module embeds a prior knowledge graph of spine anatomy into this network, forcing the model to learn valid spatial constraints.
  • The Discriminator evaluates the predicted segmentation maps against ground truth, pushing the generator to correct fine-grained errors and maintain global structural continuity.

2. Logic Reasoning (Symbolic Logic & Causal Analysis)

Once the structures are segmented, the framework transitions to symbolic AI:

  • Unsupervised Labeling: Implemented in Prolog, first-order logic programming automatically sorts and labels the isolated segments sequentially (from L1 to L5) based on deterministic anatomical rules (e.g., intervertebral discs must alternate with vertebrae).
  • Causal Analysis via Meta-Interpretive Learning (MIL): Using inductive logic programming and a clinical knowledge base, the system reasons out the causal relationships between detected abnormalities (such as Intervertebral Disc Deformation or Lumbar Vertebral Deformation) and target conditions like Neural Foraminal Stenosis (NFS).
  • Template Generation: The inferred causal links are plugged into an If-Then logical template to output the final text report.
Figure 2: An illustration of the proposed neural-symbolic learning framework.

Datasets

The framework was evaluated on a real-world clinical dataset consisting of:

  • 253 clinical patients (147 females, 106 males) with an average age of 53±38.
  • Mid-sagittal lumbar spine MRI scans (T1/T2-weighted sequences).
  • Ground-truth annotations providing pixel-level masks for 17 target structures per image, alongside diagnostic classifications double-checked by board-certified radiologists.

The data was split into an 80/20 ratio for training and testing, utilizing standard five-fold cross-validation.

Key Insights and Ideas

  • The Power of Hybrid AI: Pure connectionism (neural networks) is excellent at pixel-level perception but terrible at abstract reasoning. Conversely, symbolism (logic programming) excels at reasoning but fails on noisy image data. NSL demonstrates that integrating them inside a single pipeline creates a mutually beneficial system where the neural network "sees" and the symbolic system "thinks."
  • Latent Space Knowledge Injection: Instead of applying anatomical constraints as a post-processing step, embedding the symbolic graph reasoning module between the encoder and decoder allows prior domain knowledge to actively guide feature representation learning.
  • Bypassing Data Scarcity: By using logic templates driven by causal reasoning, the model completely bypasses the need for thousands of paired text reports, making automated report generation viable even with small medical datasets.

Key Results

The NSL framework demonstrated superior performance in both anatomy segmentation and diagnostic accuracy:

  • State-of-the-Art Segmentation: NSL achieved a 96.5% pixel accuracy and an 0.879 Dice coefficient, significantly outperforming standard architectures like FCN (91.7% / 0.754) and U-Net (92.0% / 0.797), as well as the specialized Spine-GAN.
  • Robust Ablation Proof: Removing the symbolic graph reasoning module caused the Dice coefficient to drop from 0.879 to 0.863, validating that prior knowledge injection directly improves segmentation quality.
  • Perfect Logical Sequencing: Given accurate segmentations, the first-order logic programming achieved 100% accuracy in sequentially labeling spinal structures and successfully mapped out correct clinical causal pathways (e.g., identifying when an abnormal L5 disc was the primary pathogenic factor behind L4-L5 nerve stenosis).

Limitations and Future Work

  • Rigid Text Output: Because the final reports are generated via If-Then templates, they lack the stylistic fluidity and linguistic variation of natural human dictation.
  • Error Cascade Vulnerability: The performance of the logical reasoning phase is strictly dependent on the neural network's segmentation accuracy. Severe segmentation failures will inevitably cascade into incorrect causal deductions.
  • Anatomical Specialization: The logic rules and background knowledge bases used in this study are highly specific to the lumbar spine. Scaling this framework to other complex organs or multi-disease conditions would require significant manual curation of new symbolic rules.

Summary updated

Abstract

Automated medical report generation in spine radiology, i.e., given spinal medical images and directly create radiologist-level diagnosis reports to support clinical decision making, is a novel yet fundamental study in the domain of artificial intelligence in healthcare. However, it is incredibly challenging because it is an extremely complicated task that involves visual perception and high-level reasoning processes. In this paper, we propose the neural-symbolic learning (NSL) framework that performs human-like learning by unifying deep neural learning and symbolic logical reasoning for the spinal medical report generation. Generally speaking, the NSL framework firstly employs deep neural learning to imitate human visual perception for detecting abnormalities of target spinal structures. Concretely, we design an adversarial graph network that interpolates a symbolic graph reasoning module into a generative adversarial network through embedding prior domain knowledge, achieving semantic segmentation of spinal structures with high complexity and variability. NSL secondly conducts human-like symbolic logical reasoning that realizes unsupervised causal effect analysis of detected entities of abnormalities through meta-interpretive learning. NSL finally fills these discoveries of target diseases into a unified template, successfully achieving a comprehensive medical report generation. When it employed in a real-world clinical dataset, a series of empirical studies demonstrate its capacity on spinal medical report generation as well as show that our algorithm remarkably exceeds existing methods in the detection of spinal structures. These indicate its potential as a clinical tool that contributes to computer-aided diagnosis.
BibTeX
@article{han2021unifying,
  author = {Zhongyi Han and Benzheng Wei and Yilong Yin and Shuo Li},
  title = {{Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generation}},
  journal = {Medical Image Analysis},
  year = {2021},
  doi = {10.1016/j.media.2020.101872},
}
Figure