Do Radiology Words Track Anatomical Shape? Auditing Report Modifiers in Procrustes Shape Space

Radiologists write words like "enlarged cardiac silhouette," "flattened hemidiaphragms," or "widened mediastinum," and clinical language-processing systems turn those words into structured labels that feed research cohorts and risk models. No one checks whether the words match the anatomy in the image. They may instead reflect how the patient was positioned or how the film was taken. This proposal builds that check. We place landmarks along the outlines of the heart, lungs, diaphragm and mediastinum on chest radiographs. We then use geometric morphometrics, the branch of statistics that compares shapes after removing differences in position, size and in-plane orientation, to separate each image's geometry into two parts: pose (where and how the body sits in the frame) and intrinsic shape (what remains after Procrustes alignment). Finally, we ask which part each extracted report word actually follows.

In two sentences: we extract morphological modifiers from paired chest X-ray reports and regress them against Procrustes-aligned anatomical shape and against the pose parameters that alignment strips away. This tells us, term by term, how much of the language reflects real anatomy and how much reflects acquisition artifact. It also yields a cross-modal verifier that flags report statements the patient's geometry does not support.

Core Claim

Current extraction systems treat a phrase like "mildly enlarged heart" as ground truth once it is parsed correctly. Two examples are RadGraph, which turns reports into graphs of anatomy and observation entities linked by "located at" and "modify" relations, and CheXbert, which labels findings as positive, negative or uncertain. Geometric morphometrics provides the tool these systems lack. Generalized Procrustes Analysis (GPA) centers each landmark configuration, divides it by centroid size, and rotates it onto a consensus shape, as laid out in Sorting the flock. The translation, scale and rotation it removes are not discarded. They become measured nuisance variables.

We claim three things. First, for each morphological term, an ordinal model of the extracted text label on Procrustes shape coordinates, centroid size and pose variables can attribute the term's variation to intrinsic shape versus extrinsic pose. Second, at least some terms show measurable pose dependence, especially on portable anteroposterior films. Third, a verifier that tests extracted modifiers against shape space catches documentation errors that text-only self-verification cannot see.

The closest prior work stops short of this in specific ways. Report-grounded vision models such as MAIRA-2, RIHA and Learning Segmentation from Radiology Reports link sentences to boxes, heatmaps or masks. None of them separates pose from shape. DuPLUS conditions segmentation on structured prompts rather than parsing narratives. Bullard et al. fuse notes with scalar brain volumes, not shape.

A short note on what alignment does and does not remove. If heart, lung and thoracic landmarks are aligned jointly as one configuration, the heart's size relative to the thorax survives alignment as part of the shape. Only global scale, translation and in-plane rotation are removed. This is what makes "enlarged" testable in shape space. Out-of-plane patient rotation is different. It is not removed by 2D Procrustes and will show up as apparent shape change. We therefore estimate it separately from bilateral landmark asymmetry, following the asymmetry-index logic of weighted Procrustes symmetry planes and facial asymmetry scoring.

Research Program

Text side. We run the RadGraph extractor over MIMIC-CXR Findings and Impression sections. For each image we keep the observation entities and their "modify" relations for a fixed vocabulary of about six shape-bearing terms:

  • cardiac silhouette enlargement
  • mediastinal widening
  • diaphragm flattening
  • elevated hemidiaphragm
  • low lung volumes
  • aortic tortuosity

From these we form an ordinal label: absent, uncertain, mild, moderate or severe. We separately tag hedges that name an acquisition cause, such as "rotation," "AP technique" or "low inspiratory effort."

Image side. We place fixed landmarks (costophrenic angles, lung apices, medial clavicle heads, diaphragm dome apices) and sliding semi-landmarks along the heart border, mediastinal contours and diaphragm curves. Semi-landmarks are points allowed to slide along a curve during alignment so that arbitrary spacing does not create fake shape differences. A keypoint network places these points. It is trained on a few hundred hand-landmarked radiographs that we annotate, using the morphology-aware loss of Attend-and-Refine. That paper reports a 37% lower failure rate on the AASCE spinal benchmark when anatomical shape priors are added. Its interactive correction serves as our quality-control tool. The file contains no public chest landmark set, so this annotation is a real cost and a pipeline dependency. We report landmark error on a held-out hand-annotated set before using any downstream result.

Model. For each term $t$ and image $i$, we fit an ordinal logistic model:

$$\mathrm{logit}\,P(y_{it} \le c) = \theta_{tc} - \big(w_t^\top s_i + a_t \log CS_i + b_t^\top p_i\big)$$

The symbols are:

  • $y_{it}$: the extracted ordinal label, with $\theta_{tc}$ the cutoffs.
  • $s_i$: the leading principal-component scores of Procrustes tangent coordinates (intrinsic shape).
  • $CS_i$: centroid size.
  • $p_i$: pose, made up of in-plane rotation angle, image-frame translation and scale, the asymmetry-based out-of-plane rotation proxy, and the AP/PA view flag.

We summarize each term by the share of explained deviance attributable to shape versus pose, using drop-one-block likelihood comparisons.

Verifier. From the fitted model we build an inserted step that sits after the evidence-grounding stage of Self-Verification. That pipeline requires a quoted text span for each extraction. Our step additionally requires the extracted modifier to lie within a calibrated band of the shape-predicted label. If it does not, the step flags the statement as either "pose-explained" or "unsupported by geometry."

Baselines

Naive reference: text extraction taken at face value. The reference point is report extraction with no image check. The strongest numbers in the file are RadGraph's relation-extraction micro F1 of 0.82 and entity micro F1 of 0.94 on MIMIC-CXR (RadGraph), and CheXbert's average F1 of 0.798 against a radiologist benchmark of 0.805 (CheXbert). These numbers measure fidelity to the text, not to the anatomy. Our verifier "beats" this baseline if, on an adjudicated set, its flags have meaningfully higher precision than random flagging for identifying statements radiologists judge geometrically wrong. The baseline cannot flag anything at all.

Strongest text-only verifier. This is the four-stage self-verification pipeline of Gero et al., which raised MIMIC-IV ICD-10 F1 from 0.487 to 0.533. We run it with the same language model and prompt budget, and our geometric check is appended rather than substituted. We beat it if the combined system catches adjudicated errors that self-verification passes, without lowering extraction F1 on reports with no geometric conflict.

Strongest classical measurement. For cardiac enlargement this is the cardiothoracic ratio, computed from the same landmarks. For other terms it is the nearest linear measure, such as mediastinal width over thoracic width or diaphragm dome height over chord length. These baselines are fit with the identical ordinal model and cross-validation folds.

Pixel-level image model. We fine-tune an image classifier to predict each extracted term directly from pixels. It gets the same training split and a hyperparameter budget matched in GPU-hours to our landmark network.

Ablation of the key ingredient. The ablation replaces Procrustes-aligned coordinates with raw image coordinates of the same landmarks. Keeping the landmarks fixed and removing only the alignment isolates whether the shape/pose separation does the work.

No reported result was found on Lacuna for grounding report modifiers in landmark shape, so all shape-side numbers will be new.

Why not just…?

Why not just use the cardiothoracic ratio and other standard linear measurements? For cardiac enlargement this is a serious threat. The ratio is clinically defined to capture exactly that finding, and it may match our shape model. We treat it as a hypothesis to test, not a weakness to explain away. If shape adds nothing beyond the ratio for the cardiac term, we report that and shift the claim to terms without an agreed ratio, such as diaphragm flattening, mediastinal contour and aortic tortuosity. For those, Tan et al. show in another anatomical domain that linear ratios miss multi-point shape differences. Even where a ratio predicts the label well, it is a single number: it cannot separate pose from shape, so it cannot support the attribution analysis.

Why not just train an image-text model end to end? A pixel classifier will likely predict the extracted words better than our low-dimensional shape model. That is because it can learn the same projection and positioning cues that may bias the words. The question here is not prediction but attribution. We test the classifier directly by applying controlled in-plane rotations and rescalings to held-out images and comparing view-matched AP and PA subsets. A model whose outputs shift under these nuisance changes is reproducing the confound, not auditing it. Grounded report generators such as MAIRA-2 also output boxes rather than pose-invariant coordinates.

Why not just use a better language model with self-verification? Self-verification checks that an extraction is quoted in the text. A report that confidently says "enlarged" about a rotated portable film passes every intra-text check. mCODEGPT shows that even strong models still fail on specialized structured attributes. Text-only methods do not fail because they parse badly; they have no access to the anatomy. The adjudication experiment below measures how many geometric conflicts the text-only pipeline passes.

Proposed Experiments

The first check is attribution. For each term, on MIMIC-CXR image-report pairs, we report the share of explained deviance assigned to shape versus pose. The pose share should be larger on AP than PA films. Severity grades (mild, moderate, severe) should fall in monotone order along the fitted shape direction. If they do not, the words are not ordinal in anatomy.

The second check is hedging. Reports that cite rotation or AP technique should show larger measured pose deviation, measured by the asymmetry proxy and the in-plane angle, than view-matched reports without such hedges. If they do, radiologists' stated reasons for uncertainty are geometrically real.

The third check addresses disagreement. The RadGraph authors report Cohen's kappa of 0.397 for relations on CheXpert. We take their doubly annotated reports, link them to images, and test whether reports where the two annotators disagree on a shape modifier sit nearer the fitted decision boundary in shape space than reports where they agree. If so, part of the disagreement is genuine anatomical borderlineness rather than annotation noise.

The fourth and decisive check is the audit. Two radiologists, blinded to which system flagged each case, adjudicate a stratified sample of several hundred statements:

  • statements our verifier flags,
  • statements self-verification alone passes,
  • random controls.

The verifier succeeds if its flags are confirmed as geometrically unsupported or pose-driven at a clearly higher rate than random controls, with confidence intervals reported. It also must not lower extraction F1 on unflagged reports. If the Procrustes-free raw-coordinate ablation matches it, the alignment step contributed nothing, and we will say so.