Knee alignment is reported as the angle between the thigh and shin bones on an X-ray, and it drives decisions in osteoarthritis care and knee replacement planning. Neural networks can now place the bone landmarks that define this angle about as reliably as surgeons do. They share a blind spot, though. An X-ray flattens a 3D leg into a 2D picture. If the patient's leg is turned inward or outward by 10–20 degrees, the measured angle can shift by several degrees even when the bone itself is unchanged. Each hospital has its own habits for positioning feet and aiming the beam, so this rotation error differs by site. It is a plausible reason why measurement tools validated at one hospital have not yet been shown to work at others.
We propose a landmark-based measurement network that estimates, for each leg, how far it is rotated out of the image plane. It then reports the angle the leg would show at neutral rotation. The network uses the patient's other leg in the same film as a built-in reference. Left and right bones are nearly mirror images in shape, so differences in how the two legs look on the film are mostly caused by rotation rather than by anatomy. The central test is whether this rotation-aware, two-leg measurement transfers across the institutions of two large knee cohorts better than current single-leg landmark systems and better than simply training with heavy augmentation.
Core Claim
For automated alignment angles on 2D radiographs, the error that fails to transfer between sites is mainly geometric, caused by out-of-plane limb rotation, rather than differences in image appearance. The contralateral limb supplies a patient-specific signal that separates rotation from true deformity. What is new is the use of the opposite side. Prior two-sided methods treat the opposite organ as a healthy tissue reference for detecting disease. Here it serves as a reference for projection geometry during continuous angle measurement.
This change also reverses the usual failure mode of symmetry priors. Chen et al.'s Asymmetric-Sensitive Transformer detects early femoral head necrosis by comparing the two hips. Because it relies on differences between sides, it fails when disease is bilateral and symmetric. Our rotation estimate relies on the two bones being similar in shape. Symmetric bilateral osteoarthritis therefore helps rather than hurts. The case that breaks our method is strongly one-sided deformity, and we stratify every result by measured asymmetry for that reason.
Why This Gap Is Real
Three findings in the literature motivate the work. Sun et al. projected CT reconstructions of 45 limbs at controlled internal rotations of 0°, ±10° and ±20°. Rotation shifted the mechanical hip–knee–ankle angle by up to 4.84 ± 0.66° and the anatomical femorotibial angle by up to 7.35 ± 0.88°, and the shifts grew with flexion contracture and coronal deformity. Those are exactly the patients who most need accurate measurement.
The attention-gated stacked-hourglass system in Deep Learning-Based Alignment Measurement in Knee Radiographs places over 100 landmarks on single-knee crops and computes the anatomical tibiofemoral angle. It assumes a planar projection, and the authors list the lack of an independent external dataset as its key limitation. The review by Hu et al. names noisy multi-center settings and out-of-plane rotation as open problems for the field.
A fourth line of work constrains the design. Normalization methods built for segmentation achieve correspondence by bending images. Examples are the dense displacement fields of Hu et al. and the atlas-affine feature alignment of Li et al., which prevented a ≥12% cross-institution Dice drop. Non-rigid warping changes angles and distances, and those are the quantities we must measure. So our normalization must preserve angles in the image plane, and the correction for rotation out of the image plane must be explicit rather than implied by a warp.
Research Program
Step 1: a shared frame for both legs. The pipeline first maps each bilateral radiograph into one common frame for both legs. It predicts a single similarity transform (rotation, translation and uniform scale) anchored to the line joining the two hip or knee centers. The design follows the learned affine canonicalization of CIA-Net, restricted so that angles are unchanged. The left limb is then mirrored onto the right.
Mirroring is purely geometric; the measurement arithmetic does not depend on it. Its purposes are to give the detector a consistent input and to establish point-for-point correspondence between the two legs. Naive mirroring without alignment misregisters anatomy, as Zhao et al. and Liang et al. both found. That is why we align before comparing.
Step 2: a two-leg network. One shared-weight network processes both limbs, in the style of the mirrored-patch Siamese model of Tiulpin et al.. We use it for landmarks and geometry rather than for grades. Cross-attention links corresponding locations on the two legs, as in SEAN.
The network has two outputs per limb. The first is landmark heatmaps, from which the anatomical tibiofemoral angle (and the hip–knee–ankle angle on long-leg films) is computed geometrically, keeping the measurement interpretable. The second is an estimate of the limb's out-of-plane rotation with an uncertainty. A small correction head then maps the measured 2D angle, the estimated rotation and a measure of deformity severity to a neutral-rotation angle. Including deformity severity reflects Sun et al.'s finding that rotation error depends on it.
Step 3: supervision for rotation. Real radiographs never come with rotation labels, so the rotation head is supervised on synthetic projections. We will collect a lower-limb CT cohort of roughly 100 limbs at two sites, following Sun et al.'s protocol. From it we render bilateral digitally reconstructed radiographs, meaning simulated X-rays computed from the CT volume. Each leg is rotated independently, so the network sees both shared rotation (both feet turned the same way by protocol) and different rotation between legs. The true neutral-rotation angle is known exactly.
The landmark head is trained on real annotated films. The rotation head is trained on the renderings and regularized by a soft symmetry prior. This prior is a learned distribution of left–right shape differences estimated from healthy knees, not a hard mirror constraint, which follows the warning about natural asymmetry in Fineide et al.
Step 4: data for cross-site testing. The multi-center setting uses the Multicenter Osteoarthritis Study (MOST) and the Osteoarthritis Initiative (OAI). Both contain bilateral knee films from several clinical sites, and they have already served as a train/external-test pair in Tiulpin et al. Neither cohort ships landmark annotations, so we will annotate a stratified subset from each cohort and each clinical site. Two raters will annotate part of the subset to establish human agreement.
Baselines
Do-nothing reference. A unilateral landmark network trained on one cohort with standard augmentation and applied unchanged to the other. This is the realistic current practice.
Strongest existing method. A reimplementation of the attention-gated two-stage stacked hourglass from Deep Learning-Based Alignment Measurement in Knee Radiographs. On its own single-center replacement cohort it reports:
- a mean relative point-to-point landmark error of 1.7% pre-operatively, against 4.1% for the older regression-voting baseline;
- pre-operative agreement with manual measurement of ICC > 0.9;
- a mean absolute difference of about 1° and a Bland–Altman bias below 1°.
Hu et al.'s review reports femorotibial-angle mean absolute errors as low as 0.8° with ICC > 0.95 across the literature it surveys. For cross-site transfer of any automated alignment tool on OAI/MOST, and for repeat-visit variance of automated angles, no reported result was found on Lacuna.
We will train this baseline on identical images, landmark annotations, compute budget and hyperparameter search budget as our model. It will receive the same augmentation, including synthetic in-plane rotation and scaling. It will also receive the same rendered projections as extra training images for its landmark head. That way any advantage we show comes from estimating rotation and using the opposite leg, not from extra data.
Criterion for success. Beating this baseline means three things:
- lower mean absolute error than the baseline on the external cohort, with a paired bootstrap confidence interval excluding zero;
- a smaller increase in error when moving from the internal to the external cohort;
- lower repeat-visit variability, as defined under Decisive Checks.
Ablations isolating the key ingredient. Three ablations each remove one part:
- Rotation head removed: our two-leg model without the rotation and correction heads.
- Opposite leg removed: a unilateral model that keeps the rotation and correction heads but estimates rotation from one leg only.
- Normalization removed: the full model with the similarity normalization replaced by fixed crops.
Why not just…?
Why not just train with heavy rotation augmentation? Two-dimensional augmentation can simulate rotation within the image plane, which does not change angles anyway. It cannot simulate a leg turning out of the plane, which is the actual source of error in Sun et al. Li et al.'s cross-institution drop despite conventional augmentation is suggestive but comes from a different task, so we do not treat it as settling the question. We give the augmentation baseline the same rendered out-of-plane projections. If it matches our repeat-visit and rendered-projection errors, the correction head is unnecessary and we will say so.
Why not just estimate rotation from each leg alone? This is the most serious threat. Single-leg cues such as how much the fibular head overlaps the tibia or where the kneecap sits may carry most of the rotation signal. The opposite-leg ablation decides it. We expect the two-leg model to help most in two situations: when both legs are turned by the same protocol-driven amount, and when deformity makes single-leg cues ambiguous. If the unilateral rotation-aware model matches ours across asymmetry strata, the contribution reduces to explicit rotation correction, which is still useful but a narrower result.
Why not just regress the angle directly with a CNN? Direct regression can learn to absorb a site's typical rotation. It then fails when protocols change, and it gives clinicians no landmarks to verify. Explicit keypoints with geometric computation reached ICC 0.90–0.97 against surgeons on lumbar films in Yuan et al., which supports the landmark route for continuous measurement. We will still train a direct-regression baseline under the same budget and report its cross-cohort gap.
Why not just use long-leg films and the mechanical axis? Sun et al. show that the hip–knee–ankle angle is less sensitive to rotation than the femorotibial angle, but it still shifts by up to 4.84°. Long-leg films also add radiation dose and workflow cost. We will run the method on OAI long-leg films as well, to measure how much correction the mechanical axis still needs.
Decisive Checks
Exact ground truth on rendered projections. On held-out rendered projections, where the neutral-rotation angle is exact, we plot angle error against applied rotation and deformity severity. The rotation-aware model should flatten the error curve that Sun et al. measured. The unilateral model should flatten it less when both legs are turned together.
Paired CT and radiograph subset. On real standing radiographs with same-period CT from the collected cohort, we compare the corrected angle with the CT-derived neutral angle. This tests whether training on rendered images carries over to real films.
Repeat-visit consistency (label-free). The most scalable check needs no labels. OAI images the same knees at repeated visits. For knees whose Kellgren–Lawrence grade stays unchanged and that have no surgery between visits, true alignment should barely move. The visit-to-visit standard deviation of an automated angle therefore mostly reflects positioning noise, and a working rotation correction should shrink it. We report this per site.
Cross-site transfer. We train on MOST and test on OAI, and the reverse. Where site identifiers exist, we also hold out one clinical site at a time. Following Subbaswamy et al., we audit which subgroups fail: severe varus, flexion contracture, and high left–right asymmetry.
Outcome for falsification. The claim fails if two conditions both hold on the external cohort: repeat-visit variability does not drop relative to the hourglass baseline, and the opposite-leg ablation ties the full model. In that case the premise that rotation dominates cross-site error is wrong, and appearance-level fixes should take priority.
References
- AsT: An Asymmetric-Sensitive Transformer for Osteonecrosis of the Femoral Head Detection (Student Abstract)
- Effect of limb rotation on radiographic alignment measurement in mal-aligned knees
- Deep Learning-Based Alignment Measurement in Knee Radiographs
- The Role of Radiographic Knee Alignment in Knee Replacement Outcomes and Opportunities for Artificial Intelligence-Driven Assessment
- Weakly-supervised convolutional neural networks for multimodal image registration
- FEW-SHOT Image Segmentation for Cross-Institution Male Pelvic Organs Using Registration-Assisted Prototypical Learning
- Align, Attend and Locate: Chest X-Ray Diagnosis via Contrast Induced Attention Network With Limited Supervision
- Contralaterally Enhanced Networks for Thoracic Disease Detection
- Symmetry-Enhanced Attention Network for Acute Ischemic Infarct Segmentation with Non-Contrast CT Images
- Automatic knee osteoarthritis diagnosis from plain radiographs:a deep learning-based approach
- Predicting an unstable tear film through artificial intelligence
- Artificial intelligence automatic measurement technology of lumbosacral radiographic parameters
- A data-driven framework for identifying patient subgroups on which an AI/machine learning model may underperform