When a brain tumor grows, it does two things to the image. It adds abnormal tissue that has no counterpart in a healthy brain, and it pushes the surrounding healthy tissue out of place: ventricles get squeezed and the midline bends. Methods that reconstruct a "pseudo-healthy" version of a patient scan only address the first effect. These are generators trained only on healthy scans, which then replace whatever looks abnormal with normal-looking tissue. They paint gray and white matter over the tumor and leave every displaced structure where the tumor pushed it. Registration methods, which estimate a smooth spatial warp between two scans, have the opposite problem. They can move tissue, but they either collapse the tumor or mask it out. The warp they produce also mixes tumor-caused displacement with ordinary differences between people's anatomy.
We propose a model that splits each pathological scan into three outputs: a lesion mask, an appearance correction, and an explicit displacement field that undoes the tumor's push. It is trained entirely on healthy scans into which we insert simulated lesions that both change intensities and deform their surroundings. No expert ever draws a mask. The result is a patient-specific pseudo-healthy scan with structures returned to plausible positions. It also yields two complementary anomaly signals: what looks wrong (the intensity residual) and what was moved (the displacement field). The central bet is that tumor-caused displacement can be separated from normal anatomical variability. Healthy training brains already contain normal variability, while the simulator adds only lesion-centered displacement. A model trained this way should learn to remove the second without erasing the first.
Core Claim and What Is New
The closest existing work comes from two lines of research. The first is joint segment–inpaint–register pipelines. GIRNet co-trains an unsupervised segmenter, an inpainter and a registration network to align a tumor scan to a healthy atlas. Symmetric-constrained inpainting fills a given tumor mask before running a standard registration tool. The research file notes that inpainting "does not mathematically invert the physical mechanical displacement" in GIRNet. Its registration does move tissue, but toward an atlas, so the warp absorbs the patient's whole anatomy and not just the tumor's push. The second line is synthetic-anomaly pseudo-healthy synthesis. UNA simulates lesions with advection–diffusion equations inside healthy scans and reconstructs normal anatomy with a mirrored-hemisphere input. Its lesions are purely intensity changes, it outputs no deformation, and its symmetry prior breaks under midline shift. We take UNA's label-free synthetic-lesion recipe and change two things. The simulator adds a mass-effect displacement, and the model outputs that displacement explicitly, without relying on bilateral symmetry.
Concretely, we take a healthy scan $x$ and sample a lesion mask $m$, a lesion appearance $a$, and a displacement field $\psi$. The field radiates from the lesion and decays with distance; its amplitude is randomized and loosely tied to lesion volume. We then form a training input
$$y = (1-m)\odot(x\circ\psi) + m\odot a,$$
where $x\circ\psi$ is the healthy scan warped by $\psi$ and $\odot$ is voxelwise multiplication. A network $f$ maps $y$ to $(\hat m, \hat\psi^{-1}, \hat x)$ and is trained to recover the true mask, the inverse field, and the original healthy scan. At test time, $\hat x$ is the pseudo-healthy scan in the patient's own space with the push undone. The intensity residual and the log-Jacobian of $\hat\psi^{-1}$ give two anomaly maps. The Jacobian map is the local volume change of the field, which Dong et al. used as a by-product of registration failure. Here it is a trained output, so it should not depend on a baseline network happening to squash the lesion.
Research Program
The simulator is the first deliverable. It combines UNA-style flow-shaped lesion masks with displacement fields of two kinds. Some are smooth radial pushes with tissue-dependent stiffness, in the spirit of the radial growth prior of Bach Cuadra et al. and the statistical deformation model of Mohamed. We randomize their parameters instead of fitting them per patient, which avoids the parameter-tuning burden those methods carry. A fraction of samples have zero displacement, standing in for infiltrative or stroke-like lesions, and some samples are multifocal. Everything is generated from the healthy cohorts UNA uses (ADNI, HCP, OASIS3 and others), so no expert annotation enters training.
The network is a 3D encoder with three heads: mask, inverse field, and appearance. The field is parameterized as a diffeomorphism, a smooth invertible warp obtained by integrating a velocity field. This keeps the inverse well defined. The appearance head inpaints only in the warped-back mask region and copies the input elsewhere. That restriction blocks the "identical shortcut," the failure in which a high-capacity reconstruction network simply reproduces the anomaly. Losses are L1 on $\hat x$, endpoint error on the inverse field, and Dice on the mask, all against simulator ground truth. A cycle term checks that re-applying $\hat\psi$ and re-inserting the lesion reproduces $y$.
The analysis then asks whether the learned field is specific to the lesion. On held-out healthy scans with no lesion inserted, the predicted field should be near identity. Fields inserted far from a lesion should not be partially "corrected" either. If the model instead pulls every brain toward a mean shape, the decomposition has failed. Ruling this out is the main identifiability risk.
Baselines
The naive reference for registration is affine alignment only. On BraTS-Reg, the BraTS-Reg 2022 winner reports the median landmark error falling from 7.8 mm initially to 4.18 mm after affine. For pseudo-healthy synthesis, the naive reference is the unchanged input, which gives zero ventricle restoration and zero midline-shift correction.
The strongest pseudo-healthy competitor is UNA. We will retrain it on exactly our healthy cohorts and give it the same compute and tuning budget. The research file reports a roughly 10% reconstruction gain over SynthSR, Brain-ID and PEPSI on ATLAS, but no Dice value: no reported result found on Lacuna for UNA's lesion Dice. We therefore rerun it rather than quote it. Beating UNA means two things: equal or better reconstruction and lesion Dice on ATLAS/ISLES, where mass effect is small, and clearly lower ventricle and midline errors on glioma data.
The strongest joint pipeline is GIRNet, which reports a mean deformation error below 1.0 mm and an unsupervised Dice of 0.83 on its grafted-tumor pseudo dataset. We will run it on our simulated test set and on BraTS 2020. Its atlas warp will be decomposed into a "tumor-attributable" part by subtracting the warp GIRNet produces for the same subject without the lesion. This gives it the fairest possible version of a mass-effect estimate.
For registration with absent correspondence, the references are DIRAC, which reports a mean target registration error of 3.31 mm near tumors and 1.91 mm far from them, and the BraTS-Reg winner, with a validation median absolute error of 1.64 mm and 88% robustness. We will use our model as a front end for each: pseudo-healthy pre-operative scans feed the same cLapIRN plus instance-optimization stack. Beating them means lower near-tumor error at equal far-from-tumor error.
For volume preservation, Dong et al. report a tumor-size ratio of 1.260, against 2.330 for the VTN baseline on LiTS17. We will compare tumor-size ratio on BraTS20 using their four backbones with our mask in place of their Jacobian-derived mask.
The key ablation trains the identical architecture on the identical mass-effect simulator but removes the field head. The appearance head must then absorb the displacement on its own. This ablation isolates whether explicit spatial modeling, and not just better synthetic data, produces the gain. A second ablation trains on UNA-style intensity-only lesions, to measure what the simulated displacement contributes.
Why not just…?
Why not just inpaint the lesion and then register to an atlas, as GIRNet does? Atlas registration does move the displaced tissue, but it moves everything. The resulting warp contains the patient's normal anatomy together with the tumor's push, so there is no patient-specific pseudo-healthy scan and no clean mass-effect measurement. GIRNet's authors also report that registration degrades when the inpainting leaves residual tumor boundary. The decomposed-GIRNet baseline settles this question. If subtracting the lesion-free warp recovers simulated mass effect as accurately as our field head does, our explicit decomposition adds little, and we would report that.
Why not just add mass-effect displacement to UNA's training data and keep an intensity-only output? This is the most serious threat. A strong generator might learn to repaint displaced ventricles in their correct place without ever representing a field. The no-field ablation tests exactly this. We expect it to blur structure boundaries, because large shifts are hard to express as intensity edits, and to produce no usable geometric anomaly map. If it matches our ventricle and midline errors, the claim reduces to "a better simulator," and the proposal would be reframed that way.
Why not just use masking-based registration such as DIRAC, or a fitted biomechanical growth model? Masking produces good landmark alignment but leaves the cavity interior governed only by smoothness and produces no pseudo-healthy image. Biomechanical models need per-patient seeds and stiffness values and assume a single mass-effect lesion. The BraTS-Reg front-end experiment shows whether undoing mass effect before masked registration lowers near-tumor error beyond 3.31 mm (DIRAC) or 1.64 mm median (winner). If it does not, our method's value lies in interpretation and anomaly detection, not in registration accuracy.
Why not just collect cheap weak labels such as RECIST lines or scribbles? Weak labels can give masks (Agarwal et al. reach 90.3% Dice on 2D DeepLesion; Can et al. come within 2.9–4.5% of full supervision). But no annotation, weak or dense, labels where the ventricle would have been without the tumor, and that target is precisely what we predict. As a secondary check, we compare our zero-label masks on BraTS 2017 slices against PathoGAN, which reaches 72.9% Dice with image-level labels against 86.3% under full supervision.
Proposed Experiments
The first test is synthetic and fully controlled. On held-out healthy subjects with simulated lesions, we measure endpoint error of the recovered inverse field, reconstruction error, and mask Dice, against all baselines above. The test set includes a separate held-out family of displacement shapes not seen in training, so that we are not simply grading simulator memorization.
The decisive real-data checks need no labels. On BraTS glioma scans, we measure lateral-ventricle volume asymmetry and midline shift in the pseudo-healthy outputs and compare them with the distribution observed in healthy cohorts. Success means our outputs fall inside the healthy range while UNA and the no-field ablation stay displaced. On ATLAS/ISLES strokes, which mostly lack mass effect, the predicted field should stay near identity and lesion Dice should not fall below retrained UNA. This checks that the model does not invent displacement. On BraTS-Reg, the front-end experiment reports near- and far-tumor landmark error against DIRAC and the challenge winner under identical downstream optimization. On LiTS17 and BraTS20, we report tumor-size ratio against Dong et al.
Finally, the healthy-control check feeds unlesioned scans and synthetic far-field perturbations to the model. It must return near-identity fields there; if it does not, the separation of tumor-caused displacement from normal anatomy has failed, whatever the other metrics show.
References
- Co-learning Semantic-Aware Unsupervised Segmentation for Pathological Image Registration
- Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology
- Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomization
- Preserving Tumor Volumes for Unsupervised Medical Image Registration
- Atlas-Based Segmentation of Pathological Brains Using a Model of Tumor Growth
- Deformable Registration of Brain Tumor Images Via a Statistical Model of Tumor-Induced Deformation
- Robust Image Registration with Absent Correspondences in Pre-operative and Follow-Up Brain MRI Scans of Diffuse Glioma Patients
- Unsupervised Deformable Image Registration with Absent Correspondences in Pre-operative and Post-recurrence Brain Tumor MRI Scans
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Pathology Segmentation Using Distributional Differences to Images of Healthy Origin