Vocoder Critics for Heart Contours: Learning Shape from Scribbles and a Handful of Masks

Doctors rarely have time to trace organ outlines pixel by pixel. They can, however, draw a quick stroke inside each structure, called a scribble, or fully outline a few scans. Networks trained this way do well in the middle of an organ but poorly at its edges. Nothing tells them what the boundary between two scribbles should look like, so they hedge. Their contours come out too round and too smooth where real anatomy has sharp corners, or noisy where they should be clean. Speech synthesis has the same problem in another form. A generator trained only to minimize pointwise error outputs the average of all plausible waveforms and sounds muffled. The field's fix is a learned critic, a discriminator that judges whether the output's time-frequency statistics look real. The critic only helps if it looks at the signal in a representation that exposes the relevant detail.

This note proposes treating a predicted cardiac segmentation as a sound. Each 2D slice is resampled around the left-ventricle center, which turns every structure into a periodic one-dimensional signal along the angle: how far out it sits and how thick it is at each angle. That signal is then judged with the multi-period and wavelet critics developed for vocoders. Because the critic sees a few hundred numbers per slice rather than a full image, it should learn a usable shape prior from very few dense masks. That is exactly the resource the annotation bottleneck makes scarce. The claim is testable: as the bank of real masks shrinks, a contour-signal critic should hold up while a spatial mask critic degrades.

Core Claim

Adversarial shape priors for label-efficient segmentation already exist, but they look at masks as images. The closest work is the semi-supervised cardiac framework of A Semi-Supervised Framework for Cardiac MRI Segmentation via Multi-Constraint Collaborative Self-Training. It trains global and region-specific convolutional discriminators on predicted probability maps against expert masks. It reaches 0.730 Dice with 5% of ACDC labels and 0.801 with 10%, and its authors note that the thin myocardium and the right-ventricle (RV) crescent remain hard.

Our claim is that the bottleneck is the critic's representation, not the presence of a critic. The vocoder literature motivates this directly. An Investigation of Time-Frequency Representation Discriminators for High-Fidelity Vocoder shows that critics built on a fixed-resolution Fourier transform miss sharp transients. Replacing or complementing them with constant-Q and wavelet critics cut pitch error from 56.5 to 36.1 F0 RMSE and raised singing MOS from 3.30 to 3.80 on HiFi-GAN.

The two corners where the RV wall meets the septum are the anatomical analogue of a transient: a localized, sharp change in an otherwise slowly varying contour. An angular wavelet critic should see these corners, while a downsampled spatial critic averages them away. A low-dimensional, rotation-structured input should also make the critic far less hungry for real masks. That second effect is what ties the vocoder lesson to the annotation bottleneck.

Research Program

The first component is a differentiable contour signature. The soft softmax output is resampled onto a polar grid centered on the predicted left-ventricle centroid. Let $p_k(\theta,\rho)$ be the probability of structure $k$ at angle $\theta$ and radius $\rho$. For each angle we compute two quantities. The first is the radial extent $e_k(\theta)=\sum_\rho p_k(\theta,\rho)\,\Delta\rho$, which is the thickness of structure $k$ along that ray. The second is the probability-weighted mean radius $m_k(\theta)=\sum_\rho \rho\, p_k(\theta,\rho)\,\Delta\rho / e_k(\theta)$. Together, $e_k$ and $m_k$ describe the left-ventricle cavity, the myocardial ring, and the RV crescent, which is simply zero over angles it does not cover.

The signature is periodic in $\theta$, works for rings and crescents, not only blob shapes, and is computed identically for predictions and for the real mask bank. Gradients flow through it back into the segmenter.

The second component is the critic ensemble, transplanted from vocoding with minimal change. It has three parts. A multi-period critic reshapes the angular signal at several prime periods, following the multi-period discriminators used in Two-stage Neural Network for ICASSP 2023 Speech Signal Improvement Challenge and DurIAN-E 2. This exposes angular regularities such as near-circular cavity shape and roughly uniform wall thickness. A continuous-wavelet critic, with periodic padding along $\theta$ and a temporal compressor as in the vocoder paper, targets the localized RV insertion corners. A plain Fourier critic covers global harmonic content. Following BigVSAN, we use the softplus "soft monotonization" of the least-squares adversarial loss, which keeps training stable. We adopt it as an engineering choice, not a contribution.

The third component is the training regime that links all of this to the annotation bottleneck. The segmenter is a standard 2D U-Net. It is trained with partial cross-entropy on scribbles from every training scan, plus the adversarial loss from the contour critics. The critics see real signatures computed from a small mask bank whose images are excluded from scribble training, so the masks act purely as an unpaired shape prior. We vary the bank size from a full split down to about 1% of ACDC volumes. Separately, we run the standard semi-supervised protocol, with dense masks on 5% or 10% of scans, no scribbles, and unlabeled scans otherwise. This gives a direct comparison with the multi-constraint framework.

The total objective is partial cross-entropy plus $\lambda$ times the critic loss, with one weight $\lambda$ tuned on validation. One optional extension uses the critic score to accept or reject pseudo-labels. This is analogous to the discriminator-derived importance weights of Robust Histopathology Image Analysis: To Label or to Synthesize?. It will be reported only if the core result holds.

The proposal reuses its methods from vocoding and takes its motivation and constraints from segmentation. Learning to Segment Medical Images with Scribble-Supervision Alone shows that scribble training plus a Conditional Random Field (CRF) post-processor, a classical model that pulls labels toward intensity edges, gets within about 3 to 5 Dice points of full supervision. It also shows that the remaining error is concentrated at boundaries and that hand-set CRF parameters are fragile. ScribFormer attacks the same gap with a heavier CNN-Transformer architecture. Our approach leaves the architecture alone and changes only the training signal at the boundary.

Spectral U-Net and Frequency Domain Adversarial Training for Robust Volumetric Medical Segmentation use wavelets or a discrete cosine transform inside the network or in an attack-and-defend loop. Both assume fully dense labels and apply the transform to the image, not to the predicted shape. We do the reverse on both counts.

Baselines

The naive reference is the same U-Net trained with partial cross-entropy on scribbles alone, with no critic, CRF, or shape prior. It defines how much boundary information scribbles fail to provide. A second classical reference is the Random Walk plus CRF plus Monte Carlo dropout pipeline of Can et al. That paper reports 88.0% Dice on ACDC and 78.1% on NCI-ISBI prostate, against 91.2% and 81.8% for full supervision. We rerun it on our splits rather than copying these numbers, because splits differ across papers.

The strongest scribble-only competitor is ScribFormer, which reports 0.888 Dice on ACDC against 0.848 for the previous best scribble method, at 13.96 s per case. We run its released configuration on our splits with the same epoch budget. Beating it means matching or exceeding its Dice with a plain U-Net while improving the boundary metric. Note that ScribFormer receives no mask bank, so we also report our method at its smallest bank size to keep the comparison honest about the extra information we use.

In the semi-supervised setting, the strongest competitor is the multi-constraint framework, with 0.730 Dice at 5% and 0.801 at 10% on ACDC. Its own reported Mean Teacher and Deep Co-Training results, 0.654 and 0.711 at 5%, serve as weaker references. We reimplement it with identical labeled subsets.

Boundary-distance results such as 95th-percentile Hausdorff distance (HD95): no reported result found on Lacuna for any of these baselines. We will measure HD95 ourselves for all methods.

The key ablation isolates the representation. We keep the segmenter, scribbles, mask bank, loss form, and critic parameter count fixed, and swap the contour-signal critic for a spatial mask critic modeled on the multi-constraint paper's global and local discriminators. A second variant applies a 2D wavelet critic directly to the mask image. Further ablations remove the wavelet branch, then the multi-period branch, to test whether the vocoder-specific critics matter or any 1D critic suffices.

Fairness rules apply throughout. All methods use the same preprocessing and the same training-step budget, and every method gets the same number of hyperparameter trials, about twenty, on a held-out validation split. Results are reported over three seeds.

Why not just…?

Why not just use the spatial mask discriminator that semi-supervised methods already use? This is the most direct threat, and the main ablation is designed to settle it. The multi-constraint paper's own observation that thin myocardium and RV structures remain hard is weak evidence that spatial critics miss boundary detail. It does not prove that the representation is the cause. The proposal predicts that the gap between contour and spatial critics is small with a large mask bank and widens as the bank shrinks. If the two curves coincide at 1% and 5% banks, the central claim is false, and we will report it as such.

Why not just penalize high-frequency content in the contour, a smoothness prior with no adversary? This is cheaper, stable, and might capture much of the gain. The vocoder evidence says pointwise or low-pass objectives produce the conditional mean and wash out sharp structure. Effectiveness of Generative Adversarial Network for Non-Audible Murmur-to-Whisper Speech Conversion and Opencpop both report muffled outputs that disappear only once an adversarial loss is added. For hearts, this predicts rounded RV corners. We test it with a tuned Fourier-descriptor smoothness penalty and with a principal-component shape model fit to the same mask bank, both on the same signatures. We measure error specifically at the RV insertion corners alongside overall Dice. If either matches the critic on corner error, the adversary is unnecessary.

Why not just use a stronger architecture or a CRF? ScribFormer shows architecture alone reaches 0.888 on ACDC, so this alternative is real. The two approaches address different things, though. A Transformer enlarges what the network sees, but it receives no boundary signal between scribbles. A CRF supplies one, but from hand-set intensity rules that Can et al. found fragile. We test whether the critic adds to architecture by training ScribFormer itself with the contour critic. If the gain disappears on top of ScribFormer, the method is only a cheap substitute for capacity, which is a weaker but still useful result.

Proposed Experiments

The primary benchmark is ACDC, under two protocols on one fixed split. In the scribble protocol, with the mask bank swept from full to about 1%, we report Dice and HD95 per structure and corner localization error at the RV insertion points. We also report a spectral-fidelity score: the distance between the angular wavelet power spectra of predicted and real signatures, the shape analogue of pitch error in vocoding. In the semi-supervised protocol, at 5% and 10% dense labels, we compare against our reimplementation of the multi-constraint framework, with 0.730 and 0.801 Dice as the published reference points. NCI-ISBI prostate serves as a second, simpler-shape test, where we expect smaller gains because the gland lacks sharp corners. Finding the largest gains there would suggest the mechanism is not what we claim.

Three results would decide the question. The first is the bank-size curve for contour-signal versus spatial critics. The second is corner error for the critic against the smoothness and principal-component priors. The third is whether gains persist when the critic is added to ScribFormer. We will also check one known risk carried over from Robust MelGAN: a critic that pushes for detail can add small spurious artifacts on easy inputs. For this, we count spurious connected components on slices where the scribble-only baseline is already near-perfect.