Undoing the Stretch: Correcting Anisotropic Aspect-Ratio Distortion in Face Recognition Without Discarding Face Shape

Face recognition systems assume that a face's proportions arrive intact. In practice they often do not. Video encoders, capture cards and display pipelines routinely squeeze or widen frames, for example by playing 4:3 footage as 16:9 or by using non-square pixels, and the face inside is stretched horizontally or vertically. The standard way to make a model tolerate this is augmentation: stretch the training faces at random so the network learns to ignore width-to-height proportions. That fix has a hidden cost. How wide a face is relative to its height, and how the eye spacing compares with the eye-to-mouth distance, are genuine identity cues. A network trained to ignore stretch is also trained to ignore face shape. When training identities are scarce, the model has fewer cues to spare, so this loss should matter most there.

We propose to estimate the stretch rather than ignore it. A small module predicts a single horizontal-to-vertical scale factor from cues that the stretch distorts but identity does not: the near-circular iris, and the surrounding head, neck and scene seen at their native aspect ratio. A one-parameter warp then undoes the stretch before the identity network sees the face. The estimator is trained as an auxiliary task with free labels, because we apply the stretches ourselves. Identity learning still flows through the corrected image, which lets the recognizer keep face proportions as a feature. The central empirical question is whether "estimate and correct" beats "become invariant" on clean faces, on stretched faces, and especially as the number of training identities shrinks.

Core Claim

Anisotropic stretch is confounded with intrinsic face shape. Methods that remove it by making the model insensitive to it will pay a measurable "clean tax" in verification accuracy on undistorted faces. That tax should grow as training data shrinks, and landmark-based affine normalization pays the same tax for the same reason. An auxiliary stretch-estimation task can avoid the tax, provided the estimate comes from shape-neutral cues and is applied as an explicit correction rather than learned into the shared identity features.

What would be new is threefold. The first is the measurement itself: nobody in the literature we have separates imposed stretch from intrinsic proportions in face recognition. The second is an estimator supervised directly by self-generated stretch labels. The third is evidence about where the multi-task conflict between "be sensitive to stretch" and "be insensitive to stretch" should live in the network.

Relation to Prior Work

The closest work is RDCFace. It corrects wide-angle lens distortion with a differentiable warp governed by one radial coefficient. The warp is trained only through the downstream recognition loss plus a re-correction loss that keeps already-clean images unchanged. RDCFace supplies our method template, a learned one-parameter warp placed before an ArcFace-trained recognizer. It also motivates the problem, since a standard recognizer fell to 18.42% under severe radial distortion while RDCFace held 82.68%. RDCFace differs from our proposal in three ways. It models isotropic radial distortion and assumes square pixels. It never supervises the distortion parameter directly. It does not ask whether its correction preserves identity-bearing proportions.

APA motivates the concern from the opposite side. It shows that forcing faces into a single alignment template creates "alignment noise" that inflates within-identity variation, and that geometry-preserving templates train about 1.5× faster. That is evidence that geometric normalization choices change what the recognizer can learn. APA still resizes to square crops and never considers anisotropic stretch.

PoseFace and FAN both separate a nuisance factor from identity with auxiliary objectives, pose in one case and resolution and blur in the other. We borrow that logic but differ in kind. Their nuisance factors are not confounded with identity-bearing geometry the way stretch is with face proportions. We also correct the nuisance factor in the input rather than splitting it off in feature space.

RotoGrad constrains our design. It attributes negative transfer between tasks to gradients that differ in size or point in opposing directions. It mitigates this by rescaling task gradients and rotating the shared features separately for each task. A stretch-regression head and an identity head on a shared backbone want opposite things, sensitivity versus insensitivity to stretch, so this is a predicted direct conflict. We test RotoGrad as the shared-backbone alternative to our architectural separation.

On the input side, NaViT processes images at native aspect ratio by packing variable-length patch sequences. We use that principle only for the estimator's context view, not for the recognizer, for a data-scarcity reason. Face Transformer reports that vision transformers trained on CASIA-WebFace fall well short of ResNet-100 and need MS-Celeb-1M to catch up. We therefore keep a convolutional recognizer and avoid confounding our question with transformer data hunger.

Research Program

The distortion model is a horizontal scale $s$ applied to the face frame, with $\log s$ drawn from a range covering common aspect mismatches, for example $s \in [0.75, 1.78]$, which spans 4:3 and 16:9 confusions in both directions.

The estimator is a light CNN. It sees two views: a high-resolution eye-region crop, where the iris outline is resolvable, and a wider context crop kept at native aspect ratio that includes head outline, neck and background. It outputs $\hat{s}$. A warp rescales the face by $1/\hat{s}$, and the corrected face is then aligned and passed to an IR-SE50 recognizer trained with ArcFace. The training objective is

$$L = L_{\text{arc}}\big(W_{1/\hat{s}}(I)\big) + \lambda \,\big|\log \hat{s} - \log s\big| + \mu \,\big|\log \hat{s}\big(W_{1/\hat{s}}(I)\big)\big|,$$

where $I$ is the stretched input, $s$ is the stretch we applied, $W_{1/\hat{s}}$ is the correcting warp, $L_{\text{arc}}$ is the ArcFace identity loss, and the last term is RDCFace's re-correction idea: a corrected image should be estimated as unstretched. $\lambda$ and $\mu$ weight the terms. The middle term is the auxiliary task. Its labels cost nothing, which is why it suits the scarce-data regime.

The analysis has two parts. First, we quantify how much identity information face proportions carry. We fit a linear probe from embeddings of each baseline to landmark-derived proportion ratios on unstretched faces. We also measure verification accuracy using those ratios alone. This tells us whether stretch-invariant training actually erases proportion information and whether that information is worth anything.

Second, we locate the conflict. We compare three placements: the stretch head on the shared recognizer backbone with fixed loss weights; the same with RotoGrad; and our separate estimator with input correction. For the shared-backbone variants we log the cosine similarity between task gradients to test the predicted opposition directly.

Baselines

The do-nothing reference is IR-SE50 with ArcFace trained on CASIA-WebFace with standard square five-point similarity alignment and no stretch handling. It shows how badly stretch hurts and sets the clean ceiling.

The strongest simple competitor is the same network trained with random anisotropic stretch augmentation over the same range of $s$. This is the invariance approach our claim argues against, and the one a practitioner would reach for first. To make the comparison fair, it gets identical data, epochs and augmentation range, and a tuning budget for the augmentation probability equal to our budget for $\lambda$ and $\mu$. Beating it means matching it on stretched test sets while recovering a statistically significant part of its clean-set loss. We judge significance with bootstrap confidence intervals over verification pairs, and we expect the recovered accuracy to grow as training identities shrink.

The strongest existing method is an RDCFace-style correction module with its single parameter redefined as anisotropic scale. It is trained only through the identity loss and the re-correction loss, with no direct stretch supervision. It isolates whether our explicit auxiliary task adds anything. RDCFace's own numbers are for radial distortion: 96.67% mean accuracy on RadialFace against 91.92% for a standard recognizer, and 82.68% against 18.42% on synthetic IMDB-Face at $k=-3.0$. For anisotropic stretch there is no reported result on Lacuna, so we will run it ourselves.

The landmark-affine baseline replaces the five-point similarity transform with a full affine fit, which normalizes stretch geometrically.

The shared-backbone multi-task baselines, with and without RotoGrad, test whether gradient balancing alone can host the conflict.

Our key ablations remove the direct stretch loss, remove the iris view, and remove the native-aspect context view. They tell us which shape-neutral cue carries the estimate.

Why not just…?

Why not just augment with random stretch? This is the main threat, and the experiment is built to settle it. If augmented training shows no clean-set penalty at any data scale, then face proportions are not usable identity cues for these networks, and our method is unnecessary. We would report that as the finding. The proportion probe explains the result either way.

Why not just use affine alignment from landmarks? A five-point affine fit cannot tell an imposed stretch from a naturally wide face, because both move the landmarks the same way. It should therefore erase intrinsic proportions just as augmentation does. It also depends on landmark detectors, which Joint Face Alignment and 3D Face Reconstruction notes fail under hard geometry. The landmark-affine baseline measures this directly.

Why not just pretrain on MS-Celeb-1M and fine-tune? With enough identities, the clean tax of invariance may become negligible. That would narrow our contribution to the scarce regime rather than refute it. We therefore repeat the key comparison with MS1Mv2 training to see whether the gap closes at scale. Face Transformer reports 99.80% on LFW at that scale, which suggests the clean ceiling there leaves little room.

Why not just put a stretch head on the shared backbone and balance it with RotoGrad? This may work. RotoGrad's per-task rotations exist precisely to reconcile opposing gradients. But it has never been tested with angular-margin face losses, and our gradient-cosine logging and head-to-head comparison decide the question.

Proposed Experiments

We train on CASIA-WebFace and on identity-subsampled versions at 100%, 25% and 10% of identities, to trace the data-scarcity curve. The MS1Mv2 run serves as the scale check.

No anisotropic-stretch face benchmark exists in the literature we found. We therefore build stretched versions of LFW, CPLFW and CFP-FP at fixed $s \in \{0.75, 1.0, 1.33, 1.78\}$, plus IJB-C at TAR@FAR=1e-4. The stretch is applied to full frames before detection so that detection and alignment see realistic inputs. The $s=1.0$ column is the clean-tax measurement. For reference, APA reports 91.97% on unstretched CPLFW, trained under its own setup.

We add YLFW children's faces as a scarce, shape-shifted evaluation domain. No reported result on Lacuna is available for our baselines there. We also run SCface as a real surveillance probe. FAN reports 90.3% rank-1 on SCface, though its stretch conditions are unknown.

Because the synthetic test sets are our weakest point, we will also collect a small real set: the same subjects recorded through capture paths with mismatched aspect settings, so that the true $s$ is known from the configuration.

The decisive checks are these. First, stretch augmentation should lose clean accuracy relative to the do-nothing baseline, and that loss should grow as identities are removed. Second, our model should recover most of that loss while matching augmentation on stretched sets. Third, removing the direct stretch loss should degrade the estimate, and with it the stretched-set accuracy, most at the 10% scale.

Two failure signals would each count against the idea. One is the absence of a clean tax, which would mean invariance is free. The other is an estimator that uses face proportions despite our design, visible when it produces correlated errors on naturally wide and narrow faces. We test for that by checking whether estimation error correlates with ground-truth proportion ratios on unstretched faces.