Recovering Unknown Aspect-Ratio Stretch from an Image's Own Rotated Repeats

Many images reach us geometrically wrong. Thumbnails and legacy pipelines squash pictures into fixed boxes, anamorphic video gets stored without its display flag, and scanners mismatch horizontal and vertical resolution. Once the metadata is lost, nothing in the file records that circles have become ellipses. Restoration models make this worse. A super-resolver trained on correctly proportioned photos, given a stretched input, will sharpen the stretch faithfully, or will hallucinate textures shaped for the wrong geometry. This proposal treats an unknown aspect-ratio stretch as a blind degradation, like an unknown blur kernel, that must be estimated from the pixels before restoration can be faithful.

The central idea is that an image's own repeated structure reveals its stretch. Natural and man-made scenes contain the same structure at different orientations: round objects, edges at many angles, a texture element repeated after rotation. In a correctly proportioned image, a patch and its rotated copy match. After a horizontal stretch, they only match once the stretch is undone. The stretch factor that best restores agreement among rotated self-repeats is therefore an internal estimate, meaning one computed from the test image alone. It is complementary to an external estimate from a network trained on many correctly proportioned images, which knows what faces and wheels should look like. We fuse the two estimates according to their confidence. We then restore the image in corrected coordinates using an external generative restorer and a self-similarity anchor computed on the unstretched geometry.

Core Claim

Rotated self-recurrence is a usable, content-agnostic signal for blind aspect-ratio recovery. It succeeds exactly where external shape priors are weakest: textures, facades, manga, and documents, which have no canonical object proportions. External priors succeed where internal recurrence is sparse, such as single faces and isolated objects. A confidence-weighted fusion should beat either source alone across content types. Correcting geometry before restoration should then give measurable fidelity gains over restoring in the distorted frame.

This claim contrasts directly with the two closest works. Single image super-resolution from transformed self-exemplars searches for self-exemplars under perspective, rotation, scale, and affine shear, using vanishing-point plane detection. It searches over local transforms in order to find more matches, and it assumes the image's global geometry is correct. We instead solve for one global transform that makes the image's existing matches consistent, and we pair it with an external prior. SSL: A Self-similarity Loss for Improving Generative Image Super-resolution anchors generative super-resolution to a graph of which patches resemble which other patches, computed on a fixed square pixel grid. On a stretched image, that graph encodes the stretch, so the anchor faithfully preserves the wrong geometry. We reuse the self-similarity graph as a method component, but compute it after geometric correction.

Research Program

We parameterize the distortion as an area-preserving stretch $S_r = \mathrm{diag}(r^{1/2}, r^{-1/2})$, where $r$ is the unknown ratio of horizontal to vertical scaling. The internal estimator samples edge and texture patches using a Laplacian mask, as SSL does. For each candidate $r$ and each rotation angle $\theta$ in a small set, it scores how well patches find near-duplicates elsewhere in the image under the warp $S_r R_\theta S_r^{-1}$, where $R_\theta$ is rotation by $\theta$:

$$ \hat r_{\text{int}} = \arg\max_r \sum_{\theta \neq 0} \sum_{p} \max_{q} \exp\!\left(-\| P_q - \mathcal{W}_{S_r R_\theta S_r^{-1}}(P_p)\|^2/h\right). $$

Here $P_p$ is the patch at location $p$, $\mathcal{W}_A$ resamples a patch under the linear map $A$, and $h$ is a bandwidth. The $\theta = 0$ term is excluded deliberately. Translation-only recurrence, and recurrence across isotropic scales, both survive a global stretch unchanged, because a global stretch commutes with translation and with uniform scaling. These are the signals that SSL and zero-shot super-resolution rely on, and they are blind to this degradation. We take an uncertainty $\sigma_{\text{int}}$ from the curvature of the score around its peak and from the number of distinct rotated matches. An image of pure axis-aligned stripes has no rotated matches, so it yields a flat score and a large $\sigma_{\text{int}}$.

The external estimator is a native-aspect-ratio vision transformer. Following Patch n' Pack: NaViT and NARAIM, it uses packed variable-shape inputs with factorized or fractional position embeddings, so that it never sees a squashed training image. It is trained on DIV2K and Places2 with synthetic stretches to predict $\log r$ together with a variance $\sigma^2_{\text{ext}}$. The two estimates are combined by inverse-variance weighting,

$$ \log \hat r = \frac{\sigma_{\text{ext}}^{2}\log \hat r_{\text{int}} + \sigma_{\text{int}}^{2}\log \hat r_{\text{ext}}}{\sigma_{\text{int}}^{2}+\sigma_{\text{ext}}^{2}}, $$

which is the standard precision-weighted combination of two independent Gaussian estimates. We then resample the input by $S_{\hat r}^{-1}$ and restore it with an external generative super-resolver. We fine-tune that restorer with an SSL-style self-similarity loss whose graph is computed in the corrected frame, and we use native-aspect tiling rather than square resizing.

We also run a stress study of how stretch interacts with the degradations that Meta-Transfer Learning for Zero-Shot Super-Resolution and From general to specific: Online updating for blind super-resolution adapt to. Anisotropic Gaussian blur changes patch appearance anisotropically without moving geometry. A key question is whether the rotated-recurrence score confuses the two. We will test whether computing the score on a coarser scale level separates them.

Baselines

The naive reference point is to do nothing about geometry. We feed the stretched, degraded image to the same restorer and score against the original. This bounds how much geometric correction can matter. A second trivial reference is an oracle that knows the true $r$. It gives the ceiling for any estimator.

The cheapest internal competitor is spectral isotropy. It estimates $r$ as the stretch that makes the image's gradient structure tensor, or its power spectrum, most isotropic. It costs almost nothing and needs no training, so it is the right test of whether patch recurrence adds anything. We expect it to be biased on urban and document images, which have genuine horizontal and vertical dominance. Beating it means lower median $|\log \hat r - \log r|$ on Urban100 and Manga109, not just on DIV2K.

The strongest external competitor is our native-aspect stretch regressor alone. It uses the same architecture, the same training data, and the same compute as the external branch, but has no internal branch. It isolates the fusion's contribution. The proposal beats it if fusion reduces estimation error on texture-heavy and repetitive content without losing on CelebA-HQ faces.

For restoration, the strongest existing blind methods are MZSR and ONSR. Both are retrained with stretch added to their degradation distributions, so they get the same synthetic data we use. MZSR reports 29.19 dB PSNR on Urban100 with isotropic Gaussian blur after 10 updates, against 24.14 dB for supervised RCAN, in Meta-Transfer Learning for Zero-Shot Super-Resolution. ONSR reports gains of up to 2.15 dB over ESRGAN and RRDB on DIV2KRK in From general to specific. No reported result exists on Lacuna for any method on stretched inputs, so all stretch-condition numbers will be produced here under matched test-time budgets: the same number of gradient steps, or equal wall-clock time.

The key ablation removes the $\theta \neq 0$ restriction, using translation-only recurrence. This should collapse the internal estimator, which tests the mechanism directly. A second ablation keeps the corrected geometry but computes the SSL graph in the distorted frame.

Why not just…?

Why not just train a network to predict the stretch from external data? This is a real threat. On faces and common objects it may simply win, because external shape knowledge is strong there. Our bet is narrower: on textures (DTD), facades (Urban100), and drawn content (Manga109), there is no canonical shape to learn, and the internal signal transfers. The matched external-only baseline will settle this. If it matches the fused system on all content types, the internal branch is not justified and we will report that.

Why not just train the restorer with random stretch augmentation and let it cope? Augmentation teaches a network to restore in whatever geometry it receives. Teaching it to output corrected geometry turns it into an end-to-end external estimator with no explicit uncertainty. We include this end-to-end variant as a baseline. The evidence from NARAIM is suggestive: square random-resized-crop training hurt representations relative to native-aspect training, falling from 56.8% to 54.7% ImageNet accuracy for the AIM baseline. That result concerns classification, however, and our experiment will test whether the effect carries over to restoration.

Why not just measure spectral isotropy? Isotropy assumes that orientation statistics are balanced. Buildings, text lines, and horizons violate that assumption, and they are exactly the content where stretch is common. It remains a baseline because if it suffices, the patch search is wasted compute.

Why not just use the transformed self-exemplar search of Huang et al.? That method needs vanishing points. It reports no clear advantage on BSD100 scenes lacking planes, according to Single image super-resolution from transformed self-exemplars. It also absorbs a global stretch into per-patch affine matches rather than estimating it. We run it as a restoration baseline on stretched Urban100.

Proposed Experiments

We synthesize stretches with $r \in [1/3, 3]$ on DIV2K validation, Urban100, BSD100, Manga109, DTD, and CelebA-HQ. We compose them with bicubic downsampling and with isotropic and anisotropic Gaussian blur, matching the MZSR and DIV2KRK degradation families. Estimation is scored by median and 90th-percentile $|\log \hat r - \log r|$, broken down by dataset and by stretch magnitude. Restoration is scored by PSNR, SSIM, LPIPS, and DISTS against the unstretched high-resolution original, which penalizes any residual geometric error. As an external check on whether the corrected geometry matters perceptually, we run the aspect-ratio-preserving aesthetic predictor of Aspect-Ratio-Preserving Multi-Patch Image Aesthetics Score Prediction on AVA images stretched and then corrected. That work reports an 18% MSE reduction (0.061) and up to 7.9× larger gains on extreme ratios. Our question is whether our correction recovers the score distribution of the original images.

The decisive checks come in three parts. First, the rotated-recurrence estimator must beat spectral isotropy on Urban100 and Manga109. Second, fusion must beat the external-only regressor on DTD, Urban100, and Manga109 while staying within noise on CelebA-HQ. Third, under combined stretch and anisotropic blur, the estimator must not mistake blur for stretch. We test this by holding $r = 1$ and sweeping blur anisotropy, then checking that $\hat r$ stays near 1. If the first check fails, the internal signal is not worth its cost. If the second fails, the internal-external fusion claim is wrong for this degradation.