Recovering Hidden Aspect-Ratio Distortion from Ground Texture

Many images reach a vision model already stretched. Thumbnails, web re-encodes, anamorphic video and preprocessing scripts that squash a 16:9 frame into a square all apply an unknown horizontal-versus-vertical scaling. To a model that infers 3D geometry from a single image, this stretch looks like a tilted surface. A circular pebble seen on a slanted ground plane becomes an ellipse, and so does a circular pebble on a flat plane after the image is squashed by 20%. Shape-from-texture is the classical study of how perspective compresses texture patterns and how that compression reveals surface orientation. Its methods assume square pixels, so they read any squash as false slant. Single-image camera calibrators, which predict focal length, pitch and roll from one photo, make the same assumption.

This note proposes to treat the stretch as what it physically is: an unknown pixel aspect ratio, a camera intrinsic, which can be estimated together with the other calibration parameters. The estimate comes from two sources. One is the anisotropy of texture on the ground. The other is the up-direction field that modern calibrators already predict. The two can be separated because the squash applies the same scaling at every pixel, while perspective foreshortening varies across the image in a pattern fixed by the camera's pitch and focal length. Ground texture is especially useful because the ground's surface normal is determined by gravity. Its foreshortening adds measurements to the calibration solver without adding any unknowns, and it does so in grassy, sandy or pebbled scenes where line-based methods fail.

Core Claim

Let $S=\mathrm{diag}(s,1)$ be the unknown horizontal stretch, with $s$ the pixel aspect ratio. At an image location $x$ on a textured surface, the observed local texture deformation is a 2×2 matrix

$$A_{\text{obs}}(x) = S\,A_{\text{persp}}(x;\,f,\phi,\psi,\mathbf{n}),$$

where $A_{\text{persp}}$ is the local affine foreshortening produced by a pinhole camera with focal length $f$, pitch $\phi$ and roll $\psi$, viewing a surface with normal $\mathbf{n}$. For the ground plane, $\mathbf{n}$ is the gravity direction and is therefore determined by $\phi$ and $\psi$. The minor-to-major axis ratio of a foreshortened isotropic texel at $x$ equals the cosine of the angle between the viewing ray and $\mathbf{n}$, as in the standard slant relation used by Curvature Estimation Using Shape-from-Texture. This makes the ground-texture anisotropy field a function of $(f,\phi,\psi,s)$ alone.

Under orthographic projection, $S$ and a uniform slant cannot be told apart. Under perspective they can, because $A_{\text{persp}}$ rotates and changes magnitude across the image toward the horizon while $S$ stays constant. This gives a testable prediction. The aspect-ratio estimate should become worse as the field of view narrows, and it should fail completely for a single fronto-parallel-ish plane seen through a telephoto lens, which is the limit where projection becomes orthographic. Any method that claims to recover $s$ without showing this degradation is probably exploiting dataset priors rather than geometry.

The new element is the joint estimation itself. No calibrator in the file estimates pixel aspect ratio. No shape-from-texture method models a post-projection stretch. The constant-versus-varying decomposition above is the mechanism that links the two.

Relation to Prior Work

The texture side supplies the measurement. Local spatial frequency analysis of image texture tracks how local Fourier peaks shift outward as a surface recedes, reaching about 1° tilt error on Brodatz textures. Curvature Estimation Using Shape-from-Texture shows that the affine matrix relating neighbouring spectral peaks has eigenvectors along tilt and eigenvalues giving slant in closed form. A Statistical Technique for Recovering Surface Orientation from Texture in Natural Imagery handles irregular textures by finding the inverse projection that makes contour orientations isotropic. We reuse these local anisotropy estimates, either as a learned predictor of the local spectral second-moment tensor or as a classical estimator for comparison. All three papers assume square pixels and estimate per-patch surface orientation. We instead fix the ground normal from gravity and add a global stretch parameter.

Toward a Universal Model for Shape From Texture learns a dense affine unwarping field adversarially and reports 12–20° mean angular error on synthetic scenes. It is the closest learned shape-from-texture system. Because a free per-patch affine field can absorb any global stretch, it confirms the confound rather than resolving it. Scene shape from texture of objects reads foreshortening from the bounding boxes of repeated objects such as windows. It motivates the idea that foreshortening is a scene-level cue, but it needs detectable objects. Our cue works on unstructured micro-texture.

The calibration side supplies the solver. GeoCalib predicts a dense field of per-pixel up-vectors and latitudes, then fits focal length, pitch, roll and lens distortion with a differentiable Levenberg-Marquardt optimizer. It reaches 0.28° median roll error on LaMAR and notes that vanishing-point methods fail on more than 50% of MegaDepth images. Thinking with Camera (Puffin) reasons in language about camera cues and reports 1.08° median pitch error on MegaDepth. Deep Single Image Camera Calibration With Radial Distortion regresses tilt, roll, field of view and radial distortion from fixed square crops. All three assume $s=1$. We extend the GeoCalib solver with $s$ and with ground-texture residuals.

The aspect-ratio literature motivates the setting and constrains the backbone design. NaViT packs patches from images of native shape into one sequence. NARAIM shows that native-ratio pretraining beats square resizing (56.8% vs 54.7% ImageNet-1k validation accuracy). LLaVA-UHD slices arbitrary-ratio images for language models. All of these evaluate semantics only. For this proposal they carry one design lesson: the estimator must see the image at its delivered shape. Squashing it again for the backbone would inject a second, known $S$ on top of the unknown one.

Research Program

The system has three parts. First, a native-ratio backbone, a vision transformer fed NaViT-style variable token grids. It predicts GeoCalib's up-vector and latitude fields plus two new dense outputs: a local texture-anisotropy tensor (the 2×2 second-moment matrix of local dominant frequency, normalized to remove scale) and a confidence map marking pixels that belong to roughly isotropic ground texture.

Second, an extended Levenberg-Marquardt solver over $(f,\phi,\psi,k_1,s)$. Its residuals are GeoCalib's perspective-field residuals, recomputed through $S$, plus confidence-weighted ground-texture residuals. The texture residual compares the predicted anisotropy tensor with the one implied by the camera: a foreshortening of $\cos$(ray-to-gravity angle) along the image direction of the ground's depth gradient, then mapped through $S$. Because the comparison uses axis ratios and orientation, unknown distance cancels.

Third, a training corpus. Following Puffin and GeoCalib, it is built from panoramas re-photographed with virtual cameras. We add a sampled stretch, with $\log s$ uniform in $[\log 0.5, \log 2]$, and random field of view. Blender scenes of textured ground planes, built in the style of the universal shape-from-texture benchmark, supply ground truth for the texture head.

The analysis has two parts. We first measure the problem with no new method. Off-the-shelf GeoCalib is run on MegaDepth and LaMAR images stretched by known $s$, and we record how roll, pitch and focal-length errors grow with $|\log s|$. A monocular normal estimator is run the same way on NYU Depth v2, recording normal angular error against $s$. This tests directly whether existing systems read stretch as geometry, and the result is useful even if the solver adds nothing. We then evaluate the joint estimator and its identifiability as a function of field of view and ground coverage.

Baselines

The naive reference is to assume $s=1$, as every current calibrator does. We report its error on stretched images and, trivially, its aspect-ratio error of $|\log s|$. Any method must clearly beat this at moderate stretches such as $s=0.8$. If it only works at extreme stretch, it has no practical value.

The strongest existing calibrator is GeoCalib. Its reported LaMAR median roll error is 0.28° (GeoCalib). For pitch on MegaDepth, Puffin reports a median of 1.08° (Thinking with Camera). We use the released GeoCalib as given and also fine-tune it on our stretched training corpus with the same compute and tuning budget as our model, so it is not penalized for never having seen stretched data. Beating it means two things. On unstretched images we must match its roll and pitch error within noise. On stretched images we must give clearly lower pitch, roll and focal-length error.

The strongest competitor on the new quantity is GeoCalib with $s$ added to its optimizer but no texture residuals. Because $S$ also bends the up-vector field away from the image centre, this baseline may recover $s$ on its own. It doubles as the ablation that isolates the key ingredient. It uses identical backbone, data and solver, and differs only in whether the texture residuals are switched on. We report the aspect-ratio error, the median of $|\log \hat s - \log s|$, overall and on a line-poor subset: images where a classical vanishing-point detector returns too few lines, which GeoCalib itself shows is over half of MegaDepth. No reported result exists on Lacuna for pixel-aspect-ratio estimation on any of these benchmarks, so these numbers will be new. The texture ingredient counts as useful only if it reduces aspect-ratio error substantially on the line-poor subset without hurting the line-rich one.

We also include a black-box baseline: the same backbone with a direct regression head for $s$ and no solver. For texture-only accuracy, we include the classical spectral and isotropy estimators of Local spatial frequency analysis of image texture and Witkin, each extended with a global $s$ fitted over ground pixels. This shows whether the learned anisotropy head matters.

Why not just…?

Why not just preserve the native aspect ratio, as NaViT does? If you control the pipeline, you should, and we do. NARAIM and NaViT show that this avoids self-inflicted distortion. It cannot undo a stretch applied before the image reached you, which is the case this proposal targets. The measurement study will show whether such upstream stretch harms geometry enough to matter. If GeoCalib's errors barely move at realistic stretches, the method is unneeded, and we would report that.

Why not just add $s$ to GeoCalib's solver and stop? This is the most serious alternative. The up-vector field does carry information about $s$, so this baseline may work well in man-made scenes. Our expectation is that its signal weakens where verticals are scarce or close to the image centre, which is where the gravity-fixed ground texture keeps contributing. The ablation on the line-poor subset settles this. If texture adds nothing there, the contribution reduces to the measurement study plus this one-parameter extension, and we will say so.

Why not just regress $s$ with a network trained on stretched images? A regressor can exploit dataset priors, such as typical object shapes, instead of geometry. The GeoCalib paper's comparison against unconstrained deep models (0.28° vs 0.93° median roll on LaMAR) suggests that geometry-constrained solvers generalize better. The field-of-view sweep is the discriminating test. A geometric estimator must degrade toward chance as the projection approaches orthography, while a prior-driven regressor will not. A regressor that stays accurate where geometry says $s$ is unidentifiable is reading semantics, and it should fail on out-of-distribution scenes, which we test by training on MegaDepth and evaluating on OpenPano and LaMAR.

Why not just use random-resized-crop augmentation to make models invariant to stretch? Invariance is the wrong target for geometry. A model invariant to $S$ must also be blind to genuine uniform foreshortening, which is the slant signal itself. The NYU normal-error-versus-$s$ curves will show whether augmented models avoid the bias by giving up slant accuracy on unstretched images.

Decisive Checks

Three outcomes determine whether the idea holds. First, errors from the current calibrators and the normal estimator must grow measurably at stretches of 10–25%. Without this there is no problem to solve. Second, the joint estimator's aspect-ratio error must follow the predicted geometric trend: it should improve with wider field of view and with more ground coverage, and it should become unidentifiable in the near-orthographic regime on synthetic telephoto ground planes. Third, turning on the texture residuals must reduce aspect-ratio and pitch error on the line-poor MegaDepth subset relative to GeoCalib with $s$ added, while matching GeoCalib's reported unstretched accuracy within noise. If the second check fails, the estimator is not doing geometry. If the third fails, the texture cue is redundant with the up-vector field.