Text Layers as Blur Probes for Legible 3D Reconstruction from Blurry Captures

When a phone camera moves during an exposure, every pixel records an average of the scene along a short camera path. Methods for building 3D scenes from photos now undo this averaging well for natural textures such as foliage, walls and furniture. Printed text is different. Signs, labels and book spines stay smeared or come out as confident-looking garbage. We propose to treat scene text as the most informative part of a blurry image rather than the most fragile. Text is locally flat, mostly two-colored, and made of hard edges. A blurred hard edge on a flat surface shows almost directly how the camera moved during the exposure. So if text is modeled with its own simple, strongly constrained representation, it can do two jobs at once: it can be read, and it can help pin down the camera motion that deblurs everything else.

Concretely, we build a hybrid scene. Most of the scene uses 3D Gaussian splatting, which represents a scene as millions of small colored, semi-transparent blobs rendered by projecting them onto the image. Each detected text instance is instead a planar texture layer: a high-resolution 2D image placed on a learnable plane in 3D. Because a plane maps into each view by a simple warp (a homography), the text layer can be rendered at dozens of camera poses along the exposure path at almost no cost. The text layer carries a two-tone color prior and an explicit model of sensor saturation, because bright signs often clip the sensor. The claim is that this layer improves legibility of reconstructed text (measured by optical character recognition error on held-out views) beyond any combination of existing deblurring and text-refinement methods. We also claim it measurably improves the recovered camera trajectories. That second claim is what distinguishes the idea from simply stacking two existing methods.

What exists and where it breaks

Two lines of work each solve half of the problem. Forward blur modeling reproduces the blur inside the renderer. Deblur-NeRF averages a handful (about five) of learned sparse rays per pixel. BAD-NeRF interpolates the camera pose between exposure start and end and bundle-adjusts it. BAD-Gaussians moves this into Gaussian splatting with spline trajectories. Event- and spike-camera variants such as Ev-DeblurNeRF and SpikeGS add extra sensors. These papers supply our forward model and trajectory parameterization. All of them apply one photometric loss uniformly over the image, and text strokes are a tiny fraction of the pixels.

On the text side, STRinGS is the closest work. It segments text with a 2D model, splits the Gaussians into text and non-text, and freezes the positions of the text Gaussians during a first phase so that glyph structure does not drift. It introduces character error rate from optical character recognition (OCR-CER) as a 3D text metric. It assumes sharp images and good initialization from structure-from-motion, the standard pipeline that recovers camera poses and a sparse point cloud from matched image features. Its authors state it cannot recover text that is unreadable in the inputs.

This produces a direct tension that motivates the proposal. STRinGS says text geometry must be frozen, while BAD-Gaussians says geometry and poses must move jointly to escape blur. Freezing text Gaussians initialized from blur-corrupted structure-from-motion points locks in the smear.

Older 2D text deblurring supplies the key physical insight but not a 3D method. Text Image Deblurring Using Text-Specific Properties showed that generic natural-image statistics fail on text. It also showed that an idealized two-tone text prior guides blur-kernel estimation, raising synthetic PSNR from 15.66 to 28.22 dB. Robust Blind Text Image Deblurring via Maximum Consensus makes kernel estimation robust to outliers but assumes planar 2D images and uniform blur. Face Video Deblurring Using 3D Facial Priors is a structural analogy: a strong low-dimensional 3D model of one object class (faces) improved deblurring of everything around it. We apply the same logic with planar text as the object class.

We deliberately do not use generative text-to-image priors inside reconstruction. TextInVision reports that the autoencoders these generators use to compress images retain only 51% (Flux) and 39% (SD 1.5) of words on real text images, even before any generation happens. It also reports that spelling collapses as the scene description grows complex. The character-aware encoders of Character-Aware Models Improve Visual Text Rendering fix spelling but not stroke placement. This work constrains our design: a generative prior would let the system invent plausible words, which is the failure we most want to avoid.

Core Claim

A two-tone planar texture layer, rendered with densely sampled exposure poses and a saturating sensor model, removes the freeze-versus-adjust conflict. The layer's in-plane content can be kept sharp while its plane and the camera path move freely. Because blurred step edges on a known plane are highly informative about motion, the layer should also improve trajectory accuracy for the whole scene.

Research Program

The forward model for a blurry pixel $x$ in frame $j$ is

$$\hat I_j(x) = \mathrm{clip}\Big(g\big(\tfrac{1}{N}\textstyle\sum_{k=1}^{N} L(x;\,\mathbf T_j(t_k))\big)\Big),$$

where $L(x;\mathbf T)$ is linear-radiance rendering at camera pose $\mathbf T$, $\mathbf T_j(t)$ is the spline trajectory for frame $j$ as in BAD-Gaussians, $t_k$ are $N$ sample times in the exposure, $g$ is a learned tone curve, and $\mathrm{clip}$ saturates at the sensor maximum. Averaging in linear space and then clipping matters for high-contrast signs. A saturated white sign averaged in display space looks like a sharp plateau and wrongly implies no motion; the clipped linear model attributes the plateau to saturation instead.

Where a text layer covers a pixel, $L$ composites the layer's texture over the Gaussian render, using the layer's own alpha. The texture is warped into the view through the homography induced by the layer's plane. Warping is cheap, so text pixels use large $N$ (we will test 32 to 128), while Gaussians keep the small $N$ of BAD-Gaussians. This addresses the aliasing that sparse pose sampling causes on step edges.

The text texture $A$ is regularized toward two colors per instance:

$$\mathcal R(A) = \lambda_1 \min_{c\in\{c_f,c_b\}}\|A - c\|_1 + \lambda_0 \|\nabla A\|_0,$$

where $c_f$ and $c_b$ are learned foreground and background colors of the instance. The $L_0$ gradient term, approximated by half-quadratic splitting as in the 2D text-deblurring work, favors few, hard edges.

The plane's pose and the camera trajectories are optimized jointly. The texture's content is never "frozen"; it is simply confined to its plane, which achieves what STRinGS obtains by freezing positions without blocking bundle adjustment.

Text layers are initialized by bootstrapping. We first run a short BAD-Gaussians warm-up. We then detect text on its partially deblurred sharp renders, not on the blurry inputs where 2D segmentation fails. Next we fit planes to the Gaussians under each detected region and spawn layers. We re-detect periodically so that new instances can appear as renders sharpen. Curved text, such as the cylinders and globes in STRinGS-360, is handled by allowing each layer a low-order bending field. We report flat and curved cases separately, because curved text is where the planar assumption could fail.

Baselines

The naive reference is vanilla 3D Gaussian splatting trained directly on blurry frames with structure-from-motion poses. It shows how bad the problem is, and it is run with the same iteration budget as every other method.

The strongest general deblurring competitor is BAD-Gaussians, which reports outperforming the previous best methods by an average of 3.6 dB PSNR on the Deblur-NeRF synthetic set. BAD-NeRF reports 30.94 dB there. No reported OCR-CER result was found on Lacuna for BAD-Gaussians or any deblurring method. We will run it with its released spline order and virtual-pose count on identical frames and initialization.

The strongest text competitor is STRinGS, which reports a 63.6% relative OCR-CER improvement over 3DGS at 7K iterations on sharp STRinGS-360. That is the only text-legibility figure in the file.

The most dangerous baseline is a two-stage combination that no paper reports. First, BAD-Gaussians recovers poses and sharp training-view renders. Second, STRinGS runs on those renders and poses with its frozen-position schedule. We give this pipeline the summed iteration budget of our method plus a matched tuning sweep over STRinGS learning rates.

We also include a text-weighted BAD-Gaussians, in which the loss is upweighted inside detected text regions and the pose count is raised. This isolates whether a separate representation is needed at all.

Our ablations remove, one at a time, the dense text pose sampling, the saturation model, the two-tone prior, and bootstrapped detection (replaced by detection on the blurry inputs). A further ablation replaces the planar layer with Gaussians constrained to a plane, to separate the effect of the representation from the effect of the prior.

Beating the field means two things. OCR-CER on held-out views must be lower than the two-stage pipeline's, beyond the variation across three seeds. Full-image PSNR must be no worse than BAD-Gaussians'.

Why not just…?

Why not just deblur each photo in 2D and then run STRinGS? Per-frame deblurring hallucinates independently in each view, which breaks multi-view consistency. The file credits Deblur-NeRF with evidence that forward modeling beats reconstructing from pre-deblurred frames. The 2D text deblurrers also assume uniform, planar blur, and real handheld blur is neither. We include this pipeline, using the maximum-consensus deblurrer, as a baseline rather than dismissing it.

Why not just run BAD-Gaussians, then STRinGS on its outputs? This is the real threat, and it may close most of the gap when blur is mild. Our argument is that errors in the first stage's poses and textures cannot be corrected once text positions are frozen, and that the text never feeds back into pose estimation. The experiment that settles it is a comparison binned by blur length and by saturation fraction. If the two-stage pipeline matches us in every bin, the joint layer is not worth its complexity, and we will say so. We expect the advantage to grow with blur length and saturation. We also expect our trajectory error to be lower, because the two-stage pipeline's trajectories are by construction identical to BAD-Gaussians'.

Why not just upweight text pixels in BAD-Gaussians' loss? Reweighting changes how much the optimizer cares about text. It does not add dense exposure sampling at edges, a saturation model, or two-tone structure. The text-weighted baseline and the plane-constrained-Gaussians ablation measure exactly this. If reweighting alone recovers most of the CER gain, the contribution shrinks to that finding.

Why not restore the rendered text with a character-aware diffusion model? The autoencoder bottleneck and prompt-distraction results in TextInVision make stroke-accurate restoration unlikely. Independent per-view generation is also not consistent across views. We run it on our test renders to report its CER and cross-view consistency, measured as disagreement between OCR outputs across views of the same sign.

Proposed Experiments

No existing benchmark combines text and blur, so we build two.

The first is synthetic. We fit sharp reconstructions to the STRinGS-360 scenes and to text-rich DL3DV-10K scenes. We then render blurry training frames by averaging many poses along randomly perturbed spline trajectories in linear space, with controlled exposure gain so that a known fraction of text pixels saturates. This follows the synthetic protocol behind the Deblur-NeRF dataset. It gives exact ground-truth trajectories and sharp held-out views.

The second is real. We capture handheld long-exposure sequences of signage, shelves and screens. Each scene gets a tripod short-exposure reference set, which provides sharp evaluation views and reference poses.

Metrics are OCR-CER and word accuracy on held-out sharp views, using a fixed OCR model applied identically to all methods. We also report PSNR, SSIM and LPIPS on the full image and inside text masks, and absolute trajectory error. To verify that nothing regresses on general scenes, we run on the standard Deblur-NeRF synthetic and real sets, where BAD-Gaussians' reported margin is the reference.

Two checks are decisive. The first tests the "blur probe" claim. On scenes where text covers under 5% of the image, trajectory error with text layers must be lower than BAD-Gaussians'. If text improves legibility but not poses, the idea reduces to a better text renderer. The second is a hallucination audit. For text that is illegible in every input at our blur levels, we count outputs that OCR reads as a word different from the ground truth. The two-tone prior makes confident wrong glyphs possible, and a method that trades smears for invented words has failed its purpose even if its average CER drops.