Reading Through Glare: Staged Inverse Rendering of Photographed Glossy Pages

When someone photographs a printed page with a phone, the result often contains a bright patch of glare from a ceiling lamp or window. On coated journal stock, or near the curved spine of a bound proceedings volume, that patch can wipe out whole lines of text. A second person then has to re-shoot the page, or an optical character recognition (OCR) system silently misreads what is left. We propose to treat a short handheld clip of a page as a small 3D reconstruction problem. From a few frames, we recover the page's curved shape, its true ink-on-paper pattern, and the light reflecting off it. We then read the text from the recovered pattern rather than from any single photo. The key observation is that pages are an unusually friendly case for this reconstruction. Their color pattern is nearly two-toned (ink or paper), and their shape is a gently bent sheet whose text lines were straight before bending. Those two facts rule out the usual failure mode, in which the reconstruction explains glare by painting a white smudge into the page's color. That failure mode is otherwise very hard to prevent.

We test this on pages where the stakes of a misread are highest and language models help least. These are the administrative pages of scholarly publications: journal mastheads, editorial-board listings, program-committee rosters, and reviewer acknowledgements. They are dense with personal names and affiliations and contain almost no predictable prose. An OCR error on a masthead cannot be fixed by context, and such pages are exactly what bibliographic crawlers mishandle when they misattribute editors or authors. The contribution is a capture-to-text pipeline, plus a benchmark of photographed non-research pages with registered ground truth. Together they show whether physically separating glare from ink beats simple multi-frame fusion and learned single-image de-glaring.

Core Claim

Separating a surface's shape, its material, and the incoming light from photos is ambiguous in general. A bright pixel can be a bump, a white patch, or a reflection. Staged optimization handles this by fitting coarse geometry under a matte assumption first and adding shiny reflection later. We claim that for documents, two domain constraints make the staged problem well posed from as few as three to five frames.

The first constraint is a two-tone color prior. The second is a straight-text-line shape prior. Neither suffices alone. The color prior blocks glare from leaking into the page's color. The shape prior blocks glare from leaking into fake surface bumps. We further claim that tracing reflected rays against the page itself, and not only against a distant environment map, is necessary near book spines. There the page reflects its own facing half, and methods that assume distant light produce structured errors.

What is new relative to the closest work is the combination of three things. We run a staged, hybrid inverse renderer on thin deformable sheets. We trigger the switch between stages with a document-derived signal instead of an iteration count. And we evaluate by text fidelity on name-dense pages, not by image quality on objects.

Relation to Prior Work

Staged optimization supplies the method. ReflFlow fits matte geometry on 2D Gaussian disks, which are flat splats with well-defined normals. It then freezes positions to fit reflections, and finally tunes everything jointly. Its phase boundaries are fixed schedules, and its reflections come from a single bounce off environment Gaussians. SSD-GS introduces diffuse, shadow, scattering, and specular terms in strict order to prevent "role leakage." That is the same problem we attack, but it relies on controlled one-light-at-a-time capture. 3iGS and DeLiRa likewise start from a matte or Lambertian stage and relax it later. These works motivate our design and also show its weak point: the research file notes that phase transitions are set heuristically, and that freezing geometry too late bakes highlights into depth. We replace the heuristic with a measurable document signal. We also replace generic priors with the two-tone and straight-line constraints that only documents offer.

The hybrid-rendering literature supplies the renderer and constrains its design. The idea of computing matte light with a cheap pass and reserving ray tracing for shiny transport goes back to Wallace et al.'s two-pass method. DIB-R++ and Zhu et al. use it for differentiable inverse rendering on rigid meshes. Reflective Gaussian Splatting handles inter-reflection by periodically extracting a proxy mesh and evaluates visibility only along the ideal mirror direction. Paper, however, has a moderately wide glossy lobe. RTR-GS uses reflection maps plus a single learned indirect-light term, which the file notes breaks down under near-field inter-reflection. A book gutter is exactly that case. Our page is a single developable sheet, so we can ray-trace against it analytically without periodic mesh extraction. We sample a few rays across the glossy lobe rather than one mirror ray.

Document work motivates the application. DocScanner flattens curved pages with a progressive recurrent unwarping network and a loss that keeps text lines straight. Its authors explicitly leave shadows and highlights to future work, and we reuse its straightness idea as a geometry prior. Sim2Real Docs uses Blender's ray tracer to synthesize photos of folded, glossy documents for training 2D models. It is a forward data generator, not a reconstruction loop, and it supplies our strongest learned baseline. The non-research artifacts catalogued in the file define our evaluation pages. Examples include IEEE Transactions on Cybernetics publication information, Proceedings of the IEEE publication information, DSML 2023 Committee, and Reviewer Acknowledgements. The file characterizes these as dense in proper nouns and lacking technical prose. Text classifiers such as WANDER operate only on digital strings, so they do not bear on the imaging problem. We make no claim about classifying these documents.

Research Program

The page is represented as a developable surface, meaning a sheet that bends without stretching, parameterized by a low-dimensional bending field over the flat page rectangle. A thin layer of 2D Gaussian disks sits on it and carries color and roughness. Each frame's camera pose is estimated jointly with the surface.

Rendering is hybrid. A rasterization pass resolves which page point each pixel sees and computes the matte term: color times low-frequency irradiance, stored as spherical harmonics, which are smooth functions over directions. A ray-tracing pass then launches a handful of reflected rays per pixel, sampled across a microfacet glossy lobe. Each reflected ray is first tested against the page sheet itself, which captures the facing half of a bound volume. If it misses the sheet, it looks up a learned environment map.

Optimization runs in two stages plus joint refinement. In stage one, the page is assumed matte and only shape, poses, color, and irradiance are fit. Pixels whose brightness is inconsistent across views are down-weighted, since they are glare candidates. A straightness loss asks that recovered text-line centerlines, detected on the unwarped color, be straight in page coordinates. Stage one ends when the straightness residual stops decreasing, not after a fixed number of steps. In stage two, shape is frozen, the glossy lobe and environment are switched on, and color, roughness, and light are fit. A final short joint pass unfreezes everything at a low learning rate.

Across all stages, color is regularized toward two tones. Writing $a(u)$ for the recovered color at page coordinate $u$, and $c_{\text{ink}}$ and $c_{\text{paper}}$ for two learned reference colors, the penalty is

$$ \mathcal{R}_{\text{tone}} = \sum_u \min\big(\|a(u) - c_{\text{ink}}\|^2,\ \|a(u) - c_{\text{paper}}\|^2\big), $$

which is cheap to compute and lets figures and color logos survive if the penalty is down-weighted in detected image regions. This penalty is what forbids the easy answer of explaining glare as brighter-than-paper color.

The recovered, flattened color map is passed to a fixed off-the-shelf OCR engine. The OCR engine is identical for every method, so all differences come from the image.

Benchmark

No existing benchmark in the file pairs photographed glossy pages with text ground truth, so we build one. We print and bind non-research pages whose digital source text is available. These include journal publication-information pages, society information pages, committee rosters, and proceedings front matter such as the PVLDB front matter and ACL 2024 frontmatter. As a contrast set, we include ordinary research pages, where language priors can repair OCR errors. Pages are printed on matte and coated stock, and some are captured flat while others are captured open in a bound volume.

Each page is recorded as a short phone clip under two or three lamp and window configurations. The ground truth is a flatbed scan registered to the page rectangle, together with the source text.

For synthetic control, we also render the same pages with Blender Cycles following the Sim2Real Docs recipe. There, true shape, color, and roughness are known exactly.

Baselines

The naive reference point takes the single sharpest frame, flattens it with DocScanner, and runs OCR. This is what a user gets today with no extra effort. DocScanner uses the released model with no retraining. Any method worth pursuing must lower the character error rate relative to this point, and the gain must be concentrated in glare regions.

The multi-frame fusion baseline aligns all frames to the page and takes a per-pixel robust minimum or median, with alignment done by DocScanner flattening followed by a homography. Glare moves between views while ink does not, so this is the classical fix and the most direct competitor on flat pages. It receives exactly the same frames as our method, and its alignment is tuned on the same held-out pages.

The learned single-image baseline is a de-glaring and unwarping network trained on Sim2Real-Docs-style ray-traced synthetic pages. Its training budget in GPU-hours matches the per-page optimization budget of our method summed over the test set, plus the time spent tuning our hyperparameters. It is the strongest competitor that uses no test-time optimization.

The staged-inverse-rendering baseline is a ReflFlow-style generic pipeline. It uses 2D Gaussians, a fixed three-phase schedule, and single-bounce reflection against an environment only, with no document priors. We run it at the same iteration count and tune its phase boundaries on the same held-out pages.

No reported result was found on Lacuna for any of these methods on photographed glossy or bound pages, measured by character error rate or by albedo error. The file contains no numbers for DocScanner, Sim2Real Docs, ReflFlow, or Reflective Gaussian Splatting on such a task, so all baseline numbers will be produced here.

Beating the baselines means two things. First, a statistically significant reduction in proper-noun character error rate on bound, coated pages relative to multi-frame fusion. Second, no loss relative to fusion on flat matte pages. Ablations remove one ingredient at a time: the two-tone prior, the straightness prior, the signal-triggered stage switch (replaced by the best fixed switch point), self-ray-tracing against the page (replaced by environment-only reflection), and staging altogether (replaced by joint optimization from the start).

Why not just…?

Why not just take several photos and keep the darkest value at each pixel? This is a real threat, and on flat pages shot from well-spread viewpoints we expect it to come close to our method. It fails in three conditions. The first is when viewpoints are few or close together, so glare overlaps across frames. The second is curved pages, where homography alignment smears strokes that are only a few pixels wide. The third is the book gutter, where the page's self-reflection is stable across small camera motions and therefore survives a minimum. The benchmark deliberately splits results by flat versus bound pages and by number and spread of views. If fusion matches us on bound pages too, the inverse-rendering machinery is not justified, and we would report that.

Why not just train a 2D network on ray-traced synthetic pages? Sim2Real Docs shows such data can train useful document models, so this baseline is serious. When glare saturates the sensor, the ink under it is simply absent from a single image. Any single-image network must then hallucinate it, and for personal names a hallucination is worse than a blank. We measure this directly by counting OCR outputs that are confidently wrong, not merely missing, inside saturated regions. A network given multiple frames as input is a fair extension, and we will include it if time allows.

Why not just flatten with DocScanner and let OCR's language model clean up? On research prose that may suffice, which is why the contrast set exists. On mastheads and committee rosters there is little linguistic redundancy to exploit. The gap between research and non-research pages under this baseline is itself a result: it quantifies how much of current OCR robustness comes from language priors that non-research pages do not provide.

Why not use a generic reflective-scene method such as ReflFlow or Reflective Gaussian Splatting unchanged? These are designed for objects and scenes, and their failure mode on pages is predictable: glare becomes bright color or false curvature. The prior-removal ablations test exactly this. If the generic method with a tuned schedule matches ours, the document priors add nothing and the claim fails.

Decisive Checks

The central test is proper-noun character error rate on bound, coated non-research pages, comparing our method, multi-frame fusion, and the synthetic-trained network. On the synthetic control, we additionally check that recovered color error falls and recovered-roughness correlation with ground truth rises when the two-tone prior is added. We also check that the self-tracing ablation degrades results specifically within a fixed band around the gutter, and not elsewhere on the page. Finally, we report error against number of frames. Our claim predicts the largest advantage over fusion at three to five closely spaced frames, narrowing as views spread out. A flat curve would falsify the claim that the priors are what make few-view capture work.