Scene text detectors find and outline words in photographs. They work well on the benchmark they were trained on and degrade sharply elsewhere. A detector trained on synthetic renderings and shown real curved storefront text, or trained on one dataset and deployed on another, loses many points of accuracy. Two different things change between datasets. The first is appearance: lighting, sensor noise, background texture. The second is geometry: how strongly text bends, how long lines run, how much perspective tilts them. Existing fixes for scene text are domain adaptation methods. They need a pile of unlabeled images from the deployment domain at training time, and they were built around detectors that output rotated rectangles. Nobody has shown how to train an arbitrary-shape detector, meaning one that outputs a free-form outline, so that it holds up on a domain it has never seen.
The proposal is to train such a detector against an adversary that deforms text shapes. Modern detectors already describe each word outline with a handful of numbers: weights on a small set of "eigen-shapes" (the main modes of contour variation, learned from training outlines). That compact description lets us deform a word on purpose. We nudge its shape coefficients in the direction that most increases detection loss, warp the image pixels to match, and get the new ground-truth outline exactly, for free. Paired with an adversary that perturbs only the coarse color-and-illumination part of the image spectrum, this gives the detector hard but label-correct training examples on both the geometry axis and the appearance axis. No target images, no generative model, and no extra inference cost are required.
Core Claim
The claim is that the low-rank shape space used by compact contour detectors makes label-consistent adversarial training possible for structured detection. This resolves what the research file calls the "spatial hurdle": in classification, invariance training uses transformations that leave the label unchanged, while in detection every spatial transformation changes the label.
Concretely, we take the contour basis from LRANet. LRANet fits a singular value decomposition to training contours and reconstructs outlines from 14 coefficients at 98.0% IoU. We then write each text instance as
$$c = \mu + B w,$$
where $c$ is the sampled boundary point vector, $\mu$ the mean contour, $B$ the eigen-shape matrix, and $w$ the instance's coefficients.
The geometric adversary searches for a perturbation $\delta$ that maximizes detection loss. It is constrained to $\delta^\top \Sigma^{-1} \delta \le \epsilon^2$, where $\Sigma$ is the covariance of $w$ over the source data. The warped image is produced by a thin-plate-spline map (a smooth 2D warp fitted to matching point pairs) from $c$ to $\mu + B(w+\delta)$, sampled differentiably so $\delta$ can be found by a few gradient-ascent steps. The new label is $\mu + B(w+\delta)$ itself.
The Mahalanobis budget, which measures distance in units of the source shape spread, keeps deformations on the plausible-shape manifold while pushing toward its tails. Those tails are where the file documents parametric detectors failing: S-shapes for Bezier models, U-shapes for polar rays, very long lines on CTW1500 (ABCNet, TextRay). A second adversary perturbs only the low-frequency Fourier amplitude of the image while holding phase fixed. We borrow this style/structure split from FAN-Net, which showed for multi-site MRI that low-frequency amplitude carries site style and phase carries boundaries.
Relation to Prior Work
Three works adapt scene text detectors across domains, and all three motivate this proposal by documenting the gap. GA-DAN showed that appearance-only translation misses a "geometry gap." It used a disentangled CycleGAN with a spatial discriminator to synthesize target-like warps, lifting EAST from 53.3% to 69.4% F-score on ICDAR2013→ICDAR2015. Synthetic-to-Real UDA put a gradient-reversal domain classifier inside EAST and added stroke-width-filtered self-training. SCAST replaced binary text/background labels with clustered subcategories to tame overconfident pseudo-labels, reaching 63.39% on SynthText→Total-Text with PSENet. All three need unlabeled target images. The first two output rotated boxes only.
We keep GA-DAN's insight that geometry must be adapted, not just style. We drop its requirement for target data and an image generator: the adversary works inside the detector's own training loop, and the geometric search space comes from shape statistics rather than from a discriminator trained on target images.
Episodic Training for Domain Generalization supplies the regime: train without any target exposure. Its authors flag extension to structured regression as open. We take up that challenge, but we do not port its mismatched-classifier scheme, which has no obvious analogue for a 14-dimensional coordinate head. FAN-Net supplies the appearance mechanism. The file raises a tension here: TextDCT and LRANet say text geometry is low-frequency, while FAN-Net perturbs low frequencies. We think these are different spectra, the image's versus the mask's. Our ablation tests directly whether amplitude perturbation erodes contour regression accuracy.
Keypoints and Links handles geometric variation by rectifying text with thin-plate splines at inference. It runs at 7.2–7.6 FPS and does not address appearance. We use the same warp only at training time.
Research Program
We will build the adversary as a detector-agnostic training module. It takes an image with polygon labels and returns a warped, re-styled image with exact polygon labels. This means it can wrap LRANet (our main detector), PSENet (for direct comparison with SCAST), and ABCNet.
The geometric step goes as follows. We fit $B$ and $\Sigma$ on source contours. For each instance, we run $k$ projected gradient steps on $\delta$. Background pixels are warped by the same thin-plate spline anchored at image corners, which handles neighboring instances jointly. Results are filtered when warped instances overlap.
The appearance step perturbs a low-frequency amplitude window, starting at FAN-Net's $\alpha = 0.1$ and sweeping above it. FAN-Net reports edge loss above that value.
The analysis questions are three. Does adversarial search beat random sampling at an equal budget? Which axis, geometry or appearance, accounts for the gain on each target dataset? Do the shape coefficients predicted on unseen targets land in the same region of shape space as the adversary's hardest examples?
Baselines
The naive reference is the same detector trained on the same sources with standard augmentation only. For LRANet trained on SynthText plus ABCNet's 150k synthetic curved images, then tested zero-shot on Total-Text, we found no reported result on Lacuna. We will produce it. For PSENet, SCAST reports a source-only 56.35% F-score on SynthText→Total-Text. Wu et al. report 60.5% for EAST on SynthText→ICDAR2015.
The strongest existing competitor on curved text transfer is SCAST, at 63.39% on Total-Text and 70.14% on ICDAR2015 (with EAST). It is the strongest because it is the only cited method evaluated on curved targets, and it uses target images we deliberately withhold. We will run our module on PSENet with SCAST's sources, schedule, and backbone. That makes the comparison detector-matched. Our method is disadvantaged in data, since it sees no target images. Matching 63.39% without target data would count as a strong result. Closing at least half the gap from 56.35% would count as clearly beating the naive reference.
We also re-implement Wu et al.'s gradient-reversal alignment on LRANet with access to target images. This adaptation upper reference shows how much of the adaptation benefit we recover without target access.
GA-DAN is reported only on EAST and recognition. We compare against its ICDAR2013→ICDAR2015 number of 69.4% by running our module on EAST in the same setting.
The key-ingredient ablation is random coefficient sampling under the same Mahalanobis budget and the same number of augmented samples, with random amplitude perturbation. It is reported separately for each axis. All variants get the same total GPU hours and the same hyperparameter search count (we propose eight configurations each), tuned on a held-out source split, never on target test sets.
Why not just…?
Why not just pool more real datasets? The episodic DG paper concedes that vanilla pooling often beats elaborate generalization methods, and that episodic training beat it by only 3.3% on PACS. Pooling is a real threat. Our leave-one-dataset-out experiment uses pooling as its baseline arm. If pooling two real sources matches our method trained on the same two sources, the adversary is not earning its cost.
Why not just apply random Fourier style mixing and random warps? This is the most serious alternative. It is cheap and could capture most of the gain. The equal-budget random-sampling ablation exists to settle it. If random and adversarial tie, we will report the contribution as label-exact shape-space augmentation and drop the adversarial claim. That is an honest, narrower result, and it is still the first method we know of that warps arbitrary-shape labels without breaking them.
Why not just use domain adaptation with target images, as ATA or SCAST do? When target images exist, that is reasonable, and our gradient-reversal LRANet run quantifies the advantage. But deployment domains are often unknown in advance. Wu et al.'s stroke-width filter also assumes uniform straight strokes, which curved and artistic text violates.
Why not use GA-DAN's generator to synthesize warps? It needs target images to train its spatial discriminator. It also adds a multi-stage generation pipeline, and its outputs were never tied to contour-coefficient labels.
Proposed Experiments
There are two protocols. In synthetic-to-unseen-real, we train on SynthText plus ABCNet synthetic data and test zero-shot on Total-Text, CTW1500, ICDAR2015, and the ArT test set, which aggregates curved Latin and Chinese text. In leave-one-real-dataset-out, we rotate among Total-Text, CTW1500, and ICDAR2015. CTW1500 annotates whole lines while the others annotate words. We will report that mismatch explicitly and use ArT's word-level Latin subset as the cleanest cross-check. Metrics are precision, recall, and F-score, plus FPS to confirm that inference cost is unchanged.
The decisive checks are four. First, PSENet with our module, trained without any target images, should reach or exceed SCAST's 63.39% on Total-Text. Second, adversarial perturbation should beat random perturbation at equal budget on at least three of four targets. Third, gains should concentrate on the high-curvature instances in Total-Text's curved and circular categories and on long CTW1500 lines; we will stratify instances by their coefficient distance from $\mu$. Fourth, in-domain CTW1500 accuracy should stay within one point of LRANet's reported 87.4%, which would show the adversary does not trade source accuracy for robustness. If the amplitude adversary lowers reconstruction IoU of predicted contours as $\alpha$ grows past 0.1, the frequency tension in the file is real for text. We would then restrict the appearance adversary accordingly.
References
- LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network
- ABCNet: Real-Time Scene Text Spotting With Adaptive Bezier-Curve Network
- TextRay: Contour-based Geometric Modeling for Arbitrary-shaped Scene Text Detection
- Fan-Net: Fourier-Based Adaptive Normalization for Cross-Domain Stroke Lesion Segmentation
- GA-DAN: Geometry-Aware Domain Adaptation Network for Scene Text Detection and Recognition
- Synthetic-to-Real Unsupervised Domain Adaptation for Scene Text Detection in the Wild
- Domain Adaptive Scene Text Detection via Subcategorization
- Episodic Training for Domain Generalization
- Detection and rectification of arbitrary shaped scene texts by using text keypoints and links