Recognizing curved, rotated or multi-line text in photos is, at heart, a problem of reading order. The model has to visit image regions in the sequence a human would read them. Today two families of models handle this. Attention-based readers look over the whole 2D feature map at every character and work out where to look next. They are accurate on curved text, but they are slow or quadratic in cost. Linear recurrences can process a long sequence of image patches cheaply in parallel. These are recurrent networks whose state update has no nonlinearity, so all states can be computed at once with a parallel prefix sum ("scan"). The catch is that they only work if someone hands them a 1D order, and vision versions of them use fixed orders such as raster rows, four corners or concentric rings. None of those orders follows a curved word.
We propose to let the network predict the order itself. A cheap convolutional head assigns every feature-map cell a scalar "reading position" between 0 and 1. A differentiable readout gathers cells into character slots according to that position. Scans in the encoder then run over the cells sorted by the current reading position, not by raster order. Training uses only word-level transcriptions. The underlying bet is that a linear recurrence can read curved, rotated or stacked text about as well as a 2D attention decoder, but only if its scan order follows the text, and that this order is learnable from word labels alone.
Core Claim and What Is New
The claim has two parts, and an experiment can refute each one. First, a linear-recurrent text encoder that scans in a fixed geometric order should lose clearly to 2D attention readers on curved text. Second, the same encoder with a learned, text-following order should close most of that gap at linear cost in the number of cells.
The novelty sits in a specific place. Vision state-space models already scan 2D grids, and their scan orders are fixed by geometry. 2D-CrossScan Mamba propagates state from left and upper neighbours and launches scans from four corners. It removes double-counted paths with a heuristic subtraction of the diagonal neighbour. PRIS-Mamba scans concentric rings around a fixed centre. It shows that raster scans create directional bias and brittleness under rotation, which motivates our work. Its own design, however, assumes a centred object, which rules it out for off-centre, high-aspect-ratio text. Both papers are evaluated only on classification, detection and segmentation. Neither makes the scan order depend on image content.
On the text side, three lines of work trace curved paths with 2D attention and keep the full feature grid. SAR attends over every cell at each decoding step. SATRN uses full 2D self-attention with an adaptive 2D positional encoding. Learning to Read Irregular Text with Attention Mechanisms adds coordinate maps and character-box supervision. All three pay either step-by-step decoding or attention cost quadratic in the number of cells.
Rectification-based readers straighten the pixels before reading. Towards Unconstrained End-to-End Text Spotting reports this kind of geometric warping to be fragile, which constrains our design. Our approach also "straightens" the text, but it does so by reordering feature cells. It never resamples pixels, so a bad estimate misorders features without smearing strokes.
Mechanism
The encoder is a short stack of blocks over a feature grid of $H\times W$ cells. Each block does three things. It mixes locally with a poly-scale depthwise separable convolution. It predicts or refines the reading-position field. It then runs a bidirectional Linear Recurrent Unit over the cells sorted by that field.
The Linear Recurrent Unit comes from Resurrecting Recurrent Neural Networks for Long Sequences. That paper builds a diagonal complex-valued linear recurrence, computed with a parallel scan, that matches deep state-space models on Long Range Arena and solves PathX at 94.2%. We reuse that unit unchanged, and the only thing we change is the order it receives. Sorting $HW$ scalars is cheap. The scan has depth $O(\log HW)$ and total work linear in $HW$. The sort inside the encoder uses a stop-gradient copy of the field, so encoder blocks see a hard permutation.
The field learns through the readout. Let $p$ index cells, let $f_p$ be the final feature at cell $p$, let $s_p\in[0,1]$ be its predicted reading position, and let $m_p\in[0,1]$ be a predicted "is text" weight. Let $k=1,\dots,K$ index $K$ character slots. Slot $k$ collects
$$z_k=\frac{\sum_p m_p\, e^{-(K s_p-k)^2/2\sigma^2}\, f_p}{\sum_p m_p\, e^{-(K s_p-k)^2/2\sigma^2}+\epsilon},$$
where $\sigma$ is a fixed slot width and $\epsilon$ prevents division by zero. The slot sequence $z_1,\dots,z_K$ passes through one more 1D linear scan and is trained with connectionist temporal classification (CTC), a loss that aligns a label sequence to an output sequence monotonically. Because $z_k$ is a smooth function of $s_p$, the CTC gradient tells the field which cells should come earlier or later. No character boxes are needed, which removes the costliest requirement of Learning to Read Irregular Text with Attention Mechanisms.
For multi-line text, the field must order whole lines one after another. We test whether it learns this unaided. As a fallback, we add a coarse line-index channel to $s$.
The two convolutional directions play supporting roles. The research file notes that pure linear recurrences compress local detail and can miss fine stroke boundaries. SVTR likewise needed local mixing to separate look-alike characters such as "E" and "F". We therefore give each block a cheap local mixer. The order-field head also needs a wide view, so it can see where the word bends.
Multi-scale dilation, meaning kernels with gaps that see a wider area for the same cost, provides that wide view. We assign the cyclic dilation pattern of PSConv (1, 2, 1, 4 across channels) to the depthwise stage of a depthwise separable convolution. That convolution consists of a per-channel spatial filter followed by a $1\times1$ channel mixer. PSConv spread multiple scales across a dense kernel at no extra parameters and gained 2.1 AP on COCO with Mask R-CNN. In the depthwise case, each channel owns one kernel, so PSConv's per-output-filter shift collapses. Cross-scale mixing then happens entirely in the pointwise step. The cost is $K^2M+MN$ rather than $K^2MN$, where $K$ is kernel size and $M,N$ are input and output channels.
This component resembles the dilated depthwise branches of MiniSeg, which reached 82.91K parameters and 516.3 FPS on CT segmentation. Our version is a single layer instead of parallel branches. We treat it as an efficient default to be ablated, not as a headline contribution.
Research Program
The work proceeds in three stages. The first stage builds a shared codebase. The fixed-order variants (raster, four-corner cross scan, ring scan) and the learned-order variant share every component except how the permutation is produced. This lets accuracy differences be attributed to order alone.
The second stage tests whether the field learns meaningful order. We visualize $s$ on CUTE80 and on rotated copies of the test sets. We also measure agreement between the learned order and the order of attention centroids from a trained SAR model. SAR's attention serves only as a reference; it never supervises the field. A cheap check also matters here: a field that ignores curvature and reproduces raster order would show near-perfect rank correlation with column index on curved words.
The third stage measures efficiency honestly. The file warns that depthwise and non-uniform dilated convolutions are often memory-bound on GPUs, and Optimizing Depthwise Separable Convolution Operations on GPUs addresses exactly that. We therefore report measured latency, not only FLOPs. We use a 1080Ti, the card on which SVTR and CPPD report speeds, across grid sizes from single words up to multi-line crops. The linear-versus-quadratic advantage should only appear as the grid grows.
Baselines
The naive reference point is the same encoder with a raster-order scan and the same slot readout, but with $s$ fixed to normalized column position. This is the "do nothing about order" model. If the learned field does not beat it clearly on CUTE80 and SVTP, the central claim fails. The four-corner scan from 2D-CrossScan Mamba and a ring scan in the style of PRIS-Mamba serve as stronger fixed-order baselines. Both are reimplemented inside our block with identical parameter counts.
The strongest accuracy competitors are parallel-decoding transformers. PARSeq reports 91.9% average accuracy on the standard English benchmarks with synthetic training and 96.0% with real data. CPPD reports a 93.8% English average at 400 FPS. These are the strongest numbers in the file. Under matched synthetic-only training, PARSeq's 91.9% is the comparable target.
The strongest efficiency competitor is SVTR. SVTR-T has 6.03M parameters and runs at 4.5 ms per image. For geometric robustness, the reference is SATRN. It reports roughly 90% across 0°, 90° and 180° rotations and 63.8% on multi-line text, versus 46.7% for SAR. SAR's 89.6% on CUTE80 and 86.4% on SVTP anchor the curved-text comparison.
For fairness, we retrain PARSeq, SVTR-T and SAR in our pipeline on MJSynth plus SynthText. They get the same schedule, the same augmentation and the same hyperparameter-search budget as our model. Our model is sized to SVTR-T's parameter count. Published numbers are reported alongside the reproduced ones.
We would count the result as a win under three conditions. The model must come within about one point of reproduced PARSeq on the English average. It must beat SVTR-T's latency at equal parameters. It must match SATRN-level rotation and multi-line robustness. Beating PARSeq outright is not required, because the claim concerns accuracy at linear cost.
Three ablations isolate the ingredients. The first swaps the learned field for each fixed order. The second removes the encoder scans entirely, keeping the learned field and slot readout over a convolutional encoder. The third replaces the poly-scale depthwise mixer with a plain $3\times3$ depthwise mixer and with MiniSeg-style parallel dilated branches.
Why not just…?
Why not just rectify the image with a thin-plate spline and run an ordinary 1D linear recurrence? The file's evidence goes against it. SAR's move from rectification to 2D attention on the unwarped grid gained 10.1 points on CUTE80 and 7.5 on SVTP. SATRN reports that rectification and height-collapsing models fail outright at 90° and 180°. Our design keeps the benefit, a 1D order along the text, while removing the pixel resampling that causes the failures. We still include a rectifier plus 1D recurrence as a reproduced baseline, so that this argument is tested rather than assumed.
Why not just use PARSeq or CPPD, which already decode in parallel? This is the real threat on accuracy, and we do not expect to beat them on standard benchmarks at real-data scale. Their decoders are already parallel; the cost they leave is a quadratic-attention encoder over all cells. That cost is small for single-word crops and grows for multi-line or high-resolution inputs. The grid-size latency sweep settles where, if anywhere, the scan wins. If it wins only at sizes nobody uses, we will say so.
Why not just use a vision state-space model with four fixed scan directions? Four directions do include reversed and transposed orders, so 90° and 180° rotations are partly covered by symmetry. A real chance exists that this suffices on rotation tests. No fixed direction, however, follows a curved arc or a two-line layout. The decisive comparison is therefore curved text (CUTE80) and multi-line text, where the learned order must clearly beat the four-corner scan. On pure rotation, we expect a tie.
Why not just use the learned-order readout and skip the scans? If the slot readout alone, on top of a convolutional or SVTR-style encoder, matches the full model, then ordering the encoder is unnecessary. In that case the contribution shrinks to a box-free, warp-free readout. The second ablation tests exactly this, and we will report whichever outcome holds.
Proposed Experiments
All models train on MJSynth and SynthText. Evaluation covers IIIT5K, IC13, IC15, SVTP and CUTE80. We add the rotation and multi-line stress protocols of SATRN. We also include two further stress sets for which the file contains no reported baseline numbers. Union14M-L is a large text-in-the-wild set. COO (Comic Onomatopoeia) contains extremely curved and truncated text (COO).
The first decisive check is the curved-text gap. On CUTE80, the learned order should recover most of the distance between the raster-order model and reproduced SAR, which reports 89.6%. A recovery below half would falsify the claim.
The second check is robustness. Accuracy should stay flat across 0°, 90° and 180°, as SATRN's does. Multi-line accuracy should exceed the four-corner scan's by a clear margin.
The third check is scaling. Measured latency and memory should grow roughly linearly with cell count while PARSeq's grow faster. The crossover grid size will be reported explicitly.
The fourth check is interpretability. The learned field's order should agree with SAR attention-centroid order on curved words much better than column order does. That agreement would show the model reads along the text, not around it.
References
- 2D-CrossScan Mamba: Enhancing State Space Models with Spatially Consistent Multi-Path 2D Information Propagation
- Partial Ring Scan: Revisiting Scan Order in Vision State Space Models
- Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
- On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention
- Learning to Read Irregular Text with Attention Mechanisms
- Towards Unconstrained End-to-End Text Spotting
- Resurrecting Recurrent Neural Networks for Long Sequences
- SVTR: Scene Text Recognition with a Single Visual Model
- PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer
- MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation
- Optimizing Depthwise Separable Convolution Operations on GPUs
- Scene Text Recognition with Permuted Autoregressive Sequence Models
- Context Perception Parallel Decoder for Scene Text Recognition
- COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts