A motion-blurred photograph is a record of movement, but in textbook form it is a record without a direction. If every pixel collects light over the same time window, an object sliding from A to B leaves exactly the same smear as one sliding from B to A. Methods that try to turn a single blurred image back into a short video, or into the motion field behind it, have treated this as a hard limit. They either train with a loss that accepts either direction, pick one arbitrarily, or guess from learned common sense about how the world usually moves. Most consumer cameras, however, do not expose all pixels over the same window. A rolling shutter reads the sensor row by row, so lower rows start and end their exposure slightly later than upper rows. That delay shears the blur streak in a direction set by the sign of the motion. Time-reversed motion produces the opposite shear. The textbook symmetry is an artifact of the idealized sensor that synthetic training data assumes.
We propose to treat this as a sim-to-real problem with a specific, testable shape. Synthetic blur built by averaging frames in the standard global-shutter way erases this directional cue. Because the source high-speed footage carries its own short per-frame skew, it may also plant a cue at the wrong scale. Models trained on such data therefore cannot use, or misuse, the one physical arrow of time real phone photos contain. We will synthesize blur with row-dependent exposure windows calibrated to real readout times, train blur-unrolling models with a loss that respects direction, and measure whether they recover the true direction of motion from a single real photo. We will also check whether any gain comes from the physics or merely from extra realism.
Core Claim
For a rolling-shutter sensor with per-row readout delay $\rho$, the observed blur at pixel $(x,y)$ is
$$B(x,y) = \frac{1}{\Delta t}\int_{t_0+\rho y}^{t_0+\rho y+\Delta t} I(x,y,t)\,dt,$$
where $I$ is the sharp scene irradiance, $t_0$ the exposure start of the top row, and $\Delta t$ the exposure time. Substituting $t \mapsto -t$ does not leave $B$ unchanged unless $\rho = 0$. Any motion with a horizontal component, including nearly all handheld camera shake, then yields a shear whose sign encodes the direction of time.
The strength of the cue should scale with the ratio $\rho H/\Delta t$, where $H$ is the height of the streak in rows. This follows directly from the equation above: the shear offset is proportional to the delay between rows spanned by the streak, relative to the exposure over which the streak forms. Our claim has three parts. First, this ratio is large enough on ordinary phone photos for single-image direction recovery well above chance. Second, current synthetic pipelines do not reproduce it. Third, fixing the pipeline, rather than adding video priors or event hardware, is what closes the gap.
Where This Sits in Prior Work
Four lines of work motivate the gap by treating direction as unrecoverable. Optical Flow Estimation from a Single Motion-blurred Image trains with the minimum of forward and backward flow losses, so its predicted sign is meaningless by construction. DeFMO renders fast-moving objects at continuous times and enforces that the rendered frames integrate back to the input, but states that its trajectory is only defined up to $t \mapsto 1-t$. Blur Interpolation Transformer for Real-World Motion from Blur (BiT) supervises both exposure endpoints and ensembles forward and reversed inputs. That gives temporal consistency, but for an isolated frame it leaves the sign open. Generating the Past, Present and Future from a Motion-Blurred Image breaks the symmetry with a large video diffusion model's priors, such as gravity and gait. That prior is silent on symmetric motions like camera pans, wheels and pendulums, and it costs about two minutes per frame. None of these four models the sensor's readout.
A second group supplies our method and our warnings. Realistic Blur Synthesis for Learning Image Deblurring shows that physically faithful blur synthesis, meaning frame interpolation, integration in linear light, saturation, RAW noise and a simulated image-processing pipeline, lifts real-world deblurring from 28.93 to 30.32 dB PSNR on RealBlur-J with no architecture change. We reuse that pipeline and add the one ingredient it omits: row-dependent exposure windows. That paper targets only the mid-exposure sharp frame, where direction does not matter. Reducing the Sim-to-Real Gap for Event Cameras found that calibrating a single sensor parameter to real statistics mattered most, and that the right value differs across datasets. We expect the same of readout time and plan for per-device calibration rather than one global value.
Finally, Learning and Using the Arrow of Time shows that semantic cues let a video classifier tell forward from reversed video about 80% of the time. That is our semantic competitor, but it operates on multi-frame video, not on one exposure.
Research Program
The first component is a measurement study that asks how much direction information real photos physically carry, before any learning. We will record blurry handheld and object-motion videos on several phones at normal frame rates. Ground-truth direction comes for free: the displacement between consecutive blurry frames, estimated by standard flow, fixes the true order within each exposure, so no beam-splitter rig is needed. We will calibrate each device's readout time $\rho$ with a flickering-light or fast-pan target. From these recordings we will measure the distribution of $\rho H/\Delta t$ and fit an idealized direction estimator: a shear-sign test on the local blur-kernel orientation. The fitted estimator gives an upper bound on how often the cue is usable.
The second component is the synthesis pipeline. Starting from the high-frame-rate sources used for GOPRO and Adobe240, we apply the Ji et al. pipeline. Each output row is then assembled only from sub-frames falling inside that row's shifted window, with $\rho$ sampled from the calibrated range of real devices. The source frames have their own short readout skew. We will quantify it for GoPro footage and show whether naive averaging leaves a mis-scaled shear, which would be a synthetic shortcut in the sense described by the BiT authors.
The third component is training. We take BiT as the dense unrolling model and DeFMO for fast-moving objects. The symmetric minimum loss is replaced by an ordinary direction-sensitive loss, with the readout direction supplied as a known input: most phones read top to bottom, and the direction can be read from device metadata. A model that uses the physics should behave predictably under a vertical flip, which reverses the order in which rows are read. In that case it should reverse its predicted direction for horizontal motion. A model that relies on content priors should not.
Baselines
The naive reference is a sign coin-flip. We will pair it with the sign-agnostic model of Optical Flow Estimation from a Single Motion-blurred Image, whose reported end-point error on GOPRO is 2.038 under the minimum-of-both-directions metric. Scored on true direction, that model should sit near 50%, which we will verify. Beating it means significantly higher single-image direction accuracy on our real phone benchmark, with no loss in sign-agnostic flow error.
The strongest existing unrolling model is BiT, which reports 30.45 dB PSNR on the real RBI dataset. We retrain it with identical architecture, iterations and tuning budget under three synthetic regimes: plain sRGB averaging, the Ji et al. pipeline, and ours. That isolates the rolling-shutter ingredient; it is the key ablation. Beating BiT means higher direction accuracy and direction-sensitive PSNR, where frame $t$ is compared to true frame $t$ rather than the better of $t$ and $1-t$, on real data, with RBI PSNR no worse than the reported 30.45 dB.
For fast-moving objects, DeFMO's reported Temporal IoU of 0.684 on Falling Objects is the bar, and it must hold while we add direction. The semantic competitor is the diffusion model of Tedla et al., which reports 30.01 dB PSNR and FVD 21.46 on GOPRO. We will run it on a fixed subset of our benchmark as released, since retraining is out of budget, and report its direction accuracy per motion category. For single-image direction accuracy of any method, there is no reported result on Lacuna, so the benchmark numbers will be new.
Why not just…?
Why not just rely on a video diffusion prior? Tedla et al. show such priors break symmetry when motion is semantically loaded, such as a falling person. The file itself lists symmetric motion as the case where these priors give no cue, and camera shake is almost entirely such motion. Our per-category breakdown will show where the prior works and where readout physics is the only available signal. If the prior matches us even on pans, the physical cue is redundant and we will say so.
Why not just use an event camera? Signed, timestamped events do resolve direction, as in Motion Deblurring with Real Events. But phones lack event sensors; that is exactly why EGDeblurring hallucinates events from one image. We target the billions of existing frame-only photos.
Why not just fine-tune on real paired data with a direction-sensitive loss? This is the real threat. A network trained on real rolling-shutter blur could learn the shear cue without any synthesis work. RBI has only 55 video pairs, however, and it is unknown from our file whether its capture cameras are rolling-shutter. We will test this directly: fine-tune BiT on real data alone, and on our synthetic data plus the same real data, then compare direction accuracy against the amount of real data. If real-only fine-tuning saturates at the same accuracy, our contribution shrinks to the measurement study and the cheap flow-labelled benchmark. That is still useful, but it is a smaller claim.
Why not just add rolling-shutter shear as a generic augmentation? Uncalibrated random shear teaches the model that shear is noise unrelated to motion, which is the opposite of what we want. The ablation sampling $\rho$ independently of motion sign will show whether coupling the shear to the true direction is what matters.
Decisive Checks
Three results settle the claim. The first is the measurement study: if real phone photos rarely reach a usable $\rho H/\Delta t$, the project stops there with a clean negative result. The second is the vertical-flip test, which separates physical reasoning from content priors. A model using readout physics should flip its predicted direction; a prior-based model should not. The third is a global-shutter control, using footage from a machine-vision camera. There, our model's advantage on symmetric motions must vanish, because the cue is physically absent. If it does not vanish, the model is exploiting something other than readout timing.
References
- Optical Flow Estimation from a Single Motion-blurred Image
- DeFMO: Deblurring and Shape Recovery of Fast Moving Objects
- Blur Interpolation Transformer for Real-World Motion from Blur
- Generating the Past, Present and Future from a Motion-Blurred Image
- Realistic Blur Synthesis for Learning Image Deblurring
- Reducing the Sim-to-Real Gap for Event Cameras
- Learning and Using the Arrow of Time
- Motion Deblurring with Real Events
- Diffusion-based Event Generation for High-Quality Image Deblurring