Growing Anisotropy: A Curriculum Over Blur Direction for Deblurring Networks That Learn Their Own Geometry

Most deblurring networks see a shuffled mix of blurs during training. Blur here means the smear a camera applies to a sharp image, and a mix might pair a small round defocus blob with a long diagonal motion streak. Existing curricula, meaning training schedules that go from easy examples to hard ones, rank difficulty by a single number such as blur size or the number of accumulated video frames. We propose ranking difficulty by something else: how directional the blur is. A round, isotropic blur looks the same in every direction. A long one-dimensional streak is maximally anisotropic. Our hypothesis is that direction, more than size, is what makes training hard for networks that must learn where to look. These are deformable convolutions, which learn per-pixel sampling offsets, and directional scanning models. If the hypothesis holds, the training schedule should start from isotropic blur, let the network's learned sampling pattern stretch gradually into streaks, and keep isotropic examples in every batch as an anchor.

This matters for two reasons the literature leaves open. First, deformable deblurring networks are reported to become unstable when streaks exceed their receptive field (Blind Non-Uniform Motion Deblurring using Atrous Spatial Pyramid Deformable Convolution). Second, the MC-Blur benchmark shows that models trained on motion blur fail on lens defocus, and the reverse also holds (MC-Blur). No single model currently spans the isotropic-to-anisotropic range. A curriculum that moves along exactly that range is the obvious candidate for addressing both problems. It is also cheap: it changes only the sampling of training data.

Core Claim

We claim three things. For restoration networks whose sampling geometry is learned, ordering training by blur anisotropy speeds and stabilizes the learning of that geometry, while an anchored ordering retains isotropic skill. The benefit should appear mainly on long, highly directional streaks. It should be much smaller for rigid-kernel CNNs. And it should not come from simply changing how often each kind of blur appears.

To make "anisotropy" concrete, we compute the second-moment matrix of each blur kernel, with eigenvalues $\lambda_{\max} \ge \lambda_{\min}$. We then assign each kernel a size and an anisotropy:

$$ s = \sqrt{\lambda_{\max}}, \qquad a = 1 - \lambda_{\min}/\lambda_{\max}. $$

Here $a = 0$ is a circular blur and $a \to 1$ is a line. At training step $t$, the schedule samples kernels with anisotropy up to a ceiling $a_{\max}(t)$ that rises over training. A fixed fraction $\rho$ of every batch is always drawn near $a = 0$. The size $s$ is sampled over its full range throughout.

This design differs clearly from the three closest works. On-demand Learning for Deep Image Restoration bins deblurring tasks by kernel size and reallocates batches in inverse proportion to per-bin validation performance. Its authors name multi-dimensional difficulty as unsolved. MT-RNN with Incremental Temporal Training schedules along the number of averaged frames, which is again a scalar, and it never touches direction. AR-CNN transfers from mild to severe JPEG quality. All three treat difficulty as one scalar and never ask why an ordering helps. We change the axis to geometry. We also make a falsifiable mechanistic prediction: the gain should track how well the learned offsets align with the true blur direction, and it should depend on architecture.

Research Program

The first component is a controllable blur generator applied to sharp frames. It produces a continuum running from isotropic Gaussian and disk kernels, through elongated Gaussians, to straight line streaks, and finally to curved and multi-segment camera trajectories. Every sample carries its ground-truth $(s, a)$ and its dominant orientation. For real data we attach an estimated $(s, a)$ to each pair. For GoPro, whose blurry images are averages of high-frame-rate frames, we estimate per-patch trajectories from optical flow between the first and last constituent frames. This proxy is our assumption, and we will validate it against the synthetic ground truth. The GRR and pushbroom data have anisotropy that is known from physics. GRR shutters produce vertical, row-dependent blur growth (RSS-T). Pushbroom satellite jitter separates into along-track and cross-track components (JARNet).

The second component is the schedule family. We will compare a sequential anisotropy curriculum with no anchor and an anchored, mixed anisotropy curriculum. The anchored design follows the finding in How does Multi-Task Training Affect Transformer In-Context Capabilities?: purely sequential task curricula erase earlier skills, while keeping easy tasks in the mix prevents this. That paper constrains our design by ruling out the unanchored version. We will also test a feedback variant that raises $a_{\max}(t)$ only once validation PSNR on the current top anisotropy band plateaus. This borrows the reactive idea of On-demand Learning to address its warning that fixed schedules cause fixation.

The third component is the mechanism analysis. For deformable models, we measure how the principal direction of the learned offset field at each pixel aligns with the ground-truth blur orientation. We also track the offsets' spatial extent relative to streak length and the variance of gradients reaching the offset branch over training. For XYScanNet, which scans features separately along horizontal and vertical slices, we measure how much each branch contributes as a function of blur orientation. If the curriculum works for the reason we claim, alignment should rise earlier and more smoothly under the curriculum than under shuffled training. The gap should also be concentrated on large-$s$, high-$a$ samples.

Baselines

The naive reference is uniform joint training. In this setting the same generator and real pairs are shuffled with the standard recipe of each architecture. On-demand Learning found this weaker than its feedback scheme, but only modestly: 32.11 dB against 32.30 dB on CelebA inpainting. That small gap tells us uniform training is a serious competitor and not a strawman.

The most important control is a shuffled-curriculum baseline. It sees exactly the same multiset of training samples as the anchored curriculum, with the same count per $(s, a)$ cell, but in random order. Curricula silently change the data mixture, and this control is the only way to show that ordering matters rather than reweighting. We also run a size-only curriculum, meaning the anchored schedule applied to $s$ with $a$ sampled uniformly, to isolate the anisotropy axis itself.

The strongest scheduling competitor is On-demand Learning extended to a two-dimensional grid of $(s, a)$ bins, with batch share set inversely proportional to per-bin validation PSNR. This is the extension a reviewer would propose first.

The strongest architectures on the benchmarks we use serve as backbones rather than rivals. On GoPro, MSDI-Net reports 33.28 dB PSNR, the ASPDC network reports 32.09 dB, RADN reports 31.76 dB, and XYScanNet reports a KID of 0.073. On RealBlur, MSDI-Net reports 32.35 dB. On GRR-real, RSS-T reports 28.64 dB against 24.48 dB for MIMO-UNet. On the synthetic pushbroom set, JARNet reports 39.02 dB against 37.74 dB for Stripformer. On the MC-Blur UHDM subset, MPRNet reports 23.70 dB. We found no reported result on Lacuna for any single model evaluated jointly on MC-Blur's defocus (LSD) and motion subsets.

Fairness rules apply to every comparison. Every schedule is applied to the same backbone with the same data pool, iteration count, optimizer, and augmentation. Each receives an identical tuning budget of six configurations. For our schedule these cover $\rho$ and the pacing rate. For On-demand they cover bin count and update frequency. For uniform training they cover learning rate and warmup. All runs use three seeds.

We first reproduce each backbone under our budget, so that gains are measured against our own reproduction rather than against paper numbers. A win means two things. On the pre-registered high-anisotropy, long-streak stratum, the anchored curriculum must beat both uniform training and the shuffled control by more than the seed-to-seed spread. We treat 0.2 dB as the practical threshold. It must also not lose more than 0.05 dB on the isotropic stratum.

Why not just…?

Why not just train on everything shuffled with a big model? This is the most serious threat, because the only direct image-restoration evidence against it is a 0.19 dB inpainting gap. The shuffled-curriculum control settles the question directly. If it matches the curriculum, then ordering is irrelevant, and any benefit comes from the data mixture that the curriculum implied. We would report that as a negative result and recommend mixture design instead. We also run the comparison on the largest backbone, MSDI-Net, because capacity could plausibly wash out ordering effects.

Why not just extend On-demand Learning to a 2D grid of blur size and direction? This may work as well, and we run it as a baseline rather than argue against it. The two methods make different predictions. Error-driven allocation will pour batches into the hardest long-streak cells from the start. Our hypothesis says those cells produce noisy offset gradients before the offsets have learned a direction. If 2D On-demand matches our method on accuracy, the contribution narrows to the choice of anisotropy as the difficulty axis. The alignment curves would still show whether the two schedules reach the solution the same way.

Why not just fix the optimization of deformable offsets directly, with zero initialization, a lower offset learning rate, or warmup? These are standard tricks, and they address the same instability that Huo et al. report. We include an offset-warmup variant under uniform training. If it closes the gap on long streaks, then the curriculum is only an indirect way of doing offset warmup. We would then say so. The architecture-interaction test described below also discriminates here. A rigid CNN has no offsets, so offset tricks cannot help it. A curriculum could still help it if the benefit is not about geometry.

Why not rely on coarse-to-fine multi-scale networks, which already shrink blur at low resolution? Downsampling shortens streaks but does not change their anisotropy, and it discards the high-frequency detail needed to invert them. This is the argument MT-RNN makes for scheduling at full resolution. We include MIMO-UNet, a multi-scale backbone, in the architecture comparison. Its response to the curriculum measures directly whether multi-scale processing already captures the benefit.

Proposed Experiments

The first decisive check is the core ablation on the synthetic generator applied to GoPro sharp frames. We cross four schedules (uniform, shuffled control, size-only curriculum, anchored anisotropy curriculum) with three backbones: MIMO-UNet with rigid kernels, the ASPDC deformable network, and XYScanNet with directional scans. We report PSNR and SSIM stratified by size and anisotropy on a held-out grid. The grid includes streaks longer than the deformable receptive field, which is where Huo et al. locate the failure. The hypothesis is supported only if two conditions hold together. The anisotropy curriculum must beat the shuffled control on the long-streak, high-anisotropy stratum for the deformable and directional models. And it must show a clearly smaller effect for the rigid CNN. An effect equal across all three architectures would falsify the geometry-learning explanation, even if accuracy improves.

The second check is transfer to real motion. The best schedule and its controls are fine-tuned and evaluated on GoPro and RealBlur, with NIQE also reported on RWBI. We then evaluate on GRR-real and the synthetic pushbroom set, whose directional structure is known. Here the curriculum's final stage uses vertical row-gradient kernels for GRR-real and along-track/cross-track jitter kernels for the pushbroom data. Results are reported against RSS-T and JARNet as reproduced.

The third check targets the isotropic–anisotropic unification gap. We train one model per schedule on MC-Blur's defocus (LSD) and real high-fps motion (RHM) subsets together. The anchored curriculum starts from defocus and grows toward motion. We report each subset separately against specialist models trained on one subset only. Success means a single anchored model comes within 0.3 dB of each specialist on its own subset. Uniform joint training and the unanchored sequential curriculum serve as the comparisons, with the unanchored version expected to forget defocus.

Across all three checks, we report offset–orientation alignment and offset-branch gradient variance over training. These curves are the evidence that separates "the curriculum helps because it teaches geometry" from "the curriculum helps for some other reason."