Image classifiers trained on ImageNet mostly recognize objects by surface texture, such as fur pattern or skin grain, rather than by outline. The same reliance on fine, high-frequency detail is why these models break when an image is shown at a lower resolution than the one they were trained on. We propose to attack both problems with iterated learning, a training scheme from the study of language evolution in which each "generation" of learner is trained only on labels produced by the previous one. Theory shows that such a chain forgets its starting data and drifts toward whatever the learner, together with the channel it learns through, finds easiest to learn. Our idea is to make the channel itself a resolution switch. Every other generation sees images only at low resolution, where texture is physically absent. That generation then labels a sparse sample of images for a fresh high-resolution student, which is trained under a tight budget.
Put plainly, the low-resolution generations act as a filter that texture cannot pass through, and iteration lets that filter accumulate. Over generations the high-resolution model should come to depend on features that survive downsampling. Shape is the main such feature. This should raise shape bias and shrink the accuracy gap across resolutions, without style-transfer datasets and without a fixed teacher. The proposal is built to answer a skeptical question directly: does the iteration add anything beyond one round of low-to-high distillation or blur augmentation? If it does not, we want to find that out cleanly.
Core Claim
The Markov-chain analysis of iterated learning (Iterated learning: Intergenerational knowledge transmission reveals inductive biases; Connecting human and machine learning via probabilistic models of cognition) shows that a chain of Bayesian learners converges to the learners' prior. This is a warning as much as a promise. A chain of ordinary high-resolution CNNs should converge toward their own texture-favoring bias, not away from it. Our claim is that the bias a chain amplifies belongs to the learner and the channel together. A deliberately lossy, heterogeneous channel, here alternating resolutions, therefore changes which features are stable under transmission.
Formally, let $f_t$ be the generation-$t$ classifier and $D_{r}$ the operation that downsamples an image to resolution $r$. Generation $t$ produces soft labels on a random transmission set $S_t$ (a fraction $\rho$ of an unlabeled pool) by evaluating $f_t(D_{r_t}(x))$. Generation $t+1$ is then trained from scratch at resolution $r_{t+1}$ on those labels, plus a small anchor set with true labels, for a capped number of steps $K$. With $r_t$ alternating between, say, 32 and 224 pixels, any feature that predicts a label at 224 but carries no signal at 32 cannot be reinforced by the low-resolution generations. Features that survive $D_{32}$ are reinforced in every round.
The specific, testable prediction concerns the trajectory, not only the endpoint. Shape bias in the high-resolution generations should ratchet upward across generations. It should exceed what one round of low-to-high distillation gives at matched total compute, and also what a single model trained with random-resolution or blur augmentation gives.
Why This Is Not Already Done
Three lines of prior work come close, and each stops short in a specific way.
Iterated learning and transmission bottlenecks are known to produce compositional structure in emergent languages. Compositional languages emerge in a neural iterated learning model resets agents each generation and raises topological similarity from about 0.2 to over 0.9. Ease-of-Teaching and Language Structure from Emergent Communication resets the listener periodically. In Emergent Communication: Generalization and Overfitting in Lewis Games, a partially trained listener plays a similar role. All of these operate on one-hot attribute vectors, never on pixels. They motivate the mechanism but say nothing about vision.
Iterated learning for emergent systematicity in VQA supplies our learning bottleneck. It caps student training through early stopping and spectral normalization, on the logic that regular structure is learned first. However, that paper applies the bottleneck only to the symbolic program generator and leaves the visual backbone fixed. It also reports that the approach struggled on real images in GQA.
Cross-resolution work points the other way. Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation transfers knowledge once, from a fixed high-resolution teacher to a low-resolution student. It finds that direct feature mimicry fails because high-resolution features contain detail the student cannot compute. It never measures shape bias. Resolution Attack: Exploiting Image Compression to Deceive Deep Neural Networks shows the physical fact we rely on: downsampling removes high-frequency texture and leaves coarse shape. That paper uses the fact only to attack models.
Our proposal differs from all of these in three ways. The channel is heterogeneous in resolution. The thing being filtered is a continuous visual representation, not symbols. The direction of transfer alternates in a closed loop instead of running once from a strong teacher.
One piece of motivating evidence is less obvious. Shape or Texture: Understanding Discriminative Features in CNNs reports that 97.1% of a CNN's shape-encoding capacity appears in the first 10% of training. If so, the step cap $K$ borrowed from Vani et al. should favor shape over texture for a reason specific to vision, beyond the general claim that regular structure is learned first.
Research Program
We will build the chain on ResNet-50, alternating 32- or 64-pixel generations with 224-pixel generations. Chains run on an ImageNet subset with the 16 cue-conflict categories, or the ImageNet20 setup used by Does enhanced shape bias improve neural network robustness to common corruptions?, which keeps multi-generation runs affordable. The best configuration is then run once on full ImageNet.
Three knobs set the bottleneck: the low resolution $r_{low}$, the transmission fraction $\rho$, and the step cap $K$. We sweep them on a coarse grid. The goal is to locate the "interval of advantage" that Compositional languages emerge in a neural iterated learning model found for pre-training length. Too tight a bottleneck collapses the chain, a failure the file also documents. Too loose a bottleneck lets the student simply copy its teacher.
To keep the chain tied to the actual classification task, each generation also trains on a small anchor set with true labels. This plays the role of the "interaction" phase in neural iterated learning.
We also borrow one asymmetry experiment from that paper, which found that resetting the speaker, the agent doing the compression, matters most. In one arm we reinitialize only the low-resolution generations and partially warm-start the high-resolution ones. In the other arm we do the reverse.
Throughout, we log per-generation measurements: shape bias, clean accuracy, and accuracy at each test resolution. Following Shape or Texture, we also log linear-probe readouts of shape and texture information in late layers. These logs let us distinguish a real ratchet from a one-time jump.
Baselines
The naive reference is a standard ResNet-50 trained at 224 pixels on the same subset with the same total number of gradient steps as the whole chain. On full ImageNet, this model's reported shape bias is 22% (ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness).
The strongest shape-bias competitor is training on Stylized-ImageNet, from the same paper. It reaches 81% shape bias and lowers ImageNet-C mean corruption error from 76.7 to 69.3, but it costs clean accuracy. On the corruption axis, the strongest competitor is the SE+IN superposition model from Does enhanced shape bias improve neural network robustness to common corruptions?. It reports 78.96% mean corruption accuracy on ImageNet20 with a shape-bias score of only 22. We will retrain both on our subset with the same step budget and the same hyperparameter-search budget. We will quote the published numbers only as sanity anchors.
On the low-resolution side, we compare our low-resolution generation against CRRCD, which reports 89.33% on SVHN downsampled to 8×8. We run CRRCD with the same teacher, the same data, and the same compute.
Two baselines are, in effect, ablations that isolate our key ingredient. The first is a single model trained with random-resolution and Gaussian-blur augmentation at matched compute. It tests whether the low-resolution channel works merely as blur augmentation. The second is a homogeneous chain: the same chain, the same $\rho$ and $K$, but every generation at 224 pixels. This isolates the resolution switch from the effects of self-distillation and resets.
A third ablation compares one round of 32→224 distillation against the full chain at the same total compute. This isolates iteration itself.
We would claim to beat these baselines under three conditions, all holding together at matched compute. First, the chain's final model exceeds the blur-augmented and single-round models in shape bias by a margin larger than the spread across seeds. Second, it does so without falling more than about two points of clean top-1 accuracy below the naive reference. Third, it narrows the accuracy drop between 224- and 64-pixel test inputs. Matching Stylized-ImageNet's shape bias is not required. Matching it while keeping clean accuracy would be a strong result.
Why not just…?
Why not just train one model with random-resolution or blur augmentation? This is the most serious threat. A low-resolution generation is, functionally, a model that has only ever seen blurred images. The difference is that augmentation still lets a single high-resolution model use texture whenever texture is present. In our chain, the high-resolution student is trained only to reproduce labels that were computed without texture. Natural images correlate texture with shape, however, so that student may still learn texture as a cheap proxy. The compute-matched augmentation baseline and the per-generation shape-bias curve settle the question. If augmentation matches the chain, we will report that the resolution channel reduces to augmentation and the iterated framing adds nothing.
Why not just distill once from a low-resolution teacher into a high-resolution student? The Markov-chain view predicts that one round only partly moves the hypothesis, because the student keeps its own prior. Repeated rounds are what drive the chain toward the channel's stationary point. That prediction could be wrong, and the single-round ablation at equal compute tests it directly. A flat curve after generation two would mean iteration is unnecessary.
Why not just use Stylized-ImageNet or amplitude-phase recombination? Both work as single-model shape-bias methods. Stylized-ImageNet costs clean accuracy and offers no remedy for resolution shift. The DCTMamba results (DCTMamba: Advancing JPEG Image Restoration Through Long-Sequence Modeling and Adaptive Frequency Strategy) indicate that frequency coefficients drift with spatial size, which suggests that frequency filters tuned at one resolution may not transfer. Mummadi et al. warn that shape bias alone does not buy robustness. Their edge-map models, with 46% shape bias, fall to 27% accuracy under Gaussian blur. We therefore compare against Stylized-ImageNet on all three axes: shape bias, corruption robustness, and resolution gap. We do not claim to win on shape bias alone.
Proposed Experiments and Decisive Checks
We evaluate on five benchmarks. The 1,280-image cue-conflict set measures shape bias. ImageNet-C (Benchmarking Neural Network Robustness to Common Corruptions and Perturbations) measures corruption robustness, with blur and pixelation reported separately from noise and weather. Clean accuracy is measured at test resolutions of 32, 64, 128, 224, and 448 pixels. The resolution attack, which reports a 71.4% corrective attack success rate on ResNet-50, tests whether the model's decision follows the low-frequency shape, which is the stated purpose. For the low-resolution end we use SVHN at 8×8 and TinyImageNet. All results are averaged over three seeds.
The first decisive check is the ratchet. Shape bias in high-resolution generations must increase from generation 1 to generation $T$. The homogeneous 224-pixel chain must stay flat or drift toward texture, as the prior-convergence theory predicts. If both chains rise together, the effect comes from self-distillation, not from the resolution channel.
The second check addresses Mummadi et al. head-on. If the chain raises shape bias but leaves noise and weather corruptions unchanged, while improving only blur and pixelation, we will conclude that the chain is a frequency filter, not a source of general geometric robustness. We will report it that way.
The third check is the step cap. Removing $K$, so that students train to convergence, should erase the gain if the early-learning account from Shape or Texture is the mechanism.
The proposal is falsified if the single-round or blur-augmented baselines match the chain at equal compute.
References
- Iterated learning: Intergenerational knowledge transmission reveals inductive biases
- Connecting human and machine learning via probabilistic models of cognition
- Compositional languages emerge in a neural iterated learning model
- Ease-of-Teaching and Language Structure from Emergent Communication
- Emergent Communication: Generalization and Overfitting in Lewis Games
- Iterated learning for emergent systematicity in VQA
- Low-Resolution Object Recognition With Cross-Resolution Relational Contrastive Distillation
- Resolution Attack: Exploiting Image Compression to Deceive Deep Neural Networks
- Shape or Texture: Understanding Discriminative Features in CNNs
- Does enhanced shape bias improve neural network robustness to common corruptions?
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
- DCTMamba: Advancing JPEG Image Restoration Through Long-Sequence Modeling and Adaptive Frequency Strategy
- Benchmarking Neural Network Robustness to Common Corruptions and Perturbations