For over a decade, the recipe for high-performing computer vision has been simple: scrape millions of images from the internet, hire thousands of humans to label them, and train a massive neural network. However, this "ImageNet paradigm" is hitting a wall. Real-world datasets are increasingly mired in legal battles over copyright, ethical concerns regarding privacy, and the inescapable presence of human bias.
A growing body of research suggests a radical alternative: Formula-Driven Supervised Learning (FDSL). Instead of capturing the world through a lens, researchers are generating training data from pure mathematics. By training models on fractals, sinusoidal waves, and procedural contours, we can build vision systems that rival—and sometimes exceed—those trained on natural images.
The Intuition: Learning the Language of Shapes
Why would a model trained on abstract fractals understand a cat or a car? The intuition lies in the hierarchical nature of vision. Before a Vision Transformer (ViT) can identify an object, it must first learn to recognize edges, textures, and geometric primitives.
Hirokatsu Kataoka and colleagues argue that mathematical formulas can provide a "clean" and infinite source of these fundamental visual patterns. In Replacing Labeled Real-image Datasets with Auto-generated Contours, they demonstrate that models can learn robust representations simply by looking at procedurally generated shapes. These formulas act as a "visual alphabet," teaching the model the structural rules of the universe without the "noise" of specific real-world instances.
From Fractals to Sinusoidal Waves
The evolution of FDSL has moved from complex geometry to startlingly simple patterns:
- FractalDB: Early work by Ryosuke Yamada and Hirokatsu Kataoka used 3D fractal geometry to create FractalDB. Because fractals are self-similar and infinitely detailed, they provide a rich variety of textures that mimic the complexity of natural scenes.
- Visual Atoms: In Visual Atoms: Pre-Training Vision Transformers with Sinusoidal Waves, Sora Takashima and Rio Yokota pushed this further. They showed that pre-training on simple mathematical waves allows a ViT to achieve 83.7% accuracy on ImageNet-1k—nearly matching models trained on Google’s massive, private JFT-300M dataset.
- SegRCDB: To move beyond classification, Risa Shinoda developed SegRCDB, which uses formula-driven contours to train models for semantic segmentation. This proved that FDSL could handle dense, pixel-level tasks better than some manually labeled real-world datasets like COCO-Stuff.
Efficiency and Scaling
One of the most surprising discoveries in this field is how little data is actually needed. In Pre-training Vision Transformers with Very Limited Synthesized Images, Hirokatsu Kataoka and Nakamasa Inoue found that they could pre-train a ViT using just one synthetic image per category. By applying heavy data augmentation to a single fractal instance, they matched the performance of models trained on the millions of images in ImageNet-21k.
Furthermore, because these images are generated by formulas, they don't need to be stored on disk. Rio Yokota and Edgar Josafat Martinez-Noriega developed a GPU-accelerated pipeline that renders these images on-the-fly during training. This bypasses the I/O bottlenecks of traditional deep learning, leading to a 3x speedup in training throughput.
Solving the "Data Debt"
The shift toward FDSL is driven by more than just performance; it is a response to the "data debt" of modern AI. Real-world datasets often contain:
- Privacy Risks: Capturing people in classrooms or public spaces for activity recognition or surveillance is fraught with ethical peril.
- Societal Bias: Models trained on web-scraped data inherit human prejudices. Salma Abdel Magid and Hanspeter Pfister have shown that synthetic counterfactuals can be used to "unlearn" these biases by generating diverse versions of the same scene.
While some researchers use game engines or generative models to create synthetic data, FDSL represents the "cleanest" end of the spectrum. By stripping away the semantic world and focusing on the underlying mathematical structure of vision, FDSL offers a path toward AI that is high-performing, ethically sound, and mathematically grounded.