Iris-Z: Scaling One Generative Model Past Specialist Geometry Models

Alejandro Cañada, Chema GarabitoarXiv 2026 · arXiv

TL;DR

Iris-Z demonstrates that a single generative model can outperform specialized architectures in monocular depth estimation through scale and joint RGB-D training. By treating depth as just another image modality within a unified transformer backbone, the authors show that "the bitter lesson"—that general methods scaled with compute eventually win—applies to spatial intelligence as much as it does to language.

What Problem Does This Paper Address?

In the world of computer vision, we have traditionally relied on a "fragmented" approach: one specialized model for segmentation, another for depth estimation, and yet another for image generation. While Large Language Models (LLMs) have successfully unified diverse text tasks into a single generative framework, visual intelligence has remained largely task-specific.

The authors of Iris-Z challenge this status quo. They address the question: Can a general-purpose image generator learn the underlying geometry of the world well enough to beat "specialist" models designed solely for depth estimation? Specifically, they target monocular geometry—the difficult task of estimating 3D depth from a single 2D photograph.

Methods

Iris-Z is built upon the FLUX.2 family of Diffusion Transformers (DiT). The core innovation is the move to a unified RGB-D sequence. Instead of adding a "depth head" to a vision model, the authors treat depth as an image.

Key technical highlights include:

  • Shared Backbone: A single set of weights handles both appearance (RGB) and geometry (Depth). Both modalities are encoded into latents using the same VAE encoder.
  • Modality Embeddings: The model uses learned embeddings to distinguish between generated RGB, generated depth, and "given" (conditioning) inputs.
  • Joint Denoising: During training, the model learns to denoise image and depth latents together. Because the transformer's attention mechanism runs over the entire sequence, the image and depth tokens condition each other at every step.
  • Multi-Task Training: The model is trained on a mix of tasks: generating RGB-D from a prompt, estimating depth from an image, and even "painting" an image onto a provided depth map.
The Iris-Z architecture: a single DiT backbone processes a unified sequence of prompt, image, and depth tokens.

Datasets

The models were trained on a massive pool of 3 to 4 million unique RGB-D frames. This data includes indoor captures, driving scenes with LiDAR, stereo reconstructions, and synthetic renders.

For evaluation, the authors tested Iris-Z against two major benchmark protocols:

  1. Marigold’s Protocol: NYUv2, KITTI, ETH3D, ScanNet, and DIODE.
  2. MoGe-3’s Protocol: NYUv2, KITTI, ETH3D, iBims-1, Sintel, DIODE, and HAMMER.

Crucially, the authors did not train on any of these benchmark datasets, ensuring a true zero-shot evaluation of the model's generalization capabilities.

Key Insights and Ideas

  • The Bitter Lesson in Vision: The most significant finding is that architectural tweaks for specific tasks matter less than scale. As the authors increased parameter count (from 9B to 32B) and training data, the depth error dropped consistently without changing the underlying "generalist" recipe.
  • Geometry as a Generative Prior: A model that knows how to generate a realistic scene must inherently understand its geometry. By forcing the model to generate RGB and Depth jointly, it develops a more robust spatial understanding than a model trained only to regress depth values.
  • Preserving Generative Quality: Unlike previous attempts to repurpose generators, Iris-Z does not "forget" how to draw. It remains a high-quality text-to-image model while gaining state-of-the-art depth estimation skills.
Scaling behavior: Increasing model capacity and training data leads to a predictable and significant drop in depth error.

Key Results

Iris-Z-big (32B) currently sets a new bar for generalist models in depth estimation:

  • Benchmark Dominance: On Marigold’s benchmarks, Iris-Z-big achieved a mean AbsRel of 2.92, significantly better than the specialist MoGe-3 (3.12) and the previous generative leader Marigold V2 (4.10).
  • Generalization: On MoGe-3’s own protocol, Iris-Z-big outperformed MoGe-3 on six out of seven datasets, with a mean error of 3.56 vs. 3.70.
  • Efficiency of Scale: Even the smaller Iris-Z-small (9B) proved competitive, matching the performance of specialized state-of-the-art models.
Comparison against MoGe-3: Iris-Z-big shows superior performance across nearly all benchmark datasets.

Limitations and Future Work

Despite its success, Iris-Z has a few notable hurdles:

  • Relative vs. Metric Depth: The current model predicts relative depth (scale and shift invariant). It doesn't yet provide absolute metric distances (e.g., "this wall is exactly 3.2 meters away"). The authors plan to address this by adding camera intrinsic tokens to the sequence.
  • Computational Cost: Being a large generative model, Iris-Z is slow. One prediction takes about 15 seconds on an H200, compared to just 0.4 seconds for the specialist MoGe-3. The authors suggest that distillation techniques could eventually bridge this gap.
  • Resolution: The model is currently trained for resolutions up to ~1024x1024, leaving room for improvement in ultra-high-resolution 4K/8K scenarios.

The team at SperidLabs views depth as just the first step. They intend to add more modalities—like surface normals and segmentation—to the same unified sequence, moving closer to a single, all-encompassing model for visual intelligence.

Summary updated

Abstract

Language moved from many task-specific models to one unified model: a single generative model with a single generative training task that handles all of them within the same backbone, and keeps improving with data, parameters and compute. We believe the bitter lesson applies just as much to computer vision, 3D and spatial understanding, in the end to visual intelligence: the effort should go into general architectures that solve all these tasks in a unified way, and into scaling them, rather than into task-specific designs. We test this on monocular geometry, a task that has belonged to specialised architectures for years. Iris-Z is a general image generator that produces appearance and geometry together. A single backbone denoises, reasons about and understands the image and its depth at the same time, and one set of weights generates images with their depth, estimates depth from photographs and paints images onto a given depth. At enough scale it surpasses state-of-the-art specialists on established depth benchmarks while keeping the generative abilities of the backbone it started from without specialized layers. We release two models for the community.
BibTeX
@misc{canada2026iris,
  author = {Alejandro Cañada and Chema Garabito},
  title = {{Iris-Z: Scaling One Generative Model Past Specialist Geometry Models}},
  year = {2026},
  eprint = {2611.16010},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16010},
}
Figure